r332799 - [Clang Tablegen][RFC] Allow Early Textual Substitutions in `Diagnostic` messages.

Eric Fiselier via cfe-commits cfe-commits at lists.llvm.org
Fri May 18 20:12:04 PDT 2018


Author: ericwf
Date: Fri May 18 20:12:04 2018
New Revision: 332799

URL: http://llvm.org/viewvc/llvm-project?rev=332799&view=rev
Log:
[Clang Tablegen][RFC] Allow Early Textual Substitutions in `Diagnostic` messages.

Summary:
There are cases where the same string or select is repeated verbatim in a lot of diagnostics. This can be a pain to maintain and update. Tablegen provides no way stash the common text somewhere and reuse it in the diagnostics, until now!

This patch allows diagnostic texts to contain `%sub{<definition-name>}`, where `<definition-name>` names a Tablegen record of type `TextSubstitution`. These substitutions are done early, before the diagnostic string is otherwise processed. All `%sub` modifiers will be replaced before the diagnostic definitions are emitted.

The substitution must specify all arguments used by the substitution, and modifier indexes in the substitution are re-numbered accordingly. For example:

```
def select_ovl_candidate : TextSubstitution<"%select{function|constructor}0%select{| template| %2}1">;
```
when used as
```
"candidate `%sub{select_ovl_candidate}3,2,1 not viable"
```
will act as if we wrote:
```
"candidate %select{function|constructor}3%select{| template| %1}2 not viable"
```

Reviewers: rsmith, rjmccall, aaron.ballman, a.sidorin

Reviewed By: rjmccall

Subscribers: cfe-commits

Differential Revision: https://reviews.llvm.org/D46740

Added:
    cfe/trunk/test/TableGen/DiagnosticDocs.inc
    cfe/trunk/test/TableGen/emit-diag-docs.td
    cfe/trunk/test/TableGen/text-substitution.td
Modified:
    cfe/trunk/docs/InternalsManual.rst
    cfe/trunk/include/clang/Basic/Diagnostic.td
    cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td
    cfe/trunk/test/SemaCXX/anonymous-struct.cpp
    cfe/trunk/test/SemaCXX/cxx98-compat.cpp
    cfe/trunk/test/TableGen/DiagnosticBase.inc
    cfe/trunk/test/lit.cfg.py
    cfe/trunk/utils/TableGen/ClangDiagnosticsEmitter.cpp

Modified: cfe/trunk/docs/InternalsManual.rst
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/docs/InternalsManual.rst?rev=332799&r1=332798&r2=332799&view=diff
==============================================================================
--- cfe/trunk/docs/InternalsManual.rst (original)
+++ cfe/trunk/docs/InternalsManual.rst Fri May 18 20:12:04 2018
@@ -319,6 +319,32 @@ they should be discussed before they are
 repetitive diagnostics and/or have an idea for a useful formatter, please bring
 it up on the cfe-dev mailing list.
 
+**"sub" format**
+
+Example:
+  Given the following record definition of type ``TextSubstitution``:
+
+  .. code-block:: text
+
+    def select_ovl_candidate : TextSubstitution<
+      "%select{function|constructor}0%select{| template| %2}1">;
+
+  which can be used as
+
+  .. code-block:: text
+
+    def note_ovl_candidate : Note<
+      "candidate %sub{select_ovl_candidate}3,2,1 not viable">;
+
+  and will act as if it was written
+  ``"candidate %select{function|constructor}3%select{| template| %1}2 not viable"``.
+Description:
+  This format specifier is used to avoid repeating strings verbatim in multiple
+  diagnostics. The argument to ``%sub`` must name a ``TextSubstitution`` tblgen
+  record. The substitution must specify all arguments used by the substitution,
+  and the modifier indexes in the substitution are re-numbered accordingly. The
+  substituted text must itself be a valid format string before substitution.
+
 .. _internals-producing-diag:
 
 Producing the Diagnostic

Modified: cfe/trunk/include/clang/Basic/Diagnostic.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/Diagnostic.td?rev=332799&r1=332798&r2=332799&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/Diagnostic.td (original)
+++ cfe/trunk/include/clang/Basic/Diagnostic.td Fri May 18 20:12:04 2018
@@ -39,6 +39,15 @@ def SFINAE_Suppress            : SFINAER
 def SFINAE_Report              : SFINAEResponse;
 def SFINAE_AccessControl       : SFINAEResponse;
 
+// Textual substitutions which may be performed on the text of diagnostics
+class TextSubstitution<string Text> {
+  string Substitution = Text;
+  // TODO: These are only here to allow substitutions to be declared inline with
+  // diagnostics
+  string Component = "";
+  string CategoryName = "";
+}
+
 // Diagnostic Categories.  These can be applied to groups or individual
 // diagnostics to specify a category.
 class DiagCategory<string Name> {

Modified: cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td?rev=332799&r1=332798&r2=332799&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td (original)
+++ cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td Fri May 18 20:12:04 2018
@@ -1622,13 +1622,16 @@ def warn_call_to_pure_virtual_member_fun
   "overrides of %0 in subclasses are not available in the "
   "%select{constructor|destructor}1 of %2">;
 
+def select_special_member_kind : TextSubstitution<
+  "%select{default constructor|copy constructor|move constructor|"
+  "copy assignment operator|move assignment operator|destructor}0">;
+
 def note_member_declared_at : Note<"member is declared here">;
 def note_ivar_decl : Note<"instance variable is declared here">;
 def note_bitfield_decl : Note<"bit-field is declared here">;
 def note_implicit_param_decl : Note<"%0 is an implicit parameter">;
 def note_member_synthesized_at : Note<
-  "in implicit %select{default constructor|copy constructor|move constructor|"
-  "copy assignment operator|move assignment operator|destructor}0 for %1 "
+  "in implicit %sub{select_special_member_kind}0 for %1 "
   "first required here">;
 def err_missing_default_ctor : Error<
   "%select{constructor for %1 must explicitly initialize the|"
@@ -1641,12 +1644,10 @@ def note_due_to_dllexported_class : Note
 
 def err_illegal_union_or_anon_struct_member : Error<
   "%select{anonymous struct|union}0 member %1 has a non-trivial "
-  "%select{constructor|copy constructor|move constructor|copy assignment "
-  "operator|move assignment operator|destructor}2">;
+  "%sub{select_special_member_kind}2">;
 def warn_cxx98_compat_nontrivial_union_or_anon_struct_member : Warning<
   "%select{anonymous struct|union}0 member %1 with a non-trivial "
-  "%select{constructor|copy constructor|move constructor|copy assignment "
-  "operator|move assignment operator|destructor}2 is incompatible with C++98">,
+  "%sub{select_special_member_kind}2 is incompatible with C++98">,
   InGroup<CXX98Compat>, DefaultIgnore;
 
 def note_nontrivial_virtual_dtor : Note<
@@ -1665,8 +1666,7 @@ def note_nontrivial_no_copy : Note<
   "%select{base class|field|an object}0 of type %3">;
 def note_nontrivial_user_provided : Note<
   "because %select{base class of |field of |}0type %1 has a user-provided "
-  "%select{default constructor|copy constructor|move constructor|"
-  "copy assignment operator|move assignment operator|destructor}2">;
+  "%sub{select_special_member_kind}2">;
 def note_nontrivial_in_class_init : Note<
   "because field %0 has an initializer">;
 def note_nontrivial_param_type : Note<
@@ -1733,9 +1733,7 @@ def err_covariant_return_type_class_type
 
 // C++ implicit special member functions
 def note_in_declaration_of_implicit_special_member : Note<
-  "while declaring the implicit "
-  "%select{default constructor|copy constructor|move constructor|"
-  "copy assignment operator|move assignment operator|destructor}1"
+  "while declaring the implicit %sub{select_special_member_kind}1"
   " for %0">;
 
 // C++ constructors
@@ -3837,13 +3835,7 @@ def note_ovl_candidate_bad_target : Note
     "%select{__device__|__global__|__host__|__host__ __device__|invalid}1 function from"
     " %select{__device__|__global__|__host__|__host__ __device__|invalid}2 function">;
 def note_implicit_member_target_infer_collision : Note<
-    "implicit %select{"
-    "default constructor|"
-    "copy constructor|"
-    "move constructor|"
-    "copy assignment operator|"
-    "move assignment operator|"
-    "destructor}0 inferred target collision: call to both "
+    "implicit %sub{select_special_member_kind}0 inferred target collision: call to both "
     "%select{__device__|__global__|__host__|__host__ __device__}1 and "
     "%select{__device__|__global__|__host__|__host__ __device__}2 members">;
 
@@ -3885,9 +3877,7 @@ def err_ovl_deleted_oper : Error<
   "overload resolution selected %select{unavailable|deleted}0 operator '%1'%2">;
 def err_ovl_deleted_special_oper : Error<
   "object of type %0 cannot be %select{constructed|copied|moved|assigned|"
-  "assigned|destroyed}1 because its %select{default constructor|"
-  "copy constructor|move constructor|copy assignment operator|"
-  "move assignment operator|destructor}1 is implicitly deleted">;
+  "assigned|destroyed}1 because its %sub{select_special_member_kind}1 is implicitly deleted">;
 def err_ovl_no_viable_subscript :
     Error<"no viable overloaded operator[] for type %0">;
 def err_ovl_no_oper :
@@ -7767,9 +7757,8 @@ def err_defaulted_special_member_quals :
   "an explicitly-defaulted %select{copy|move}0 assignment operator may not "
   "have 'const'%select{, 'constexpr'|}1 or 'volatile' qualifiers">;
 def err_defaulted_special_member_volatile_param : Error<
-  "the parameter for an explicitly-defaulted %select{<<ERROR>>|"
-  "copy constructor|move constructor|copy assignment operator|"
-  "move assignment operator|<<ERROR>>}0 may not be volatile">;
+  "the parameter for an explicitly-defaulted %sub{select_special_member_kind}0 "
+  "may not be volatile">;
 def err_defaulted_special_member_move_const_param : Error<
   "the parameter for an explicitly-defaulted move "
   "%select{constructor|assignment operator}0 may not be const">;
@@ -7781,17 +7770,13 @@ def err_defaulted_copy_assign_not_ref :
   "the parameter for an explicitly-defaulted copy assignment operator must be an "
   "lvalue reference type">;
 def err_incorrect_defaulted_exception_spec : Error<
-  "exception specification of explicitly defaulted %select{default constructor|"
-  "copy constructor|move constructor|copy assignment operator|move assignment "
-  "operator|destructor}0 does not match the "
-  "calculated one">;
+  "exception specification of explicitly defaulted "
+  "%sub{select_special_member_kind}0 does not match the calculated one">;
 def err_incorrect_defaulted_constexpr : Error<
-  "defaulted definition of %select{default constructor|copy constructor|"
-  "move constructor|copy assignment operator|move assignment operator}0 "
+  "defaulted definition of %sub{select_special_member_kind}0 "
   "is not constexpr">;
 def err_out_of_line_default_deletes : Error<
-  "defaulting this %select{default constructor|copy constructor|move "
-  "constructor|copy assignment operator|move assignment operator|destructor}0 "
+  "defaulting this %sub{select_special_member_kind}0 "
   "would delete it after its first declaration">;
 def warn_vbase_moved_multiple_times : Warning<
   "defaulted move assignment operator of %0 will move assign virtual base "

Modified: cfe/trunk/test/SemaCXX/anonymous-struct.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/SemaCXX/anonymous-struct.cpp?rev=332799&r1=332798&r2=332799&view=diff
==============================================================================
--- cfe/trunk/test/SemaCXX/anonymous-struct.cpp (original)
+++ cfe/trunk/test/SemaCXX/anonymous-struct.cpp Fri May 18 20:12:04 2018
@@ -16,7 +16,7 @@ struct E {
   struct {
     S x;
 #if __cplusplus <= 199711L
-    // expected-error at -2 {{anonymous struct member 'x' has a non-trivial constructor}}
+    // expected-error at -2 {{anonymous struct member 'x' has a non-trivial default constructor}}
 #endif
   };
   static struct {

Modified: cfe/trunk/test/SemaCXX/cxx98-compat.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/SemaCXX/cxx98-compat.cpp?rev=332799&r1=332798&r2=332799&view=diff
==============================================================================
--- cfe/trunk/test/SemaCXX/cxx98-compat.cpp (original)
+++ cfe/trunk/test/SemaCXX/cxx98-compat.cpp Fri May 18 20:12:04 2018
@@ -241,13 +241,13 @@ namespace UnionOrAnonStructMembers {
     ~NonTrivDtor(); // expected-note 2{{user-provided destructor}}
   };
   union BadUnion {
-    NonTrivCtor ntc; // expected-warning {{union member 'ntc' with a non-trivial constructor is incompatible with C++98}}
+    NonTrivCtor ntc; // expected-warning {{union member 'ntc' with a non-trivial default constructor is incompatible with C++98}}
     NonTrivCopy ntcp; // expected-warning {{union member 'ntcp' with a non-trivial copy constructor is incompatible with C++98}}
     NonTrivDtor ntd; // expected-warning {{union member 'ntd' with a non-trivial destructor is incompatible with C++98}}
   };
   struct Wrap {
     struct {
-      NonTrivCtor ntc; // expected-warning {{anonymous struct member 'ntc' with a non-trivial constructor is incompatible with C++98}}
+      NonTrivCtor ntc; // expected-warning {{anonymous struct member 'ntc' with a non-trivial default constructor is incompatible with C++98}}
       NonTrivCopy ntcp; // expected-warning {{anonymous struct member 'ntcp' with a non-trivial copy constructor is incompatible with C++98}}
       NonTrivDtor ntd; // expected-warning {{anonymous struct member 'ntd' with a non-trivial destructor is incompatible with C++98}}
     };
@@ -348,7 +348,7 @@ namespace rdar11736429 {
   };
 
   union S {
-    X x; // expected-warning{{union member 'x' with a non-trivial constructor is incompatible with C++98}}
+    X x; // expected-warning{{union member 'x' with a non-trivial default constructor is incompatible with C++98}}
   };
 }
 

Modified: cfe/trunk/test/TableGen/DiagnosticBase.inc
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/TableGen/DiagnosticBase.inc?rev=332799&r1=332798&r2=332799&view=diff
==============================================================================
--- cfe/trunk/test/TableGen/DiagnosticBase.inc (original)
+++ cfe/trunk/test/TableGen/DiagnosticBase.inc Fri May 18 20:12:04 2018
@@ -1,35 +1,130 @@
-// Define the diagnostic mappings.
-class DiagMapping;
-def MAP_IGNORE  : DiagMapping;
-def MAP_WARNING : DiagMapping;
-def MAP_ERROR   : DiagMapping;
-def MAP_FATAL   : DiagMapping;
+//===--- DiagnosticBase.inc - A test file mimicking Diagnostic.td ---------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+//  This file defines the TableGen core definitions for the diagnostics
+//  and diagnostic control.
+//
+//===----------------------------------------------------------------------===//
+
+// See the Internals Manual, section The Diagnostics Subsystem for an overview.
+
+// Define the diagnostic severities.
+class Severity<string N> {
+  string Name = N;
+}
+def SEV_Ignored : Severity<"Ignored">;
+def SEV_Remark  : Severity<"Remark">;
+def SEV_Warning : Severity<"Warning">;
+def SEV_Error   : Severity<"Error">;
+def SEV_Fatal   : Severity<"Fatal">;
 
 // Define the diagnostic classes.
 class DiagClass;
 def CLASS_NOTE      : DiagClass;
+def CLASS_REMARK    : DiagClass;
 def CLASS_WARNING   : DiagClass;
 def CLASS_EXTENSION : DiagClass;
 def CLASS_ERROR     : DiagClass;
 
+// Responses to a diagnostic in a SFINAE context.
+class SFINAEResponse;
+def SFINAE_SubstitutionFailure : SFINAEResponse;
+def SFINAE_Suppress            : SFINAEResponse;
+def SFINAE_Report              : SFINAEResponse;
+def SFINAE_AccessControl       : SFINAEResponse;
+
+// Textual substitutions which may be performed on the text of diagnostics
+class TextSubstitution<string Text> {
+  string Substitution = Text;
+  // TODO: These are only here to allow substitutions to be declared inline with
+  // diagnostics
+  string Component = "";
+  string CategoryName = "";
+}
+
+// Diagnostic Categories.  These can be applied to groups or individual
+// diagnostics to specify a category.
+class DiagCategory<string Name> {
+  string CategoryName = Name;
+}
+
+// Diagnostic Groups.
 class DiagGroup<string Name, list<DiagGroup> subgroups = []> {
   string GroupName = Name;
   list<DiagGroup> SubGroups = subgroups;
   string CategoryName = "";
+  code Documentation = [{}];
 }
 class InGroup<DiagGroup G> { DiagGroup Group = G; }
+//class IsGroup<string Name> { DiagGroup Group = DiagGroup<Name>; }
+
+include "DiagnosticDocs.inc"
 
 // All diagnostics emitted by the compiler are an indirect subclass of this.
-class Diagnostic<string text, DiagClass DC, DiagMapping defaultmapping> {
-  string      Text = text;
-  DiagClass   Class = DC;
-  DiagMapping DefaultMapping = defaultmapping;
-  DiagGroup   Group;
-  string      CategoryName = "";
+class Diagnostic<string text, DiagClass DC, Severity defaultmapping> {
+  /// Component is specified by the file with a big let directive.
+  string         Component = ?;
+  string         Text = text;
+  DiagClass      Class = DC;
+  SFINAEResponse SFINAE = SFINAE_Suppress;
+  bit            AccessControl = 0;
+  bit            WarningNoWerror = 0;
+  bit            ShowInSystemHeader = 0;
+  Severity       DefaultSeverity = defaultmapping;
+  DiagGroup      Group;
+  string         CategoryName = "";
+}
+
+class SFINAEFailure {
+  SFINAEResponse SFINAE = SFINAE_SubstitutionFailure;
+}
+class NoSFINAE {
+  SFINAEResponse SFINAE = SFINAE_Report;
+}
+class AccessControl {
+  SFINAEResponse SFINAE = SFINAE_AccessControl;
 }
 
-class Error<string str>     : Diagnostic<str, CLASS_ERROR, MAP_ERROR>;
-class Warning<string str>   : Diagnostic<str, CLASS_WARNING, MAP_WARNING>;
-class Extension<string str> : Diagnostic<str, CLASS_EXTENSION, MAP_IGNORE>;
-class ExtWarn<string str>   : Diagnostic<str, CLASS_EXTENSION, MAP_WARNING>;
-class Note<string str>      : Diagnostic<str, CLASS_NOTE, MAP_FATAL/*ignored*/>;
+class ShowInSystemHeader {
+  bit ShowInSystemHeader = 1;
+}
+
+class SuppressInSystemHeader {
+  bit ShowInSystemHeader = 0;
+}
+
+// FIXME: ExtWarn and Extension should also be SFINAEFailure by default.
+class Error<string str>     : Diagnostic<str, CLASS_ERROR, SEV_Error>, SFINAEFailure {
+  bit ShowInSystemHeader = 1;
+}
+// Warnings default to on (but can be default-off'd with DefaultIgnore).
+// This is used for warnings about questionable code; warnings about
+// accepted language extensions should use Extension or ExtWarn below instead.
+class Warning<string str>   : Diagnostic<str, CLASS_WARNING, SEV_Warning>;
+// Remarks can be turned on with -R flags and provide commentary, e.g. on
+// optimizer decisions.
+class Remark<string str>    : Diagnostic<str, CLASS_REMARK, SEV_Ignored>;
+// Extensions are warnings about accepted language extensions.
+// Extension warnings are default-off but enabled by -pedantic.
+class Extension<string str> : Diagnostic<str, CLASS_EXTENSION, SEV_Ignored>;
+// ExtWarns are warnings about accepted language extensions.
+// ExtWarn warnings are default-on.
+class ExtWarn<string str>   : Diagnostic<str, CLASS_EXTENSION, SEV_Warning>;
+// Notes can provide supplementary information on errors, warnings, and remarks.
+class Note<string str>      : Diagnostic<str, CLASS_NOTE, SEV_Fatal/*ignored*/>;
+
+
+class DefaultIgnore { Severity DefaultSeverity = SEV_Ignored; }
+class DefaultWarn   { Severity DefaultSeverity = SEV_Warning; }
+class DefaultError  { Severity DefaultSeverity = SEV_Error; }
+class DefaultFatal  { Severity DefaultSeverity = SEV_Fatal; }
+class DefaultWarnNoWerror {
+  bit WarningNoWerror = 1;
+}
+class DefaultRemark { Severity DefaultSeverity = SEV_Remark; }

Added: cfe/trunk/test/TableGen/DiagnosticDocs.inc
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/TableGen/DiagnosticDocs.inc?rev=332799&view=auto
==============================================================================
--- cfe/trunk/test/TableGen/DiagnosticDocs.inc (added)
+++ cfe/trunk/test/TableGen/DiagnosticDocs.inc Fri May 18 20:12:04 2018
@@ -0,0 +1,75 @@
+
+def GlobalDocumentation {
+  code Intro =[{..
+  -------------------------------------------------------------------
+  NOTE: This file is automatically generated by running clang-tblgen
+  -gen-diag-docs. Do not edit this file by hand!!
+  -------------------------------------------------------------------
+
+.. Add custom CSS to output. FIXME: This should be put into <head> rather
+   than the start of <body>.
+.. raw:: html
+
+    <style>
+    table.docutils {
+      width: 1px;
+    }
+    table.docutils td {
+      border: none;
+      padding: 0 0 0 0.2em;
+      vertical-align: middle;
+      white-space: nowrap;
+      width: 1px;
+      font-family: monospace;
+    }
+    table.docutils tr + tr {
+      border-top: 0.2em solid #aaa;
+    }
+    .error {
+      font-family: monospace;
+      font-weight: bold;
+      color: #c00;
+    }
+    .warning {
+      font-family: monospace;
+      font-weight: bold;
+      color: #80a;
+    }
+    .remark {
+      font-family: monospace;
+      font-weight: bold;
+      color: #00c;
+    }
+    .diagtext {
+      font-family: monospace;
+      font-weight: bold;
+    }
+    </style>
+
+.. FIXME: rST doesn't support formatting this, so we format all <td> elements
+          as monospace font face instead.
+.. |nbsp| unicode:: 0xA0
+   :trim:
+
+.. Roles generated by clang-tblgen.
+.. role:: error
+.. role:: warning
+.. role:: remark
+.. role:: diagtext
+.. role:: placeholder(emphasis)
+
+=========================
+Diagnostic flags in Clang
+=========================
+.. contents::
+   :local:
+
+Introduction
+============
+
+This page lists the diagnostic flags currently supported by Clang.
+
+Diagnostic flags
+================
+}];
+}

Added: cfe/trunk/test/TableGen/emit-diag-docs.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/TableGen/emit-diag-docs.td?rev=332799&view=auto
==============================================================================
--- cfe/trunk/test/TableGen/emit-diag-docs.td (added)
+++ cfe/trunk/test/TableGen/emit-diag-docs.td Fri May 18 20:12:04 2018
@@ -0,0 +1,78 @@
+// RUN: clang-tblgen -gen-diag-docs -I%S %s -o - 2>&1 | \
+// RUN:    FileCheck --strict-whitespace %s
+include "DiagnosticBase.inc"
+
+def MyGroup : DiagGroup<"MyGroupName">;
+
+def MyKinds : TextSubstitution<"%select{food|forests}0">;
+def MyGoodBad : TextSubstitution<"%select{good|bad}0">;
+def MySubNested : TextSubstitution<"%sub{MyGoodBad}1 %sub{MyKinds}2 are %sub{MyGoodBad}1 according to %0">;
+
+// CHECK: -WMyGroupName
+// CHECK: **Diagnostic text:**
+
+let Group = MyGroup in {
+
+// CHECK:      |:warning:`warning:` |nbsp| :diagtext:`this is my diff text`|
+// CHECK-NEXT: +-----------------------------------------------------------+
+def CheckDiff : Warning<"%diff{$ is not $|this is my diff text}0,1">;
+
+
+// CHECK:      |:warning:`warning:` |nbsp| :placeholder:`A` |nbsp| :diagtext:`is my modifier test` |nbsp| :placeholder:`B`|
+// CHECK-NEXT: +----------------------------------------------------------------------------------------------------------+
+def CheckModifier : Warning<"%0 is my modifier test %1">;
+
+
+// CHECK:      |:warning:`warning:` |nbsp| :diagtext:`This is the` |nbsp| :placeholder:`A` |nbsp| :diagtext:`test I've written`|
+// CHECK-NEXT: +---------------------------------------------------------------------------------------------------------------+
+def CheckOrdinal : Warning<"This is the %ordinal0 test I've written">;
+
+// CHECK:      |:warning:`warning:` |nbsp| :diagtext:`I wrote` |nbsp| |+----------------+| |nbsp| :diagtext:`tests`|
+// CHECK-NEXT: |                                                      ||:diagtext:`no`  ||                         |
+// CHECK-NEXT: |                                                      |+----------------+|                         |
+// CHECK-NEXT: |                                                      ||:diagtext:`one` ||                         |
+// CHECK-NEXT: |                                                      |+----------------+|                         |
+// CHECK-NEXT: |                                                      ||:placeholder:`A`||                         |
+// CHECK-NEXT: |                                                      |+----------------+|                         |
+// CHECK-NEXT: +------------------------------------------------------+------------------+-------------------------+
+def CheckPlural : Warning<"I wrote %plural{0:no|1:one|:%0}0 tests">;
+
+
+// CHECK:      |:warning:`warning:` |nbsp| :diagtext:`bad type` |nbsp| :placeholder:`A`|
+// CHECK-NEXT: +-----------------------------------------------------------------------+
+def CheckQ : Warning<"bad type %q0">;
+
+
+// CHECK:      |:warning:`warning:` |nbsp| :diagtext:`My test`|+-------------+| |nbsp| :diagtext:`are the best!`|
+// CHECK-NEXT: |                                              ||             ||                                 |
+// CHECK-NEXT: |                                              |+-------------+|                                 |
+// CHECK-NEXT: |                                              ||:diagtext:`s`||                                 |
+// CHECK-NEXT: |                                              |+-------------+|                                 |
+// CHECK-NEXT: +----------------------------------------------+---------------+---------------------------------+
+def CheckS : Warning<"My test%s0 are the best!">;
+
+
+// CHECK:      |:warning:`warning:` |nbsp| :diagtext:`this is my select test:` |nbsp| |+---------------+|
+// CHECK-NEXT: |                                                                      ||:diagtext:`one`||
+// CHECK-NEXT: |                                                                      |+---------------+|
+// CHECK-NEXT: |                                                                      ||:diagtext:`two`||
+// CHECK-NEXT: |                                                                      |+---------------+|
+// CHECK-NEXT: +----------------------------------------------------------------------+-----------------+
+def CheckSelect : Warning<"this is my select test: %select{one|two}0 and it is %select{good|bad}1">;
+
+
+// CHECK:      +-------------------------------------------------------+------------------+--------+---------------------+-------------------------------+------------------+--------------------------------------------------------+
+// CHECK-NEXT: |:warning:`warning:` |nbsp| :diagtext:`They say` |nbsp| |+----------------+| |nbsp| |+-------------------+| |nbsp| :diagtext:`are` |nbsp| |+----------------+| |nbsp| :diagtext:`according to` |nbsp| :placeholder:`D`|
+// CHECK-NEXT: |                                                       ||:diagtext:`good`||        ||:diagtext:`food`   ||                               ||:diagtext:`good`||                                                        |
+// CHECK-NEXT: |                                                       |+----------------+|        |+-------------------+|                               |+----------------+|                                                        |
+// CHECK-NEXT: |                                                       ||:diagtext:`bad` ||        ||:diagtext:`forests`||                               ||:diagtext:`bad` ||                                                        |
+// CHECK-NEXT: |                                                       |+----------------+|        |+-------------------+|                               |+----------------+|                                                        |
+// CHECK-NEXT: +-------------------------------------------------------+------------------+--------+---------------------+-------------------------------+------------------+--------------------------------------------------------+
+def CheckSubstitution : Warning<"They say %sub{MySubNested}3,1,0">;
+
+
+// CHECK:      |:warning:`warning:` |nbsp| :diagtext:`this is my warning text`|
+// CHECK-NEXT: +--------------------------------------------------------------+
+def CheckText : Warning<"this is my warning text">;
+
+}

Added: cfe/trunk/test/TableGen/text-substitution.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/TableGen/text-substitution.td?rev=332799&view=auto
==============================================================================
--- cfe/trunk/test/TableGen/text-substitution.td (added)
+++ cfe/trunk/test/TableGen/text-substitution.td Fri May 18 20:12:04 2018
@@ -0,0 +1,38 @@
+// RUN: clang-tblgen -gen-clang-diags-defs -I%S %s -o - 2>&1 | \
+// RUN:    FileCheck --strict-whitespace %s
+include "DiagnosticBase.inc"
+
+def yes_no : TextSubstitution<"%select{yes|no}0">;
+def says_yes : TextSubstitution<"%1 says %sub{yes_no}0">;
+
+
+def sub_test_rewrite : TextSubstitution<
+  "SELECT! %select{one|two}3. "
+  "DIFF! %diff{$ is $|or not}0,1. "
+  "PLURAL! %plural{0:zero items|[1,2]:one or two item|:multiple items}2. "
+  "ORDINAL! %ordinal1. "
+  "S! item%s2. "
+  "Q! %q4. "
+  "PLACEHOLDER! %5."
+  "OBJCCLASS! %objcclass0. "
+  "OBJCINSTANCE! %objcinstance1. ">;
+
+// CHECK: DIAG(test_rewrite,
+// CHECK-SAME: SELECT! %select{one|two}2.
+// CHECK-SAME: DIFF! %diff{$ is $|or not}5,4.
+// CHECK-SAME: PLURAL! %plural{0:zero items|[1,2]:one or two item|:multiple items}3.
+// CHECK-SAME: ORDINAL! %ordinal4.
+// CHECK-SAME: S! item%s3.
+// CHECK-SAME: Q! %q1.
+// CHECK-SAME: PLACEHOLDER! %0.OBJCCLASS!
+// CHECK-SAME: %objcclass5. OBJCINSTANCE!
+// CHECK-SAME: %objcinstance4.  DONE!",
+def test_rewrite: Error<"%sub{sub_test_rewrite}5,4,3,2,1,0 DONE!">;
+
+def test_sub_basic : Error<"%sub{yes_no}0">;
+// CHECK: test_sub_basic
+// CHECK-SAME: "%select{yes|no}0",
+
+def test_sub_nested : Error<"%sub{says_yes}2,4">;
+// CHECK: test_sub_nested
+// CHECK-SAME: "%4 says %select{yes|no}2",

Modified: cfe/trunk/test/lit.cfg.py
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/lit.cfg.py?rev=332799&r1=332798&r2=332799&view=diff
==============================================================================
--- cfe/trunk/test/lit.cfg.py (original)
+++ cfe/trunk/test/lit.cfg.py Fri May 18 20:12:04 2018
@@ -57,7 +57,8 @@ config.substitutions.append(('%PATH%', c
 tool_dirs = [config.clang_tools_dir, config.llvm_tools_dir]
 
 tools = [
-    'c-index-test', 'clang-check', 'clang-diff', 'clang-format', 'opt',
+    'c-index-test', 'clang-check', 'clang-diff', 'clang-format', 'clang-tblgen',
+    'opt',
     ToolSubst('%clang_func_map', command=FindTool(
         'clang-func-mapping'), unresolved='ignore'),
 ]

Modified: cfe/trunk/utils/TableGen/ClangDiagnosticsEmitter.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/utils/TableGen/ClangDiagnosticsEmitter.cpp?rev=332799&r1=332798&r2=332799&view=diff
==============================================================================
--- cfe/trunk/utils/TableGen/ClangDiagnosticsEmitter.cpp (original)
+++ cfe/trunk/utils/TableGen/ClangDiagnosticsEmitter.cpp Fri May 18 20:12:04 2018
@@ -14,12 +14,13 @@
 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/PointerUnion.h"
+#include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringMap.h"
-#include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/Twine.h"
+#include "llvm/Support/Casting.h"
 #include "llvm/TableGen/Error.h"
 #include "llvm/TableGen/Record.h"
 #include "llvm/TableGen/StringToOffsetTable.h"
@@ -441,6 +442,733 @@ void InferPedantic::compute(VecOrSet Dia
   }
 }
 
+namespace {
+enum PieceKind {
+  MultiPieceClass,
+  TextPieceClass,
+  PlaceholderPieceClass,
+  SelectPieceClass,
+  PluralPieceClass,
+  DiffPieceClass,
+  SubstitutionPieceClass,
+};
+
+enum ModifierType {
+  MT_Unknown,
+  MT_Placeholder,
+  MT_Select,
+  MT_Sub,
+  MT_Plural,
+  MT_Diff,
+  MT_Ordinal,
+  MT_S,
+  MT_Q,
+  MT_ObjCClass,
+  MT_ObjCInstance,
+};
+
+static StringRef getModifierName(ModifierType MT) {
+  switch (MT) {
+  case MT_Select:
+    return "select";
+  case MT_Sub:
+    return "sub";
+  case MT_Diff:
+    return "diff";
+  case MT_Plural:
+    return "plural";
+  case MT_Ordinal:
+    return "ordinal";
+  case MT_S:
+    return "s";
+  case MT_Q:
+    return "q";
+  case MT_Placeholder:
+    return "";
+  case MT_ObjCClass:
+    return "objcclass";
+  case MT_ObjCInstance:
+    return "objcinstance";
+  case MT_Unknown:
+    llvm_unreachable("invalid modifier type");
+  }
+}
+
+struct Piece {
+  // This type and its derived classes are move-only.
+  Piece(PieceKind Kind) : ClassKind(Kind) {}
+  Piece(Piece const &O) = delete;
+  Piece &operator=(Piece const &) = delete;
+  virtual ~Piece() {}
+
+  PieceKind getPieceClass() const { return ClassKind; }
+  static bool classof(const Piece *) { return true; }
+
+private:
+  PieceKind ClassKind;
+};
+
+struct MultiPiece : Piece {
+  MultiPiece() : Piece(MultiPieceClass) {}
+  MultiPiece(std::vector<Piece *> Pieces)
+      : Piece(MultiPieceClass), Pieces(std::move(Pieces)) {}
+
+  std::vector<Piece *> Pieces;
+
+  static bool classof(const Piece *P) {
+    return P->getPieceClass() == MultiPieceClass;
+  }
+};
+
+struct TextPiece : Piece {
+  StringRef Role;
+  std::string Text;
+  TextPiece(StringRef Text, StringRef Role = "")
+      : Piece(TextPieceClass), Role(Role), Text(Text.str()) {}
+
+  static bool classof(const Piece *P) {
+    return P->getPieceClass() == TextPieceClass;
+  }
+};
+
+struct PlaceholderPiece : Piece {
+  ModifierType Kind;
+  int Index;
+  PlaceholderPiece(ModifierType Kind, int Index)
+      : Piece(PlaceholderPieceClass), Kind(Kind), Index(Index) {}
+
+  static bool classof(const Piece *P) {
+    return P->getPieceClass() == PlaceholderPieceClass;
+  }
+};
+
+struct SelectPiece : Piece {
+protected:
+  SelectPiece(PieceKind Kind, ModifierType ModKind)
+      : Piece(Kind), ModKind(ModKind) {}
+
+public:
+  SelectPiece(ModifierType ModKind) : SelectPiece(SelectPieceClass, ModKind) {}
+
+  ModifierType ModKind;
+  std::vector<Piece *> Options;
+  int Index;
+
+  static bool classof(const Piece *P) {
+    return P->getPieceClass() == SelectPieceClass ||
+           P->getPieceClass() == PluralPieceClass;
+  }
+};
+
+struct PluralPiece : SelectPiece {
+  PluralPiece() : SelectPiece(PluralPieceClass, MT_Plural) {}
+
+  std::vector<Piece *> OptionPrefixes;
+  int Index;
+
+  static bool classof(const Piece *P) {
+    return P->getPieceClass() == PluralPieceClass;
+  }
+};
+
+struct DiffPiece : Piece {
+  DiffPiece() : Piece(DiffPieceClass) {}
+
+  Piece *Options[2] = {};
+  int Indexes[2] = {};
+
+  static bool classof(const Piece *P) {
+    return P->getPieceClass() == DiffPieceClass;
+  }
+};
+
+struct SubstitutionPiece : Piece {
+  SubstitutionPiece() : Piece(SubstitutionPieceClass) {}
+
+  std::string Name;
+  std::vector<int> Modifiers;
+
+  static bool classof(const Piece *P) {
+    return P->getPieceClass() == SubstitutionPieceClass;
+  }
+};
+
+/// Diagnostic text, parsed into pieces.
+
+
+struct DiagnosticTextBuilder {
+  DiagnosticTextBuilder(DiagnosticTextBuilder const &) = delete;
+  DiagnosticTextBuilder &operator=(DiagnosticTextBuilder const &) = delete;
+
+  DiagnosticTextBuilder(RecordKeeper &Records) {
+    // Build up the list of substitution records.
+    for (auto *S : Records.getAllDerivedDefinitions("TextSubstitution")) {
+      EvaluatingRecordGuard Guard(&EvaluatingRecord, S);
+      Substitutions.try_emplace(
+          S->getName(), DiagText(*this, S->getValueAsString("Substitution")));
+    }
+
+    // Check that no diagnostic definitions have the same name as a
+    // substitution.
+    for (Record *Diag : Records.getAllDerivedDefinitions("Diagnostic")) {
+      StringRef Name = Diag->getName();
+      if (Substitutions.count(Name))
+        llvm::PrintFatalError(
+            Diag->getLoc(),
+            "Diagnostic '" + Name +
+                "' has same name as TextSubstitution definition");
+    }
+  }
+
+  std::vector<std::string> buildForDocumentation(StringRef Role,
+                                                 const Record *R);
+  std::string buildForDefinition(const Record *R);
+
+  Piece *getSubstitution(SubstitutionPiece *S) const {
+    auto It = Substitutions.find(S->Name);
+    if (It == Substitutions.end())
+      PrintFatalError("Failed to find substitution with name: " + S->Name);
+    return It->second.Root;
+  }
+
+  void PrintFatalError(llvm::Twine const &Msg) const {
+    assert(EvaluatingRecord && "not evaluating a record?");
+    llvm::PrintFatalError(EvaluatingRecord->getLoc(), Msg);
+  }
+
+private:
+  struct DiagText {
+    DiagnosticTextBuilder &Builder;
+    std::vector<Piece *> AllocatedPieces;
+    Piece *Root = nullptr;
+
+    template <class T, class... Args> T *New(Args &&... args) {
+      static_assert(std::is_base_of<Piece, T>::value, "must be piece");
+      T *Mem = new T(std::forward<Args>(args)...);
+      AllocatedPieces.push_back(Mem);
+      return Mem;
+    }
+
+    DiagText(DiagnosticTextBuilder &Builder, StringRef Text)
+        : Builder(Builder), Root(parseDiagText(Text)) {}
+
+    Piece *parseDiagText(StringRef &Text, bool Nested = false);
+    int parseModifier(StringRef &) const;
+
+  public:
+    DiagText(DiagText &&O) noexcept
+        : Builder(O.Builder), AllocatedPieces(std::move(O.AllocatedPieces)),
+          Root(O.Root) {
+      O.Root = nullptr;
+    }
+
+    ~DiagText() {
+      for (Piece *P : AllocatedPieces)
+        delete P;
+    }
+  };
+
+private:
+  const Record *EvaluatingRecord = nullptr;
+  struct EvaluatingRecordGuard {
+    EvaluatingRecordGuard(const Record **Dest, const Record *New)
+        : Dest(Dest), Old(*Dest) {
+      *Dest = New;
+    }
+    ~EvaluatingRecordGuard() { *Dest = Old; }
+    const Record **Dest;
+    const Record *Old;
+  };
+
+  StringMap<DiagText> Substitutions;
+};
+
+template <class Derived> struct DiagTextVisitor {
+  using ModifierMappingsType = Optional<std::vector<int>>;
+
+private:
+  Derived &getDerived() { return static_cast<Derived &>(*this); }
+
+public:
+  std::vector<int>
+  getSubstitutionMappings(SubstitutionPiece *P,
+                          const ModifierMappingsType &Mappings) const {
+    std::vector<int> NewMappings;
+    for (int Idx : P->Modifiers)
+      NewMappings.push_back(mapIndex(Idx, Mappings));
+    return NewMappings;
+  }
+
+  struct SubstitutionContext {
+    SubstitutionContext(DiagTextVisitor &Visitor, SubstitutionPiece *P)
+        : Visitor(Visitor) {
+      Substitution = Visitor.Builder.getSubstitution(P);
+      OldMappings = std::move(Visitor.ModifierMappings);
+      std::vector<int> NewMappings =
+          Visitor.getSubstitutionMappings(P, OldMappings);
+      Visitor.ModifierMappings = std::move(NewMappings);
+    }
+
+    ~SubstitutionContext() {
+      Visitor.ModifierMappings = std::move(OldMappings);
+    }
+
+  private:
+    DiagTextVisitor &Visitor;
+    Optional<std::vector<int>> OldMappings;
+
+  public:
+    Piece *Substitution;
+  };
+
+public:
+  DiagTextVisitor(DiagnosticTextBuilder &Builder) : Builder(Builder) {}
+
+  void Visit(Piece *P) {
+    switch (P->getPieceClass()) {
+#define CASE(T)                                                                \
+  case T##PieceClass:                                                          \
+    return getDerived().Visit##T(static_cast<T##Piece *>(P))
+      CASE(Multi);
+      CASE(Text);
+      CASE(Placeholder);
+      CASE(Select);
+      CASE(Plural);
+      CASE(Diff);
+      CASE(Substitution);
+#undef CASE
+    }
+  }
+
+  void VisitSubstitution(SubstitutionPiece *P) {
+    SubstitutionContext Guard(*this, P);
+    Visit(Guard.Substitution);
+  }
+
+  int mapIndex(int Idx,
+                    ModifierMappingsType const &ModifierMappings) const {
+    if (!ModifierMappings)
+      return Idx;
+    if (ModifierMappings->size() <= static_cast<unsigned>(Idx))
+      Builder.PrintFatalError("Modifier value '" + std::to_string(Idx) +
+                              "' is not valid for this mapping (has " +
+                              std::to_string(ModifierMappings->size()) +
+                              " mappings)");
+    return (*ModifierMappings)[Idx];
+  }
+
+  int mapIndex(int Idx) const {
+    return mapIndex(Idx, ModifierMappings);
+  }
+
+protected:
+  DiagnosticTextBuilder &Builder;
+  ModifierMappingsType ModifierMappings;
+};
+
+void escapeRST(StringRef Str, std::string &Out) {
+  for (auto K : Str) {
+    if (StringRef("`*|_[]\\").count(K))
+      Out.push_back('\\');
+    Out.push_back(K);
+  }
+}
+
+template <typename It> void padToSameLength(It Begin, It End) {
+  size_t Width = 0;
+  for (It I = Begin; I != End; ++I)
+    Width = std::max(Width, I->size());
+  for (It I = Begin; I != End; ++I)
+    (*I) += std::string(Width - I->size(), ' ');
+}
+
+template <typename It> void makeTableRows(It Begin, It End) {
+  if (Begin == End)
+    return;
+  padToSameLength(Begin, End);
+  for (It I = Begin; I != End; ++I)
+    *I = "|" + *I + "|";
+}
+
+void makeRowSeparator(std::string &Str) {
+  for (char &K : Str)
+    K = (K == '|' ? '+' : '-');
+}
+
+struct DiagTextDocPrinter : DiagTextVisitor<DiagTextDocPrinter> {
+  using BaseTy = DiagTextVisitor<DiagTextDocPrinter>;
+  DiagTextDocPrinter(DiagnosticTextBuilder &Builder,
+                     std::vector<std::string> &RST)
+      : BaseTy(Builder), RST(RST) {}
+
+  void gatherNodes(
+      Piece *OrigP, const ModifierMappingsType &CurrentMappings,
+      std::vector<std::pair<Piece *, ModifierMappingsType>> &Pieces) const {
+    if (auto *Sub = dyn_cast<SubstitutionPiece>(OrigP)) {
+      ModifierMappingsType NewMappings =
+          getSubstitutionMappings(Sub, CurrentMappings);
+      return gatherNodes(Builder.getSubstitution(Sub), NewMappings, Pieces);
+    }
+    if (auto *MD = dyn_cast<MultiPiece>(OrigP)) {
+      for (Piece *Node : MD->Pieces)
+        gatherNodes(Node, CurrentMappings, Pieces);
+      return;
+    }
+    Pieces.push_back(std::make_pair(OrigP, CurrentMappings));
+  }
+
+  void VisitMulti(MultiPiece *P) {
+    if (P->Pieces.empty()) {
+      RST.push_back("");
+      return;
+    }
+
+    if (P->Pieces.size() == 1)
+      return Visit(P->Pieces[0]);
+
+    // Flatten the list of nodes, replacing any substitution pieces with the
+    // recursively flattened substituted node.
+    std::vector<std::pair<Piece *, ModifierMappingsType>> Pieces;
+    gatherNodes(P, ModifierMappings, Pieces);
+
+    std::string EmptyLinePrefix;
+    size_t Start = RST.size();
+    bool HasMultipleLines = true;
+    for (const std::pair<Piece *, ModifierMappingsType> &NodePair : Pieces) {
+      std::vector<std::string> Lines;
+      DiagTextDocPrinter Visitor{Builder, Lines};
+      Visitor.ModifierMappings = NodePair.second;
+      Visitor.Visit(NodePair.first);
+
+      if (Lines.empty())
+        continue;
+
+      // We need a vertical separator if either this or the previous piece is a
+      // multi-line piece, or this is the last piece.
+      const char *Separator = (Lines.size() > 1 || HasMultipleLines) ? "|" : "";
+      HasMultipleLines = Lines.size() > 1;
+
+      if (Start + Lines.size() > RST.size())
+        RST.resize(Start + Lines.size(), EmptyLinePrefix);
+
+      padToSameLength(Lines.begin(), Lines.end());
+      for (size_t I = 0; I != Lines.size(); ++I)
+        RST[Start + I] += Separator + Lines[I];
+      std::string Empty(Lines[0].size(), ' ');
+      for (size_t I = Start + Lines.size(); I != RST.size(); ++I)
+        RST[I] += Separator + Empty;
+      EmptyLinePrefix += Separator + Empty;
+    }
+    for (size_t I = Start; I != RST.size(); ++I)
+      RST[I] += "|";
+    EmptyLinePrefix += "|";
+
+    makeRowSeparator(EmptyLinePrefix);
+    RST.insert(RST.begin() + Start, EmptyLinePrefix);
+    RST.insert(RST.end(), EmptyLinePrefix);
+  }
+
+  void VisitText(TextPiece *P) {
+    RST.push_back("");
+    auto &S = RST.back();
+
+    StringRef T = P->Text;
+    while (!T.empty() && T.front() == ' ') {
+      RST.back() += " |nbsp| ";
+      T = T.drop_front();
+    }
+
+    std::string Suffix;
+    while (!T.empty() && T.back() == ' ') {
+      Suffix += " |nbsp| ";
+      T = T.drop_back();
+    }
+
+    if (!T.empty()) {
+      S += ':';
+      S += P->Role;
+      S += ":`";
+      escapeRST(T, S);
+      S += '`';
+    }
+
+    S += Suffix;
+  }
+
+  void VisitPlaceholder(PlaceholderPiece *P) {
+    RST.push_back(std::string(":placeholder:`") +
+                  char('A' + mapIndex(P->Index)) + "`");
+  }
+
+  void VisitSelect(SelectPiece *P) {
+    std::vector<size_t> SeparatorIndexes;
+    SeparatorIndexes.push_back(RST.size());
+    RST.emplace_back();
+    for (auto *O : P->Options) {
+      Visit(O);
+      SeparatorIndexes.push_back(RST.size());
+      RST.emplace_back();
+    }
+
+    makeTableRows(RST.begin() + SeparatorIndexes.front(),
+                  RST.begin() + SeparatorIndexes.back() + 1);
+    for (size_t I : SeparatorIndexes)
+      makeRowSeparator(RST[I]);
+  }
+
+  void VisitPlural(PluralPiece *P) { VisitSelect(P); }
+
+  void VisitDiff(DiffPiece *P) { Visit(P->Options[1]); }
+
+  std::vector<std::string> &RST;
+};
+
+struct DiagTextPrinter : DiagTextVisitor<DiagTextPrinter> {
+public:
+  using BaseTy = DiagTextVisitor<DiagTextPrinter>;
+  DiagTextPrinter(DiagnosticTextBuilder &Builder, std::string &Result)
+      : BaseTy(Builder), Result(Result) {}
+
+  void VisitMulti(MultiPiece *P) {
+    for (auto *Child : P->Pieces)
+      Visit(Child);
+  }
+  void VisitText(TextPiece *P) { Result += P->Text; }
+  void VisitPlaceholder(PlaceholderPiece *P) {
+    Result += "%";
+    Result += getModifierName(P->Kind);
+    addInt(mapIndex(P->Index));
+  }
+  void VisitSelect(SelectPiece *P) {
+    Result += "%";
+    Result += getModifierName(P->ModKind);
+    if (P->ModKind == MT_Select) {
+      Result += "{";
+      for (auto *D : P->Options) {
+        Visit(D);
+        Result += '|';
+      }
+      if (!P->Options.empty())
+        Result.erase(--Result.end());
+      Result += '}';
+    }
+    addInt(mapIndex(P->Index));
+  }
+
+  void VisitPlural(PluralPiece *P) {
+    Result += "%plural{";
+    assert(P->Options.size() == P->OptionPrefixes.size());
+    for (unsigned I = 0, End = P->Options.size(); I < End; ++I) {
+      if (P->OptionPrefixes[I])
+        Visit(P->OptionPrefixes[I]);
+      Visit(P->Options[I]);
+      Result += "|";
+    }
+    if (!P->Options.empty())
+      Result.erase(--Result.end());
+    Result += '}';
+    addInt(mapIndex(P->Index));
+  }
+
+  void VisitDiff(DiffPiece *P) {
+    Result += "%diff{";
+    Visit(P->Options[0]);
+    Result += "|";
+    Visit(P->Options[1]);
+    Result += "}";
+    addInt(mapIndex(P->Indexes[0]));
+    Result += ",";
+    addInt(mapIndex(P->Indexes[1]));
+  }
+
+  void addInt(int Val) { Result += std::to_string(Val); }
+
+  std::string &Result;
+};
+
+int DiagnosticTextBuilder::DiagText::parseModifier(StringRef &Text) const {
+  if (Text.empty() || !isdigit(Text[0]))
+    Builder.PrintFatalError("expected modifier in diagnostic");
+  int Val = 0;
+  do {
+    Val *= 10;
+    Val += Text[0] - '0';
+    Text = Text.drop_front();
+  } while (!Text.empty() && isdigit(Text[0]));
+  return Val;
+}
+
+Piece *DiagnosticTextBuilder::DiagText::parseDiagText(StringRef &Text,
+                                                      bool Nested) {
+  std::vector<Piece *> Parsed;
+
+  while (!Text.empty()) {
+    size_t End = (size_t)-2;
+    do
+      End = Nested ? Text.find_first_of("%|}", End + 2)
+                   : Text.find_first_of('%', End + 2);
+    while (End < Text.size() - 1 && Text[End] == '%' &&
+           (Text[End + 1] == '%' || Text[End + 1] == '|'));
+
+    if (End) {
+      Parsed.push_back(New<TextPiece>(Text.slice(0, End), "diagtext"));
+      Text = Text.slice(End, StringRef::npos);
+      if (Text.empty())
+        break;
+    }
+
+    if (Text[0] == '|' || Text[0] == '}')
+      break;
+
+    // Drop the '%'.
+    Text = Text.drop_front();
+
+    // Extract the (optional) modifier.
+    size_t ModLength = Text.find_first_of("0123456789{");
+    StringRef Modifier = Text.slice(0, ModLength);
+    Text = Text.slice(ModLength, StringRef::npos);
+    ModifierType ModType = llvm::StringSwitch<ModifierType>{Modifier}
+                               .Case("select", MT_Select)
+                               .Case("sub", MT_Sub)
+                               .Case("diff", MT_Diff)
+                               .Case("plural", MT_Plural)
+                               .Case("s", MT_S)
+                               .Case("ordinal", MT_Ordinal)
+                               .Case("q", MT_Q)
+                               .Case("objcclass", MT_ObjCClass)
+                               .Case("objcinstance", MT_ObjCInstance)
+                               .Case("", MT_Placeholder)
+                               .Default(MT_Unknown);
+
+    switch (ModType) {
+    case MT_Unknown:
+      Builder.PrintFatalError("Unknown modifier type: " + Modifier);
+    case MT_Select: {
+      SelectPiece *Select = New<SelectPiece>(MT_Select);
+      do {
+        Text = Text.drop_front(); // '{' or '|'
+        Select->Options.push_back(parseDiagText(Text, true));
+        assert(!Text.empty() && "malformed %select");
+      } while (Text.front() == '|');
+      // Drop the trailing '}'.
+      Text = Text.drop_front(1);
+      Select->Index = parseModifier(Text);
+      Parsed.push_back(Select);
+      continue;
+    }
+    case MT_Plural: {
+      PluralPiece *Plural = New<PluralPiece>();
+      do {
+        Text = Text.drop_front(); // '{' or '|'
+        size_t End = Text.find_first_of(":");
+        if (End == StringRef::npos)
+          Builder.PrintFatalError("expected ':' while parsing %plural");
+        ++End;
+        assert(!Text.empty());
+        Plural->OptionPrefixes.push_back(
+            New<TextPiece>(Text.slice(0, End), "diagtext"));
+        Text = Text.slice(End, StringRef::npos);
+        Plural->Options.push_back(parseDiagText(Text, true));
+        assert(!Text.empty() && "malformed %select");
+      } while (Text.front() == '|');
+      // Drop the trailing '}'.
+      Text = Text.drop_front(1);
+      Plural->Index = parseModifier(Text);
+      Parsed.push_back(Plural);
+      continue;
+    }
+    case MT_Sub: {
+      SubstitutionPiece *Sub = New<SubstitutionPiece>();
+      Text = Text.drop_front(); // '{'
+      size_t NameSize = Text.find_first_of('}');
+      assert(NameSize != size_t(-1) && "failed to find the end of the name");
+      assert(NameSize != 0 && "empty name?");
+      Sub->Name = Text.substr(0, NameSize).str();
+      Text = Text.drop_front(NameSize);
+      Text = Text.drop_front(); // '}'
+      if (!Text.empty()) {
+        while (true) {
+          if (!isdigit(Text[0]))
+            break;
+          Sub->Modifiers.push_back(parseModifier(Text));
+          if (Text.empty() || Text[0] != ',')
+            break;
+          Text = Text.drop_front(); // ','
+          assert(!Text.empty() && isdigit(Text[0]) &&
+                 "expected another modifier");
+        }
+      }
+      Parsed.push_back(Sub);
+      continue;
+    }
+    case MT_Diff: {
+      DiffPiece *Diff = New<DiffPiece>();
+      Text = Text.drop_front(); // '{'
+      Diff->Options[0] = parseDiagText(Text, true);
+      Text = Text.drop_front(); // '|'
+      Diff->Options[1] = parseDiagText(Text, true);
+
+      Text = Text.drop_front(); // '}'
+      Diff->Indexes[0] = parseModifier(Text);
+      Text = Text.drop_front(); // ','
+      Diff->Indexes[1] = parseModifier(Text);
+      Parsed.push_back(Diff);
+      continue;
+    }
+    case MT_S: {
+      SelectPiece *Select = New<SelectPiece>(ModType);
+      Select->Options.push_back(New<TextPiece>(""));
+      Select->Options.push_back(New<TextPiece>("s", "diagtext"));
+      Select->Index = parseModifier(Text);
+      Parsed.push_back(Select);
+      continue;
+    }
+    case MT_Q:
+    case MT_Placeholder:
+    case MT_ObjCClass:
+    case MT_ObjCInstance:
+    case MT_Ordinal: {
+      Parsed.push_back(New<PlaceholderPiece>(ModType, parseModifier(Text)));
+      continue;
+    }
+    }
+  }
+
+  return New<MultiPiece>(Parsed);
+}
+
+std::vector<std::string>
+DiagnosticTextBuilder::buildForDocumentation(StringRef Severity,
+                                             const Record *R) {
+  EvaluatingRecordGuard Guard(&EvaluatingRecord, R);
+  StringRef Text = R->getValueAsString("Text");
+
+  DiagText D(*this, Text);
+  TextPiece *Prefix = D.New<TextPiece>(Severity, Severity);
+  Prefix->Text += ": ";
+  auto *MP = dyn_cast<MultiPiece>(D.Root);
+  if (!MP) {
+    MP = D.New<MultiPiece>();
+    MP->Pieces.push_back(D.Root);
+    D.Root = MP;
+  }
+  MP->Pieces.insert(MP->Pieces.begin(), Prefix);
+  std::vector<std::string> Result;
+  DiagTextDocPrinter{*this, Result}.Visit(D.Root);
+  return Result;
+}
+
+std::string DiagnosticTextBuilder::buildForDefinition(const Record *R) {
+  EvaluatingRecordGuard Guard(&EvaluatingRecord, R);
+  StringRef Text = R->getValueAsString("Text");
+  DiagText D(*this, Text);
+  std::string Result;
+  DiagTextPrinter{*this, Result}.Visit(D.Root);
+  return Result;
+}
+
+} // namespace
+
 //===----------------------------------------------------------------------===//
 // Warning Tables (.inc file) generation.
 //===----------------------------------------------------------------------===//
@@ -455,6 +1183,7 @@ static bool isRemark(const Record &Diag)
   return ClsName == "CLASS_REMARK";
 }
 
+
 /// ClangDiagsDefsEmitter - The top-level class emits .def files containing
 /// declarations of Clang diagnostics.
 namespace clang {
@@ -470,8 +1199,9 @@ void EmitClangDiagsDefs(RecordKeeper &Re
     OS << "#endif\n\n";
   }
 
-  const std::vector<Record*> &Diags =
-    Records.getAllDerivedDefinitions("Diagnostic");
+  DiagnosticTextBuilder DiagTextBuilder(Records);
+
+  std::vector<Record *> Diags = Records.getAllDerivedDefinitions("Diagnostic");
 
   std::vector<Record*> DiagGroups
     = Records.getAllDerivedDefinitions("DiagGroup");
@@ -520,7 +1250,7 @@ void EmitClangDiagsDefs(RecordKeeper &Re
 
     // Description string.
     OS << ", \"";
-    OS.write_escaped(R.getValueAsString("Text")) << '"';
+    OS.write_escaped(DiagTextBuilder.buildForDefinition(&R)) << '"';
 
     // Warning associated with the diagnostic. This is stored as an index into
     // the alphabetically sorted warning table.
@@ -882,261 +1612,6 @@ void EmitClangDiagsIndexName(RecordKeepe
 namespace docs {
 namespace {
 
-/// Diagnostic text, parsed into pieces.
-struct DiagText {
-  struct Piece {
-    // This type and its derived classes are move-only.
-    Piece() {}
-    Piece(Piece &&O) {}
-    Piece &operator=(Piece &&O) { return *this; }
-
-    virtual void print(std::vector<std::string> &RST) = 0;
-    virtual ~Piece() {}
-  };
-  struct TextPiece : Piece {
-    StringRef Role;
-    std::string Text;
-    void print(std::vector<std::string> &RST) override;
-  };
-  struct PlaceholderPiece : Piece {
-    int Index;
-    void print(std::vector<std::string> &RST) override;
-  };
-  struct SelectPiece : Piece {
-    SelectPiece() {}
-    SelectPiece(SelectPiece &&O) noexcept : Options(std::move(O.Options)) {}
-    std::vector<DiagText> Options;
-    void print(std::vector<std::string> &RST) override;
-  };
-
-  std::vector<std::unique_ptr<Piece>> Pieces;
-
-  DiagText();
-  DiagText(DiagText &&O) noexcept : Pieces(std::move(O.Pieces)) {}
-
-  DiagText(StringRef Text);
-  DiagText(StringRef Kind, StringRef Text);
-
-  template<typename P> void add(P Piece) {
-    Pieces.push_back(llvm::make_unique<P>(std::move(Piece)));
-  }
-  void print(std::vector<std::string> &RST);
-};
-
-DiagText parseDiagText(StringRef &Text, bool Nested = false) {
-  DiagText Parsed;
-
-  while (!Text.empty()) {
-    size_t End = (size_t)-2;
-    do
-      End = Nested ? Text.find_first_of("%|}", End + 2)
-                   : Text.find_first_of('%', End + 2);
-    while (End < Text.size() - 1 && Text[End] == '%' && Text[End + 1] == '%');
-
-    if (End) {
-      DiagText::TextPiece Piece;
-      Piece.Role = "diagtext";
-      Piece.Text = Text.slice(0, End);
-      Parsed.add(std::move(Piece));
-      Text = Text.slice(End, StringRef::npos);
-      if (Text.empty()) break;
-    }
-
-    if (Text[0] == '|' || Text[0] == '}')
-      break;
-
-    // Drop the '%'.
-    Text = Text.drop_front();
-
-    // Extract the (optional) modifier.
-    size_t ModLength = Text.find_first_of("0123456789{");
-    StringRef Modifier = Text.slice(0, ModLength);
-    Text = Text.slice(ModLength, StringRef::npos);
-
-    // FIXME: Handle %ordinal here.
-    if (Modifier == "select" || Modifier == "plural") {
-      DiagText::SelectPiece Select;
-      do {
-        Text = Text.drop_front();
-        if (Modifier == "plural")
-          while (Text[0] != ':')
-            Text = Text.drop_front();
-        Select.Options.push_back(parseDiagText(Text, true));
-        assert(!Text.empty() && "malformed %select");
-      } while (Text.front() == '|');
-      Parsed.add(std::move(Select));
-
-      // Drop the trailing '}n'.
-      Text = Text.drop_front(2);
-      continue;
-    }
-
-    // For %diff, just take the second alternative (tree diagnostic). It would
-    // be preferable to take the first one, and replace the $ with the suitable
-    // placeholders.
-    if (Modifier == "diff") {
-      Text = Text.drop_front(); // '{'
-      parseDiagText(Text, true);
-      Text = Text.drop_front(); // '|'
-
-      DiagText D = parseDiagText(Text, true);
-      for (auto &P : D.Pieces)
-        Parsed.Pieces.push_back(std::move(P));
-
-      Text = Text.drop_front(4); // '}n,m'
-      continue;
-    }
-
-    if (Modifier == "s") {
-      Text = Text.drop_front();
-      DiagText::SelectPiece Select;
-      Select.Options.push_back(DiagText(""));
-      Select.Options.push_back(DiagText("s"));
-      Parsed.add(std::move(Select));
-      continue;
-    }
-
-    assert(!Text.empty() && isdigit(Text[0]) && "malformed placeholder");
-    DiagText::PlaceholderPiece Placeholder;
-    Placeholder.Index = Text[0] - '0';
-    Parsed.add(std::move(Placeholder));
-    Text = Text.drop_front();
-    continue;
-  }
-  return Parsed;
-}
-
-DiagText::DiagText() {}
-
-DiagText::DiagText(StringRef Text) : DiagText(parseDiagText(Text, false)) {}
-
-DiagText::DiagText(StringRef Kind, StringRef Text) : DiagText(parseDiagText(Text, false)) {
-  TextPiece Prefix;
-  Prefix.Role = Kind;
-  Prefix.Text = Kind;
-  Prefix.Text += ": ";
-  Pieces.insert(Pieces.begin(),
-                llvm::make_unique<TextPiece>(std::move(Prefix)));
-}
-
-void escapeRST(StringRef Str, std::string &Out) {
-  for (auto K : Str) {
-    if (StringRef("`*|_[]\\").count(K))
-      Out.push_back('\\');
-    Out.push_back(K);
-  }
-}
-
-template<typename It> void padToSameLength(It Begin, It End) {
-  size_t Width = 0;
-  for (It I = Begin; I != End; ++I)
-    Width = std::max(Width, I->size());
-  for (It I = Begin; I != End; ++I)
-    (*I) += std::string(Width - I->size(), ' ');
-}
-
-template<typename It> void makeTableRows(It Begin, It End) {
-  if (Begin == End) return;
-  padToSameLength(Begin, End);
-  for (It I = Begin; I != End; ++I)
-    *I = "|" + *I + "|";
-}
-
-void makeRowSeparator(std::string &Str) {
-  for (char &K : Str)
-    K = (K == '|' ? '+' : '-');
-}
-
-void DiagText::print(std::vector<std::string> &RST) {
-  if (Pieces.empty()) {
-    RST.push_back("");
-    return;
-  }
-
-  if (Pieces.size() == 1)
-    return Pieces[0]->print(RST);
-
-  std::string EmptyLinePrefix;
-  size_t Start = RST.size();
-  bool HasMultipleLines = true;
-  for (auto &P : Pieces) {
-    std::vector<std::string> Lines;
-    P->print(Lines);
-    if (Lines.empty())
-      continue;
-
-    // We need a vertical separator if either this or the previous piece is a
-    // multi-line piece, or this is the last piece.
-    const char *Separator = (Lines.size() > 1 || HasMultipleLines) ? "|" : "";
-    HasMultipleLines = Lines.size() > 1;
-
-    if (Start + Lines.size() > RST.size())
-      RST.resize(Start + Lines.size(), EmptyLinePrefix);
-
-    padToSameLength(Lines.begin(), Lines.end());
-    for (size_t I = 0; I != Lines.size(); ++I)
-      RST[Start + I] += Separator + Lines[I];
-    std::string Empty(Lines[0].size(), ' ');
-    for (size_t I = Start + Lines.size(); I != RST.size(); ++I)
-      RST[I] += Separator + Empty;
-    EmptyLinePrefix += Separator + Empty;
-  }
-  for (size_t I = Start; I != RST.size(); ++I)
-    RST[I] += "|";
-  EmptyLinePrefix += "|";
-
-  makeRowSeparator(EmptyLinePrefix);
-  RST.insert(RST.begin() + Start, EmptyLinePrefix);
-  RST.insert(RST.end(), EmptyLinePrefix);
-}
-
-void DiagText::TextPiece::print(std::vector<std::string> &RST) {
-  RST.push_back("");
-  auto &S = RST.back();
-
-  StringRef T = Text;
-  while (!T.empty() && T.front() == ' ') {
-    RST.back() += " |nbsp| ";
-    T = T.drop_front();
-  }
-
-  std::string Suffix;
-  while (!T.empty() && T.back() == ' ') {
-    Suffix += " |nbsp| ";
-    T = T.drop_back();
-  }
-
-  if (!T.empty()) {
-    S += ':';
-    S += Role;
-    S += ":`";
-    escapeRST(T, S);
-    S += '`';
-  }
-  
-  S += Suffix;
-}
-
-void DiagText::PlaceholderPiece::print(std::vector<std::string> &RST) {
-  RST.push_back(std::string(":placeholder:`") + char('A' + Index) + "`");
-}
-
-void DiagText::SelectPiece::print(std::vector<std::string> &RST) {
-  std::vector<size_t> SeparatorIndexes;
-  SeparatorIndexes.push_back(RST.size());
-  RST.emplace_back();
-  for (auto &O : Options) {
-    O.print(RST);
-    SeparatorIndexes.push_back(RST.size());
-    RST.emplace_back();
-  }
-
-  makeTableRows(RST.begin() + SeparatorIndexes.front(),
-                RST.begin() + SeparatorIndexes.back() + 1);
-  for (size_t I : SeparatorIndexes)
-    makeRowSeparator(RST[I]);
-}
-
 bool isRemarkGroup(const Record *DiagGroup,
                    const std::map<std::string, GroupInfo> &DiagsInGroup) {
   bool AnyRemarks = false, AnyNonRemarks = false;
@@ -1181,12 +1656,13 @@ void writeHeader(StringRef Str, raw_ostr
   OS << Str << "\n" << std::string(Str.size(), Kind) << "\n";
 }
 
-void writeDiagnosticText(StringRef Role, StringRef Text, raw_ostream &OS) {
+void writeDiagnosticText(DiagnosticTextBuilder &Builder, const Record *R,
+                         StringRef Role, raw_ostream &OS) {
+  StringRef Text = R->getValueAsString("Text");
   if (Text == "%0")
     OS << "The text of this diagnostic is not controlled by Clang.\n\n";
   else {
-    std::vector<std::string> Out;
-    DiagText(Role, Text).print(Out);
+    std::vector<std::string> Out = Builder.buildForDocumentation(Role, R);
     for (auto &Line : Out)
       OS << Line << "\n";
     OS << "\n";
@@ -1209,8 +1685,11 @@ void EmitClangDiagDocs(RecordKeeper &Rec
 
   OS << Documentation->getValueAsString("Intro") << "\n";
 
+  DiagnosticTextBuilder Builder(Records);
+
   std::vector<Record*> Diags =
       Records.getAllDerivedDefinitions("Diagnostic");
+
   std::vector<Record*> DiagGroups =
       Records.getAllDerivedDefinitions("DiagGroup");
   llvm::sort(DiagGroups.begin(), DiagGroups.end(), diagGroupBeforeByName);
@@ -1300,7 +1779,8 @@ void EmitClangDiagDocs(RecordKeeper &Rec
         Severity[0] = tolower(Severity[0]);
         if (Severity == "ignored")
           Severity = IsRemarkGroup ? "remark" : "warning";
-        writeDiagnosticText(Severity, D->getValueAsString("Text"), OS);
+
+        writeDiagnosticText(Builder, D, Severity, OS);
       }
     }
 




More information about the cfe-commits mailing list