[llvm-bugs] [Bug 48358] New: [analyzer] Plist macro-expansion crash

[via llvm-bugs]

https://bugs.llvm.org/show_bug.cgi?id=48358

            Bug ID: 48358
           Summary: [analyzer] Plist macro-expansion crash
           Product: clang
           Version: 11.0
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: enhancement
          Priority: P
         Component: Static Analyzer
          Assignee: dcoughlin at apple.com
          Reporter: benicsbalazs at gmail.com
                CC: dcoughlin at apple.com, llvm-bugs at lists.llvm.org

Created attachment 24225
  --> https://bugs.llvm.org/attachment.cgi?id=24225&action=edit
complete stack trace

Here is the minimal repro (attached patch file):
```
// RUN: %clang_analyze_cc1 -analyzer-checker=core %s  \
// RUN:   -analyzer-output=plist -o %t.plist \
// RUN:   -analyzer-config expand-macros=true

#define STRANGE_FN(x) STRANGE_FN(x, 0)
void test_strange_macro_expansion() {
  char *path;
  STRANGE_FN(path); // no-crash
  // expected-warning at -1 {{implicit declaration of function}}
  // expected-warning at -2 {{1st function call argument is an uninitialized
value}}
}

// CHECK: <key>name</key><string>STRANGE_FN</string>
// CHECK-NEXT: <key>expansion</key><string>STRANGE_FN(path, 0)</string>
```

Analyzer call (debug build):
```
./bin/clang -cc1 -internal-isystem
git/llvm-project/build/debug/lib/clang/12.0.0/include -nostdsysteminc -analyze
-analyzer-constraints=range -setup-static-analyzer -analyzer-checker=core
git/llvm-project/clang/test/Analysis/plist-macros-with-expansion.c    
-analyzer-output=plist -o tmp.plist    -analyzer-config expand-macros=true
```

Assertion triggered:
```
clang: ../../clang/lib/StaticAnalyzer/Core/PlistDiagnostics.cpp:1261:
{anonymous}::MacroExpansionInfo getMacroExpansionInfo(const
{anonymous}::MacroParamMap&, clang::SourceLocation, const
clang::Preprocessor&): Assertion `TheTok.is(tok::r_paren) && "Expanded macro
argument acquisition failed! After the end of the loop" " this token should be
')'!"' failed.
```

Relevant part of the backtrace:
```
 #9 0x00007efe6fb5cf77 getMacroExpansionInfo((anonymous
namespace)::MacroParamMap const&, clang::SourceLocation, clang::Preprocessor
const&)
git/llvm-project/build/debug/../../clang/lib/StaticAnalyzer/Core/PlistDiagnostics.cpp:1259:0
#10 0x00007efe6fb5c2ba getMacroNameAndPrintExpansion((anonymous
namespace)::TokenPrinter&, clang::SourceLocation, clang::Preprocessor const&,
(anonymous namespace)::MacroParamMap const&,
llvm::SmallPtrSet<clang::IdentifierInfo*, 8u>&)
git/llvm-project/build/debug/../../clang/lib/StaticAnalyzer/Core/PlistDiagnostics.cpp:1015:0
#11 0x00007efe6fb5c4d9 getMacroNameAndPrintExpansion((anonymous
namespace)::TokenPrinter&, clang::SourceLocation, clang::Preprocessor const&,
(anonymous namespace)::MacroParamMap const&,
llvm::SmallPtrSet<clang::IdentifierInfo*, 8u>&)
git/llvm-project/build/debug/../../clang/lib/StaticAnalyzer/Core/PlistDiagnostics.cpp:1050:0
#12 0x00007efe6fb5c108 getExpandedMacro(clang::SourceLocation,
clang::Preprocessor const&, clang::cross_tu::CrossTranslationUnitContext
const&)
git/llvm-project/build/debug/../../clang/lib/StaticAnalyzer/Core/PlistDiagnostics.cpp:1002:0
#13 0x00007efe6fb59900 (anonymous
namespace)::PlistPrinter::ReportMacroExpansions(llvm::raw_ostream&, unsigned
int)
git/llvm-project/build/debug/../../clang/lib/StaticAnalyzer/Core/PlistDiagnostics.cpp:393:0
#14 0x00007efe6fb5a783 (anonymous
namespace)::PlistDiagnostics::printBugPath(llvm::raw_ostream&,
llvm::DenseMap<clang::FileID, unsigned int, llvm::DenseMapInfo<clang::FileID>,
llvm::detail::DenseMapPair<clang::FileID, unsigned int> > const&,
clang::ento::PathPieces const&)
git/llvm-project/build/debug/../../clang/lib/StaticAnalyzer/Core/PlistDiagnostics.cpp:603:0
```

Full trace attached.


Besides this, I've observed that the handwritten macro expansion in plist
diagnostics is non-conformant. There are several examples where it produces the
wrong expansion.
Such as for this:
 - for the generated plist: https://godbolt.org/z/YhE8zY
 - for the expected macro expansions:
https://cppinsights.io/lnk?code=dm9pZCBjbGFuZ19hbmFseXplcl93YXJuSWZSZWFjaGVkKCk7CgoKI2RlZmluZSByZXRBcmcoeCkgeAojZGVmaW5lIHJldEFyZ1VuY2xvc2VkIHJldEFyZyhjbGFuZ19hbmFseXplcl93YXJuSWZSZWFjaGVkKCkKCgojZGVmaW5lIEJCIENDCiNkZWZpbmUgYXBwbHlJbnQgQkIoaW50KQojZGVmaW5lIENDKHgpIHJldEFyZ1VuY2xvc2VkCgoKdm9pZCB1bmJhbGFuY2VkTWFjcm9zKCkgewogIGFwcGx5SW50ICk7Cn0KCiNkZWZpbmUgZXhwYW5kQXJnVW5jbG9zZWRDb21tYUV4cHIoeCkgKHgsIGNsYW5nX2FuYWx5emVyX3dhcm5JZlJlYWNoZWQoKSwgMQojZGVmaW5lIGYgZXhwYW5kQXJnVW5jbG9zZWRDb21tYUV4cHIKCnZvaWQgdW5iYWxhbmNlZE1hY3JvczIoKSB7CiAgaW50IHggPSAgZihmKDEpKSAgKSk7Cn0=&std=cpp2a&rev=1.0

The code:
```
void clang_analyzer_warnIfReached();


#define retArg(x) x
#define retArgUnclosed retArg(clang_analyzer_warnIfReached()


#define BB CC
#define applyInt BB(int)
#define CC(x) retArgUnclosed


void unbalancedMacros() {
  applyInt );
}

#define expandArgUnclosedCommaExpr(x) (x, clang_analyzer_warnIfReached(), 1
#define f expandArgUnclosedCommaExpr

void unbalancedMacros2() {
  int x =  f(f(1))  )); // note the 'extra' rparens.
}
```
The code contains no typos, the parens are deliberately not matching at each
context.

Besides these bugs, I'm expecting many more - even crashing ones.
Might fuzzing could help with resolving the crashes, but that wouldn't help to
make it conformant.
I'm currently investigating how difficult it would be to use the clang's
preprocessor to achieve correct macro expansions.
But I suspect that aquiring the state of the PP at the macro expansion location
might be difficult.

I've also read some discussion about this on the cfe-dev mailing list as
'[cfe-dev] [analyzer] Retrieving macro expansions in the plist output'
(http://lists.llvm.org/pipermail/cfe-dev/2018-September/059226.html).

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20201202/4419fbb1/attachment-0001.html>