[PATCH] D55463: Introduce a source minimizer that reduces source to directives that might affect the dependency list for a compilation

[Alex Lorenz via Phabricator via cfe-commits]

arphaman created this revision.
arphaman added reviewers: dexonsmith, Bigcheese, dblaikie, vsapsai, sammccall, rsmith, bruno.
Herald added subscribers: jkorous, mgorny.

This patch introduces a dependency directives source minimizer to clang that minimizes header and source files to the minimum necessary preprocessor directives for evaluating includes. It reduces the source down to `#define`, `#include`, `#import`, `@import`, and any conditional preprocessor logic that contains one of those.

The source minimizer works by lexing the input with a custom fast lexer that recognizes the preprocessor directives it cares about, and emitting those directives in the minimized source. It ignores source code, comments, and normalizes whitespace. It gives up and fails if seems any directives that it doesn't recognize as valid (e.g. `#define 0`).

In addition to the source minimizer this patch adds a `print-dependency-directives-minimized-source` CC1 option that allows you to invoke the minimizer using clang directly.

There a couple of known issues with the source minimizer:

- It fails to detect `@import` that was formed in a macro expansion. We are planning to add a warning to discourage this use.
- It fails to detect `_Pragma ("clang import")`. We are planning to probably add a warning to discourage this use.
- It assumes raw string literals are valid when minimizing source without respecting language mode.

This is based on code that was included in the original WIP patch I posted before the dev meeting: https://reviews.llvm.org/D53354 . It's based on the original filter-to-includes code written by @dexonsmith . We are planning to use to implement fast dependency scanning for explicit module builds.


Repository:
  rC Clang

https://reviews.llvm.org/D55463

Files:
  include/clang/Basic/DiagnosticFrontendKinds.td
  include/clang/Driver/CC1Options.td
  include/clang/Frontend/FrontendActions.h
  include/clang/Frontend/FrontendOptions.h
  include/clang/Lex/DependencyDirectivesSourceMinimizer.h
  lib/Frontend/CompilerInvocation.cpp
  lib/Frontend/FrontendActions.cpp
  lib/FrontendTool/ExecuteCompilerInvocation.cpp
  lib/Lex/CMakeLists.txt
  lib/Lex/DependencyDirectivesSourceMinimizer.cpp
  test/Frontend/minimize_source_to_dependency_directives.c
  test/Frontend/minimize_source_to_dependency_directives_error.c
  unittests/Lex/CMakeLists.txt
  unittests/Lex/DependencyDirectivesSourceMinimizerTest.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D55463.177344.patch
Type: text/x-patch
Size: 47552 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20181208/2f88dd40/attachment-0001.bin>