How to write and use chained PCHs

[Yao SHEN]

I want to try in following way. Would you please check if it is feasible 
for me?

1. For each .h file, I generate one PCH file;

2. When I generate a PCH file, I store its included .h files (with full 
path information) in the PreprocessorOpt.Includes

3. When I open a source code, say xxx.cpp file to parse, in the 
PPCallback function for each #include in xxx.cpp, I check if there is a 
corresponding PCH file. If no, I first generate the PCH file. Then load 
the PCH file using ASTReader.

4. When I load a PCH file, I use ASTReaderListener to check if there are 
any #includes in the PCH's PreprocessorOpt.Includes. If so, I load those 
PCHs first. This can be a nested invocation of the ASTReaderListener. In 
this way, I hope the chained includes can be implemented.

5. After loading a PCH file, I use ASTReader's InitializeSema and 
ASTContext.setExternalSource, so the preprocessor can find all the 
identifiers defined in the PCH.

Why I use one PCH for each .h file is because I can compare the two 
files' time to verify if the source .h is changed. If changed, I have to 
generate a new PCH for it. Do you know any information stored in a PCH I 
can use for this purpose?

Thank you so much!
-Yao



On  Tue, Aug 13, 2013 at 6:08 PM, Eli Friedman 
<eli.friedman at gmail.com> wrote:

> On Tue, Aug 13, 2013 at 6:19 AM, Yao SHEN <yaoshen at andrew.cmu.edu 
> <mailto:yaoshen at andrew.cmu.edu>> wrote:
>
>     Hello,
>
>     This is Yao Shen. I got a problem recently and hope someone here
>     can help me.
>
>     I am using llvm+clang to parse c/c++ codes. My objective is to do
>     some instrumentations towards these codes by inserting
>     instructions into the codes.
>
>     The problem is: the codes are somewhat large. They usually larger
>     than 50M. And there are a lot of header files shared by c/c++
>     files. All of these make the parsing process very slowly (> 30
>     minutes).
>
>     So I am think if I can use PCH. In the first step, I will compile
>     all of the header files into PCHs. And then when parsing c/c++
>     files, I load corresponding PCH files in the PPCallback using
>     ASTReader. It works when the header file is simple (not including
>     other headers). However, if there is a chained including, it
>     cannot work.
>
>     For example, if A.h includes B.h, and if I first convert B.h into
>     B.h.pch, and A.h into A.h.pch, then when parsing main.c which
>     includes A.h, the identifiers in B.h cannot be recognized. And if
>     the main.c include both A.h and B.h, the parse will not pass
>     successfully even if I use #ifdef... in B.h to avoid redefinitions.
>
>     Maybe I have to use chainedincludesSource. However, there is few
>     guides I can get. I don't know how to use it.
>
>     Is there anyone who can help? Do you think using
>     PPCallback::InclusionDirective is a right way in this situation?
>
>
> You might want to consider modifying your source code to use modules; 
> see http://clang.llvm.org/docs/Modules.html .  Alternatively, instead 
> of compiling individual headers using PCH, you could precompile one 
> big header which includes all the commonly used headers in your project.
>
> This is a hard problem; if it were easy, clang would do it for you 
> already. :)
>
> -Eli

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20130813/e97f8fda/attachment.html>