[LLVMdev] Compile a large project with llvm?

[Wink zhang]

Dear John,

Thank you very much for your help! I will try the libLTO's approach. I 
believe the situation can fit me well. Again, thank you for your time.


-Wink
John Criswell wrote:
> Wink zhang wrote:
>> Dear John,
>>
>> Thanks for your reply. I want to do  whole program analysis, all what 
>> I need are just the separate .bc files. What instructions should I 
>> follow?
>>   
> I'm a little confused.  Whole-program analysis requires that you link 
> all of the individual bitcode files from different compilation unit 
> (i.e., individual .c files) together into a single bitcode file.  Are 
> you wanting to do whole program analysis, or are you just analyzing 
> individual compilation units?
>>
>> I run the configure on apache in this way:
>>
>> CC=clang CFLAGS=-emit-llvm ./configure
>>
>> while I get the error message like:
>> checking for gcc... clang
>> checking for C compiler default output file name... configure: error: 
>> C compiler cannot create executables
>>
>>
>>
>> Did I miss something? Thanks.
>>   
>
> The problem is that your linker does not know how to link LLVM bitcode 
> files together.  When the compiler tries to link two "object files" 
> (in reality, LLVM bitcode files) together, it executes the linker to 
> do linking, but the linker fails because the object files are LLVM 
> bitcode files.
>
> This is obviously a problem for link-time optimization, so what the 
> LLVM Developers decided to do was to teach the linker how to link LLVM 
> bitcode files together.  They built a dynamic library called libLTO 
> that links LLVM bitcode files together, can do interprocedural 
> optimization on them, and generate native code from them.  The gold 
> linker on Linux and the XCode linker on Mac OS X can load this dynamic 
> library; once it does, it can link and optimize LLVM bitcode files, 
> allowing link-time optimization to occur transparently.
>
> So, if you are going to do whole program analysis, you have one of two 
> options:
>
> 1) Modify the program's build scripts and Makefiles to use llvm-ld to 
> link LLVM bitcode files together.  For Apache, this is not too 
> difficult, but it has to be done for every program you compile.
>
> 2) You can modify libLTO to run your LLVM passes when creating a final 
> executable (libLTO is told whether it is doing incremental linking or 
> final linking).  This approach, in theory, does not require you to 
> change a program's build scripts or Makefiles.  I have tried out this 
> approach with SAFECode (which does whole-program analysis).  While I 
> have not used it thoroughly yet, I believe it is a promising approach.
>
> Note that the libLTO approach can be used for analysis/optimization 
> that does not require whole program analysis; you can have your LLVM 
> passes run whenever libLTO is used or just during final linking.  It's 
> up to you.
>
> My recommendation is to first determine how many large programs you 
> want to analyze/transform.  If it's only 1 or 2, it may be easier to 
> modify the build scripts/Makefiles.  If you plan to try your LLVM 
> passes on lots of large programs, then investigating how well the 
> libLTO option works may be worth your time.
>
> If you decide to go with option #1, I have modified Apache's build 
> system once to generate a single bitcode file.  I can probably provide 
> some guidance in that process if you need it.
>
> -- John T.
>>
>> -Wink
>>
>> John Criswell wrote:
>>  
>>> Wink zhang wrote:
>>>    
>>>> Hi,
>>>>
>>>> How to compile a large project (such as Apache) by using llvm-gcc?
>>>> I tried to replace CC with llvm-gcc and CFLAGS with -emit-llvm 
>>>> while running configure, but it didn't work.
>>>>         
>>> If you just want to compile the program with llvm-gcc, then Diego's 
>>> instructions are fine.
>>>
>>> If, however, you want to do inter-procedural optimization or whole 
>>> program analysis, then there's additional things you need to do.
>>>
>>> If you just want to use LLVM's link-time optimizations, you will 
>>> need to use the libLTO library with your linker.  Your linker, in 
>>> turn, must be able to support plugins (like the gold linker, for 
>>> example).  For directions on how to set up libLTO, see 
>>> http://llvm.org/docs/LinkTimeOptimization.html and 
>>> http://llvm.org/docs/GoldPlugin.html.
>>>
>>> If you want to write and use your own link-time passes, then you 
>>> have several options:
>>>
>>> 1) Modify the program's Makefiles to use llvm-ld to link bitcode 
>>> files together.  This will allow you to generate a single bitcode 
>>> image.
>>>
>>> 2) Modify libLTO to run your analysis and transform passes.  If you 
>>> plan to work with several large applications, this is probably the 
>>> way to go.  SAFECode has a version of libLTO that does whole-program 
>>> analysis; you can find this tool in the safecode/tools/LTO directory 
>>> in the SAFECode source code (directions for getting SAFECode are at 
>>> http://safecode.cs.illinois.edu/downloads.html).  This approach can 
>>> sometimes alleviate the need for changing the Makefiles.
>>>
>>> -- John T.
>>>    
>>>> Thank you for your help.
>>>>
>>>> -Wink
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>         
>>>     
>>
>>   
>
>