[llvm-dev] Building LLVM through Bazel

Thu Aug 16 14:13:36 PDT 2018

You can run `ninja -t commands <target>` to get all the commands that are needed to build a particular target.

From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of David Blaikie via llvm-dev <llvm-dev at lists.llvm.org>
Reply-To: David Blaikie <dblaikie at gmail.com>
Date: Thursday, August 16, 2018 at 1:37 PM
To: Douglas Parker <dgp1130422 at gmail.com>
Cc: llvm-dev <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Building LLVM through Bazel

Yeah, compile_commands.json would be insufficient - it's just the compile commands, not tblgen. (compile_commands is for clang tools to consume so they know what command line arguments are used for a given source file, so they can reproduce that build to show clang-tidy tips, etc)

A log output from something like ninja would likely be complete, I think.

On Thu, Aug 16, 2018 at 1:28 PM Douglas Parker <dgp1130422 at gmail.com<mailto:dgp1130422 at gmail.com>> wrote:
I tried running all the commands in the compile_commands.json (thanks for pointing that out), but it runs into the same "No such file or directory: *.inc". I don't see those files built anywhere in that list. Does it take tblgen into account?

Doug

On Thu, Aug 16, 2018, 1:02 PM Sanjoy Das <sanjoy at playingwithpointers.com<mailto:sanjoy at playingwithpointers.com>> wrote:
TensorFlow uses bazel to build LLVM:
https://github.com/tensorflow/tensorflow/tree/master/third_party/llvm
The script that generates llvm.autogenerated.BUILD is not open sourced
yet unfortunately but looking skimming through the generated file
should give you a rough idea of what's involved.

-- Sanjoy
On Thu, Aug 16, 2018 at 11:16 AM Isaiah Norton via llvm-dev
<llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
>>
>> You could look at the cmake+ninja (or other build system) build and dump the commands it executes (I think ninja produces a log, or can do so) which should show you all the commands needed to build any part of LLVM.
>
>
> There's a switch to dump all compiler commands as a JSON file:
>
> https://cmake.org/cmake/help/v3.12/variable/CMAKE_EXPORT_COMPILE_COMMANDS.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__cmake.org_cmake_help_v3.12_variable_CMAKE-5FEXPORT-5FCOMPILE-5FCOMMANDS.html&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=SwZE6EeUcW5DXygSPrhf28SPBqr6MEBEg6ZpVmaiHW4&s=c8dWQO0-w6fpxbQnWt2kW-MbFgN21WE9iAM550QSS34&e=>
>
> On Wed, Aug 15, 2018 at 8:30 PM David Blaikie via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
>>
>>
>>
>> On Wed, Aug 15, 2018 at 2:05 PM Douglas Parker <dgp1130422 at gmail.com<mailto:dgp1130422 at gmail.com>> wrote:
>>>
>>> I believe it would be possible to run a cmake command to generate a BUILD file, though I don't know if that would be easier to maintain on the LLVM side. Would definitely be happy to see direct support, though I was just trying to figure out what's needed to hack this together on my end.
>>>
>>> I guess my real question is what underlying commands are necessary to build all the source files (without actually compiling the C/C++)? If I can understand how to do this manually with terminal/cmake/tblgen, then I could probably get it to work with Bazel.
>>>
>>> Looking at TensorFlow's setup, it looks like tblgen has dependencies on support which goes down to zlib and gets a lot of complexity there. I get the impression that tblgen isn't so complicated as to require all that. What would be the minimum steps to build tblgen from source?
>>
>>
>> You could look at the cmake+ninja (or other build system) build and dump the commands it executes (I think ninja produces a log, or can do so) which should show you all the commands needed to build any part of LLVM.
>>
>> But yeah, I think you do have to build support to build tblgen, to run tblgen to generate the rest of the files to continue the build.
>>
>> - Dave
>>
>>>
>>>
>>> Doug
>>>
>>> On Wed, Aug 15, 2018 at 1:28 PM Chris Bieneman <chris.bieneman at me.com<mailto:chris.bieneman at me.com>> wrote:
>>>>
>>>> There have been discussions about adding a Bazel generator before:
>>>> https://cmake.org/pipermail/cmake-developers/2017-July/030144.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__cmake.org_pipermail_cmake-2Ddevelopers_2017-2DJuly_030144.html&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=SwZE6EeUcW5DXygSPrhf28SPBqr6MEBEg6ZpVmaiHW4&s=oAWAXGa5gSeVOQMHrVG3qZOYnIK3P2Imys327wAyKEs&e=>
>>>>
>>>> There does seem to be interest in having that support in CMake, and I can't imagine any insurmountable reason why it couldn't be done. The real issue is that nobody has put in the time to do it.
>>>>
>>>> -Chris
>>>>
>>>> On Aug 15, 2018, at 11:21 AM, David Blaikie via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
>>>>
>>>>
>>>>
>>>> On Wed, Aug 15, 2018 at 10:04 AM Chris Lattner <clattner at nondot.org<mailto:clattner at nondot.org>> wrote:
>>>>>
>>>>>
>>>>>
>>>>> > On Aug 14, 2018, at 2:43 PM, David Blaikie via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
>>>>> >
>>>>> > Yeah - not sure we're quite at the point where LLVM wants to start supporting two build systems again (used to be Configure+Make and the CMake system, now it's just the Cmake system), but if you want to make it work out-of-tree it shouldn't be too difficult (Google does this internally with the internal version of Bazel). Writing a short BUILD extension for running tblgen should be possible without too much complexity - not sure what Tensorflow is doing that makes its solution so complicated, it doesn't seem like it should be terribly hard, just a genrule to run tblgen and generate the appropriate files from the td files.
>>>>>
>>>>> Would it be technically possible for cmake to generate bazel files?  If so, it seems that that could be a great bridge, and useful to other projects as well.
>>>>
>>>>
>>>> /maybe/? It's an interesting thought - not sure I know enough about either build system to have a very informed opinion here, though.
>>>>
>>>> - Dave
>>>>
>>>>>
>>>>>
>>>>> -Chris
>>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=SwZE6EeUcW5DXygSPrhf28SPBqr6MEBEg6ZpVmaiHW4&s=1avnocVwELxhTKAmvxuCATIih1jPSzAfvHvCtT5K2u4&e=>
>>>>
>>>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=SwZE6EeUcW5DXygSPrhf28SPBqr6MEBEg6ZpVmaiHW4&s=1avnocVwELxhTKAmvxuCATIih1jPSzAfvHvCtT5K2u4&e=>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=SwZE6EeUcW5DXygSPrhf28SPBqr6MEBEg6ZpVmaiHW4&s=1avnocVwELxhTKAmvxuCATIih1jPSzAfvHvCtT5K2u4&e=>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180816/4ecd7097/attachment.html>