<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br class="">
<blockquote
cite="mid:EB29F3D0-154F-458C-87ED-48FB7011A261@apple.com"
type="cite">
<div class="">
<div>
<blockquote type="cite" class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
<ul class="">
<li class="">To get cmake to work, I have to set
HAVE_CXX_ATOMICS_WITHOUT_LIB, even though I have no
intention of building LLVM. I then get to set
LIBCXX_HAVE_CXX_ATOMICS_WITHOUT_LIB too, because
reasons.</li>
</ul>
</div>
</blockquote>
This is bad. I’m curious why you need to set those ever. Have
you diagnosed this? For you to need to set that it means the
host toolchain isn’t properly passing the CMake checks.<br
class="">
</div>
</div>
</blockquote>
It looks like I don't need to set these anymore, but the comment
that I left myself at the time was that HAVE_CXX_ATOMICS_WITHOUT_LIB
looked at the state of my local machine's toolchain, as opposed to
the toolchain that I was about to use. My local machine's toolchain
is gcc 4.6.3, and it doesn't have an <atomic> header.<br>
<br>
<blockquote
cite="mid:EB29F3D0-154F-458C-87ED-48FB7011A261@apple.com"
type="cite">
<div class="">
<div>
<blockquote type="cite" class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
<ul class="">
<li class="">Multi-libs require multiple independent
build directories, with all the associated cmake
overhead.<br class="">
</li>
</ul>
</div>
</blockquote>
I strongly believe that for building multi-lib or more
generally when building for multiple targets you want multiple
build directories, and in particular the multiple-cmake
invocations. I believe this because you want the checks to be
relevant for the target, and the only way to do that is to run
the checks once per target.</div>
<div><br class="">
</div>
<div>That said, I also strongly believe that for any user of our
projects we should find a way to have a single simple CMake
invocation that gets the end result that you want. I don’t
believe these are mutually exclusive goals.</div>
<div><br class="">
</div>
<div>One of the development steps of the new runtime directory
will be supporting specifying multiple targets to build the
runtimes for, and having CMake construct the appropriate
number of build directories and manage building them all
through a single top-level configuration and build directory.
If you’re skeptical about how doable this is I’d encourage you
to look at this bot -> <a moz-do-not-send="true"
href="http://lab.llvm.org:8011/builders/clang-3stage-ubuntu"
class="">http://lab.llvm.org:8011/builders/clang-3stage-ubuntu</a>.</div>
<div><br class="">
</div>
<div>That bot does a full 3-stage clang build from a single
CMake invocation:</div>
<div><br class="">
</div>
<div>cmake -C ../llvm.src/tools/clang/cmake/caches/3-stage.cmake
-GNinja -DLLVM_TARGETS_TO_BUILD=all
-DLLVM_BINUTILS_INCDIR=/opt/binutils/include ../llvm.src</div>
<div><br class="">
</div>
</div>
</blockquote>
I'm fine if I can invoke cmake once and get multiple library
variants out. If that means that behind-the-scenes, cmake has
multiple build sub-directories, then that's fine by me. On Windows,
this already happens to some degree, as the Visual Studio project is
allowed to switch between Release, Debug, RelWithDebInfo, etc
without re-running cmake.<br>
<br>
<blockquote
cite="mid:EB29F3D0-154F-458C-87ED-48FB7011A261@apple.com"
type="cite">
<div class="">
<div>
<blockquote type="cite" class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
<p class="">So why not run out of tree instead you may
ask?</p>
<ul class="">
<li class="">No lit tests or lit utilities (FileCheck,
not, etc…)<br class="">
</li>
</ul>
</div>
</blockquote>
I don’t know if the runtimes you’re building are setup for
this or not, but you can get out-of-tree tests working if you
have an LLVM installation on the system or a build directory
that you can point at. Compiler-RT does this. It isn’t ideal
but it is workable.</div>
<div><br class="">
</div>
<div>We should have a better solution.<br class="">
</div>
</div>
</blockquote>
The nightly builds that my organization produces is generally a
"customer build", and not a "developer build". Stated otherwise, it
doesn't include llvm-config, libclang.a, or any of the other things
that people building against llvm would want. That means that if I
wanted to use this particular out-of-tree solution, I would still
need to clone down LLVM and rebuild and reinstall it on occasion.<br>
<br>
<blockquote
cite="mid:EB29F3D0-154F-458C-87ED-48FB7011A261@apple.com"
type="cite">
<div class="">
<div>
<blockquote type="cite" class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
<ul class="">
</ul>
<p class="">So some things I would like to see...</p>
<ul class="">
<li class="">Standalone runtime builds should use the
"normal" build interfaces (bare make, make all, make
check, make install. s/make/ninja as desired).</li>
</ul>
</div>
</blockquote>
I think this is doable, but I’m hesitant to rope it in with
what I’m trying to do here. Nothing I want to do would prevent
this or make it any harder than it already is.<br class="">
</div>
</div>
</blockquote>
Completely fair. I figured I'd get my grievances and wishlist out
first, just so that some agreement on a direction can be figured
out. It does extend beyond the directory re-org patch.<br>
<br class="">
<blockquote
cite="mid:EB29F3D0-154F-458C-87ED-48FB7011A261@apple.com"
type="cite">
<div class="">
<div>
<blockquote type="cite" class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
<ul class="">
<li class="">Break out testing infrastructure to a
common repo, so that the runtimes can have access to
the testing "banana" without dragging along the LLVM
"gorilla”.</li>
</ul>
</div>
</blockquote>
I’m hesitant to suggest more and more repos because I think
there are some challenges and additional burdens with that. I
do understand the benefit of what you’re asking for here, and
I think it is worth considering. I think there is an argument
for splitting out the LLVM testing infrastructure, as well as
an argument for splitting out the LLVM build infrastructure.</div>
<div><br class="">
</div>
<div>In both cases I think those changes are larger than what
I’m proposing, but worth considering.</div>
<div><br class="">
</div>
<div>-Chris</div>
<div><br class="">
</div>
<div>Obligatory troll: Maybe we should move to github and change
the whole repo structure in the process?<br class="">
</div>
</div>
</blockquote>
I figured the github thread was crazy enough without me sabotaging
it with this kind of suggestion :)<br>
<br>
<blockquote
cite="mid:EB29F3D0-154F-458C-87ED-48FB7011A261@apple.com"
type="cite">
<div class="">
<div>
<blockquote type="cite" class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
<p class=""> </p>
<br class="">
<div class="moz-cite-prefix">On 6/9/2016 12:20 PM, Chris
Bieneman via llvm-dev wrote:<br class="">
</div>
<blockquote
cite="mid:99120307-37D3-427F-81D1-14BC9C22FEBF@apple.com"
type="cite" class="">
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8" class="">
<div class="">Moving to llvm-dev (I think this has gone
a bit further than a patch review discussion)</div>
<div class=""><br class="">
</div>
<div class="">In hindsight I probably should have
explained more of my thinking on this with the patch,
or done an RFC on llvm-dev to start with. I’l do that
now, and answer the questions along the way. I sent a
separate email discussing Justin’s patch review
feedback.</div>
<div class=""><br class="">
</div>
<div class="">In the build system today there is no
strong distinction between ‘projects’ and ‘tools’.
There are a few subtle differences, but I’m not sure
any of them really matter. The differences are:</div>
<div class=""><br class="">
</div>
<div class="">(1) The projects directory is always
configured, tools can be disabled using
LLVM_INCLUDE_TOOLS=Off (projects and tools can both be
individually disabled too)</div>
<div class="">(2) Projects are configured before tools,
so tools can rely on targets being created for
projects (we don’t really use this, and anywhere we
are is probably a bug)</div>
<div class="">(3) Some projects have special handling.
For example test-suite isn’t actually treated as a
project, it has special handling in
LLVM/CMakeLists.txt:727, and Compiler-RT is handled by
clang if you set LLVM_BUILD_EXTERNAL_COMPILER_RT=On.</div>
<div class=""><br class="">
</div>
<div class="">With this in mind I was thinking about the
general usability of our build system. The distinction
between a project and a tool is not very clear. At a
high level I see three different use cases that are
covered by our current projects & tools
directories.</div>
<div class=""><br class="">
</div>
<div class="">(1) Projects that are configured with LLVM</div>
<div class="">(2) Runtime projects that should be
configured using the just-built tools</div>
<div class="">(3) The LLVM test-suite, which is really
just external tests that should be configured and run
with the just-built tools</div>
<div class=""><br class="">
</div>
<div class="">My proposal is that we make the tools
subdirectory the *only* place for projects that fall
into category 1. I don’t think there is any technical
reason to drop an in-tree project into projects over
tools today, and I think we migrating people who are
doing that away from it should be easy.</div>
<div class=""><br class="">
</div>
<div class="">Second I want to add a “runtimes”
directory to LLVM to cover case 2 (see D20992). The
idea behind this is to use common code in LLVM to
support building runtimes. This will allow the full
LLVM toolchain to be visible during configuration. I
will abstract this functionality into an installed
CMake module so that Clang can use it for out-of-tree
clang builds.</div>
<div class=""><br class="">
</div>
<div class="">Lastly we need to give the test-suite a
new home. I’m not super concerned with where we do
that. It could be under tests, it could just be at the
root of the LLVM directory. I don’t think it matters
too much because it is a one-off. Thoughts welcome.</div>
<div class=""><br class="">
</div>
<div class="">My proposed patch makes the runtimes
directory work for Compiler-RT, but it doesn’t yet
handle libcxxabi, libcxx and libunwind. There is some
special case handling between libcxxabi and libcxx
that will need to be handled to make the dependencies
work between the two, and I still need to work that
out.</div>
<div class=""><br class="">
</div>
<div class="">If we want to go with this proposal I
envision the transition being multi-staged:</div>
<div class=""><br class="">
</div>
<div class="">(1) Adding the new functionality, getting
it up and fully working for all runtime projects -
this will involve changes to runtime projects</div>
<div class="">(2) Work with bot maintainers to migrate
bots, and fix any issues that come up</div>
<div class="">(3) Add support for a new secondary
location for the test-suite</div>
<div class="">(4) Set a date for removing the projects
directory, post patches including updated
documentation</div>
<div class="">(5) Remove the projects directory entirely</div>
<div class=""><br class="">
</div>
<div class="">Thoughts?</div>
<div class="">-Chris</div>
<div class=""><br class="">
<div class="">
<blockquote type="cite" class="">
<div class="">On Jun 8, 2016, at 6:59 PM, Chandler
Carruth <<a moz-do-not-send="true"
href="mailto:chandlerc@gmail.com" class="">chandlerc@gmail.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" style="font-family: Helvetica;
font-size: 12px; font-style: normal;
font-variant-caps: normal; font-weight:
normal; letter-spacing: normal; orphans: auto;
text-align: start; text-indent: 0px;
text-transform: none; white-space: normal;
widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;" class="">
<div class="gmail_quote">
<div dir="ltr" class="">On Wed, Jun 8, 2016
at 4:39 PM Justin Bogner via llvm-commits
<<a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>>
wrote:<br class="">
</div>
<blockquote class="gmail_quote"
style="margin: 0px 0px 0px 0.8ex;
border-left-width: 1px; border-left-color:
rgb(204, 204, 204); border-left-style:
solid; padding-left: 1ex;">Chris Bieneman
<<a moz-do-not-send="true"
href="mailto:beanz@apple.com"
target="_blank" class="">beanz@apple.com</a>>
writes:<br class="">
> beanz created this revision.<br
class="">
> beanz added reviewers: chandlerc,
bogner.<br class="">
> beanz added a subscriber:
llvm-commits.<br class="">
><br class="">
> There are a few LLVM projects that
produce runtime libraries. Ideally<br
class="">
> runtime libraries should be built
differently than other projects,<br
class="">
> specifically they should be built
using the just-built toolchain.<br
class="">
><br class="">
> There is support for building
compiler-rt in this way from the clang<br
class="">
> build. Moving this logic into the
LLVM build is interesting because it<br
class="">
> provides a simpler way to extend the
just-built toolchain to include<br
class="">
> LLD and the LLVM object file tools.<br
class="">
><br class="">
> Once this functionality is better
fleshed out and tested we’ll want to<br
class="">
> encapsulate it in a module that can
be used for clang standalone<br class="">
> builds, and we’ll want to make it the
default way to build compiler-rt.<br
class="">
><br class="">
> With this patch applied there is no
immediate change in the build.<br class="">
> Moving compiler-rt out from
llvm/projects into llvm/runtimes enables<br
class="">
> the functionality.<br class="">
<br class="">
This seems reasonable, but I am a little
worried about how transitioning<br
class="">
to the new system will work. Will everyone
have to move their<br class="">
compiler-rt checkout? Will we continue to
support compiler-rt in either<br class="">
place? Both of these are workable, but
neither is great. Thoughts?<br class="">
</blockquote>
<div class=""><br class="">
</div>
<div class="">I share your concerns, but I
also kind of like the direction this is
going.</div>
<div class=""><br class="">
</div>
<div class="">But there is a higher-level
meta-point: do we want to keep the
'projects' directory *at all*.</div>
<div class=""><br class="">
</div>
<div class="">Every single resident of it I
can think of except for the test-suite is
either dead (dragonegg) or a runtime
library.</div>
<div class=""><br class="">
</div>
<div class="">I think we should either have
all the build-integrated projects in a
single 'projects' directory (including LLD
and Clang), or we should have none of them
and use more domain relevant organization
(today "tools", you're adding "runtimes",
maybe we move the test-suite to go under
one of the test directories).</div>
<div class=""><br class="">
</div>
<div class="">I think we should have a
consistent plan here before moving stuff.
But once we have it, I think we shouldn't
be afraid of re-organizing stuff to make
more sense, and just work to get folks to
update their checkouts.</div>
<div class=""><br class="">
</div>
<div class="">-Chandler</div>
<div class=""> </div>
<blockquote class="gmail_quote"
style="margin: 0px 0px 0px 0.8ex;
border-left-width: 1px; border-left-color:
rgb(204, 204, 204); border-left-style:
solid; padding-left: 1ex;"><br class="">
> This code has a few improvements over
the method provided by<br class="">
> LLVM_BUILD_EXTERNAL_COMPILER_RT.
Specifically the sub-ninja command is<br
class="">
> always invoked, so changes to
compiler-rt source files will get built<br
class="">
> properly, so this patch can be used
for iterative development with<br class="">
> just-built tools.<br class="">
><br class="">
><span class="Apple-converted-space"> </span><a
moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://reviews.llvm.org/D20992"><a class="moz-txt-link-freetext" href="http://reviews.llvm.org/D20992">http://reviews.llvm.org/D20992</a></a><br
class="">
><br class="">
> Files:<br class="">
> CMakeLists.txt<br class="">
>
cmake/modules/LLVMExternalProjectUtils.cmake<br
class="">
> runtimes/CMakeLists.txt<br class="">
><br class="">
> Index: runtimes/CMakeLists.txt<br
class="">
>
===================================================================<br
class="">
> --- /dev/null<br class="">
> +++ runtimes/CMakeLists.txt<br
class="">
> @@ -0,0 +1,16 @@<br class="">
> +include(LLVMExternalProjectUtils)<br
class="">
> +<br class="">
> +# Discover the projects that use
CMake in the subdirectories.<br class="">
> +# Note that explicit cmake
invocation is required every time a new
project is<br class="">
> +# added or removed.<br class="">
> +<br class="">
> +add_custom_target(runtimes)<br
class="">
> +<br class="">
> +file(GLOB entries *)<br class="">
> +foreach(entry ${entries})<br
class="">
> + if(IS_DIRECTORY ${entry} AND
EXISTS ${entry}/CMakeLists.txt)<br
class="">
> + get_filename_component(projName
${entry} NAME)<br class="">
> +
llvm_ExternalProject_Add(${projName}
${entry} USE_TOOLCHAIN)<br class="">
> + add_dependencies(runtimes
${projName})<br class="">
> + endif()<br class="">
> +endforeach(entry)<br class="">
> Index:
cmake/modules/LLVMExternalProjectUtils.cmake<br
class="">
>
===================================================================<br
class="">
> ---
cmake/modules/LLVMExternalProjectUtils.cmake<br
class="">
> +++
cmake/modules/LLVMExternalProjectUtils.cmake<br
class="">
> @@ -29,7 +29,8 @@<br class="">
> # Extra targets in the
subproject to generate targets for<br
class="">
> # )<br class="">
> function(llvm_ExternalProject_Add
name source_dir)<br class="">
> - cmake_parse_arguments(ARG
"USE_TOOLCHAIN;EXCLUDE_FROM_ALL;NO_INSTALL"<br
class="">
> + cmake_parse_arguments(ARG<br
class="">
> +
"USE_TOOLCHAIN;EXCLUDE_FROM_ALL;NO_INSTALL;ALWAYS_CLEAN"<br
class="">
> "SOURCE_DIR"<br class="">
>
"CMAKE_ARGS;TOOLCHAIN_TOOLS;RUNTIME_LIBRARIES;DEPENDS;EXTRA_TARGETS"
${ARGN})<br class="">
> canonicalize_tool_name(${name}
nameCanon)<br class="">
> @@ -52,6 +53,10 @@<br class="">
> endif()<br class="">
> endforeach()<br class="">
><br class="">
> + if(ARG_ALWAYS_CLEAN)<br class="">
> + set(always_clean clean)<br
class="">
> + endif()<br class="">
> +<br class="">
> list(FIND TOOLCHAIN_TOOLS clang
FOUND_CLANG)<br class="">
> if(FOUND_CLANG GREATER -1)<br
class="">
> set(CLANG_IN_TOOLCHAIN On)<br
class="">
> @@ -135,6 +140,14 @@<br class="">
> CMAKE_ARGS
${${nameCanon}_CMAKE_ARGS}<br class="">
> ${compiler_args}<br
class="">
>
-DCMAKE_INSTALL_PREFIX=${CMAKE_INSTALL_PREFIX}<br
class="">
> +
-DLLVM_CONFIG_PATH=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-config<br
class="">
> +
-DLLVM_LIBRARY_OUTPUT_INTDIR=${LLVM_LIBRARY_OUTPUT_INTDIR}<br
class="">
> +
-DLLVM_RUNTIME_OUTPUT_INTDIR=${LLVM_RUNTIME_OUTPUT_INTDIR}<br
class="">
> +
-DLLVM_LIBDIR_SUFFIX=${LLVM_LIBDIR_SUFFIX}<br
class="">
> +
-DLLVM_ENABLE_WERROR=${LLVM_ENABLE_WERROR}<br
class="">
> +
-DPACKAGE_VERSION=${PACKAGE_VERSION}<br
class="">
> +
-DCMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE}<br
class="">
> +
-DCMAKE_MAKE_PROGRAM=${CMAKE_MAKE_PROGRAM}<br
class="">
<br class="">
How did you decide which variables need to
be passed through like this?<br class="">
The set seems somewhat arbitrary, but I
may be missing something<br class="">
obvious.<br class="">
<br class="">
> ${ARG_CMAKE_ARGS}<br
class="">
>
${PASSTHROUGH_VARIABLES}<br class="">
> INSTALL_COMMAND ""<br class="">
> @@ -152,7 +165,7 @@<br class="">
> ExternalProject_Add_Step(${name}
force-rebuild<br class="">
> COMMAND ${run_build}<br
class="">
> COMMENT "Forcing rebuild of
${name}"<br class="">
> - DEPENDEES configure clean<br
class="">
> + DEPENDEES configure
${always_clean}<br class="">
<br class="">
I'm not sure I understand what this does.
If I had to guess I'd say that<br class="">
when ALWAYS_CLEAN is passed the rebuild of
the external project always<br class="">
invokes clean first, but if it's not
passed we'll just invoke the<br class="">
external build and allow it to be
incremental if appropriate. Is that<br
class="">
right?<br class="">
<br class="">
> DEPENDS ${ALWAYS_REBUILD}
${ARG_DEPENDS} ${TOOLCHAIN_BINS}<br
class="">
> ${cmake_3_4_USES_TERMINAL} )<br
class="">
> endif()<br class="">
> Index: CMakeLists.txt<br class="">
>
===================================================================<br
class="">
> --- CMakeLists.txt<br class="">
> +++ CMakeLists.txt<br class="">
> @@ -720,6 +720,8 @@<br class="">
> add_subdirectory(tools)<br
class="">
> endif()<br class="">
><br class="">
> +add_subdirectory(runtimes)<br
class="">
> +<br class="">
> if( LLVM_INCLUDE_EXAMPLES )<br
class="">
> add_subdirectory(examples)<br
class="">
> endif()<br class="">
> @@ -730,7 +732,8 @@<br class="">
>
llvm_ExternalProject_Add(test-suite
${LLVM_MAIN_SRC_DIR}/projects/test-suite<br
class="">
> USE_TOOLCHAIN<br class="">
> EXCLUDE_FROM_ALL<br class="">
> - NO_INSTALL)<br class="">
> + NO_INSTALL<br class="">
> + ALWAYS_CLEAN)<br class="">
> endif()<br class="">
> add_subdirectory(test)<br class="">
> add_subdirectory(unittests)<br
class="">
><br class="">
<br class="">
_______________________________________________<br class="">
llvm-commits mailing list<br class="">
<a moz-do-not-send="true"
href="mailto:llvm-commits@lists.llvm.org"
target="_blank" class="">llvm-commits@lists.llvm.org</a><br
class="">
<a moz-do-not-send="true"
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits"
rel="noreferrer" target="_blank"
class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a></blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
<br class="">
<fieldset class="mimeAttachmentHeader"></fieldset>
<br class="">
<pre class="" wrap="">_______________________________________________
LLVM Developers mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a>
</pre>
</blockquote>
<br class="">
<pre class="moz-signature" cols="72">--
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
</pre>
</div>
_______________________________________________<br class="">
LLVM Developers mailing list<br class="">
<a moz-do-not-send="true"
href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br
class="">
<a class="moz-txt-link-freetext" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br
class="">
</blockquote>
</div>
<br class="">
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
</pre>
</body>
</html>