[LLVMdev] make check rebuilds the project?

[Eric Christopher]

On Wed, Apr 17, 2013 at 1:05 PM, Hal Finkel <hfinkel at anl.gov> wrote:
> ----- Original Message -----
>> From: "Jim Grosbach" <grosbach at apple.com>
>> To: "Daniel Berlin" <dberlin at dberlin.org>
>> Cc: llvmdev at cs.uiuc.edu
>> Sent: Wednesday, April 17, 2013 2:46:49 PM
>> Subject: Re: [LLVMdev] make check rebuilds the project?
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> cmake+ninja is a non-option for many developers.
>>
>>
>> By the way, i'm curious what this means. It can be interpreted in a
>> number of ways, but it would be good to know what problems you are
>> specifically saying making it a non-option for many developers
>>
>>
>> Hi Danny,
>>
>>
>> Sorry for being unclear. My fault for posting when I’m in a hurry.
>>
>>
>> I don’t think there’s any unsolvable problems. Well, not technical
>> ones anyway. It’s a question of supporting a broad enough cross
>> section of environments and workflows that devs not only can move to
>> cmake+ninja, but actively want to do so.
>>
>>
>> For me personally, it’s more a question of supported workflows being
>> inconvenient than it being a strict non-starter. That is, it’s a
>> question of which system has fewer inconveniences for me in my daily
>> work. Right now, that’s configure+make. I’m very much looking
>> forward to the day when the balance of that equation changes.
>>
>> Note, I’m explicitly excluding the whole “remove autoconf+make” thing
>> from this. That’s another topic with a whole extra set of issues.
>> Likewise, I’m not going to make any sorts of claims about
>> exhaustiveness here, but rather just relate mostly what I’ve
>> encountered.
>>
>>
>> AFAIK ninja still only has very preliminary support for Windows, so
>> anyone working there can’t use it. Perhaps that’s changed or I’m
>> misinformed? Last I tried it, OSX support was better, but still very
>> much “try it and see if it works for you.” I personally found
>> ninja’s OSX support to be fine, but I didn’t really beat on it to
>> put it through its paces, either. Relatedly, some work environments
>> have varying ability to install and use “non-standard” tools. I’ve
>> been in workplaces before that I’d have had to fight tooth and nail
>> to get something like cmake or ninja available to me on my
>> workstation. Even now when I don’t have that policy sort of problem,
>> it’s still a pain when every time I install a new system (which I do
>> disturbingly often), I need to install additional tools before I can
>> start using llvm. Not a big problem on its own of course, but an
>> inconvenience.
>>
>>
>> cmake+ninja doesn’t support building and running the llvm test suite.
>> That’s pure configure+make still. I rely on configure automatically
>> setting up the test suite to run on whatever compiler I just built.
>>
>>
>> For example, I build Debug+Asserts, Release+Asserts and Release
>> configurations all in the same build directory and rely on the
>> configure+make machinery to create separate build results
>> subdirectories named for those. That is, it’s pretty common for me
>> to do something that boils down to (with lots of steps in between of
>> course), “<src>/configure && make && make check && make
>> ENABLE_OPTIMIZED=1 && make check ENABLE_OPTIMIZED=1 && make
>> ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1 && make check
>> ENABLE_OPTIMIZED=1 DISABLE_ASSERTIONS=1”. Depending on what I’m
>> doing, I bounce back and forth between them. With cmake, I have to
>> have completely separate configured build directories. I already
>> have 10+ llvm build directories lying around that I regularly use.
>> I’d prefer to keep that number from growing too horribly. When I
>> last tried using cmake+ninja, this is the one that I kept running
>> into and getting annoyed with.
>>
>>
>> Do the cmake scripts support cross-compiling? Last I tried, I had
>> trouble with that. Many moons ago, I had to beat the plain Makefiles
>> into some sort of submission for that. I not infrequently build llvm
>> to run on ARM, for example.
>>
>>
>> Somewhat relatedly, I like to keep my development builds as close as
>> reasonably possible to production builds. There are few things that
>> make my stomach churn more than bugs that only reproduce on the
>> production builders. That’s somewhat off-topic, though, as that’s
>> getting into “what would it take to remove configure+make support”
>> territory.
>>
>>
>> In short, I’m a big fan of ninja and really want to use it. CMake,
>> I’m less thrilled about, but I’ll put up with it if it gets me
>> ninja. Right now though, it’s a close but not quite a fully baked
>> replacement for my uses.
>>
>>
>> Anyways, none of this should be construed to be a slight to the work
>> people have put into cmake+ninja support for llvm/clang. Quite the
>> opposite. I’m excited enough about what’s been done that I’m
>> frustrated it doesn’t fit my use-cases well enough for me to switch.
>>
>>
>> My point with that comment, which Eli correctly and effectively
>> addressed in his reply, is that we can’t assume we can define away
>> problems with the configure+make system by just telling people it’s
>> deprecated and they should use cmake+ninja instead. We’re not there
>> yet.
>
> I'd like to second all of this.
>
> Also, I'll add that I also sometimes build on machines where linking a Debug+Asserts clang takes several minutes, and I like to be able to cut off that process to run make check. However, I discovered that if you 'cd test' before running make check, then there are no dependency problems and it will just run the tests with your current build as-is. So long as that works, I'm fine with keeping the top-level make check dependent on the rest of the build.
>
>

Yeah, check is just: make -C test so that'll work as well.

The point was that if you've made source changes you should really
probably retest what you've changed and the dependency highlights that
in the normal use case. There are many ways around it, but the general
goodness should still be there.

-eric