[cfe-dev] icecream - what should /proc/cpuinfo contain on remote machines?

Thu May 11 13:39:36 PDT 2017

Ah, so I should just merge that fix into 4.0.1 then? Let me look into that.

Teresa

On Thu, May 11, 2017 at 1:33 PM, Miller Henry <MillerHenry at johndeere.com>
wrote:

> Looks like that is from April, while clang 4.0 was released in March(?).
> My users tend to stick to released versions.
>
>
>
> *From:* Teresa Johnson [mailto:tejohnson at google.com]
> *Sent:* Thursday, May 11, 2017 3:22 PM
> *To:* Miller Henry <MillerHenry at JohnDeere.com>
> *Cc:* Reid Kleckner <rnk at google.com>; cfe-dev at lists.llvm.org
>
> *Subject:* Re: [cfe-dev] icecream - what should /proc/cpuinfo contain on
> remote machines?
>
>
>
> I just realized, as Reid pointed out, the code already does return -1 when
> we hit this situation (which should cause fall back to
> std::thread::hardware_concurrency). This was fixed in
> https://reviews.llvm.org/D32032 (r300267).
>
>
>
> So you would be getting an error message, but it should still fall back to
> the right behavior. Is it just the existence of the printed message that is
> causing you grief? It looks like you are actually getting a non-zero exit
> code, how is that happening?
>
>
>
> Teresa
>
>
>
> On Thu, May 11, 2017 at 1:05 PM, Miller Henry <MillerHenry at johndeere.com>
> wrote:
>
> Looks like consensus is this is a bug not intended behavior so I wrote up
> bug 33008 on it.  Thanks for the feedback.
>
>
>
> *From:* Teresa Johnson [mailto:tejohnson at google.com]
> *Sent:* Wednesday, May 10, 2017 6:56 PM
> *To:* Reid Kleckner <rnk at google.com>
> *Cc:* Miller Henry <MillerHenry at JohnDeere.com>; cfe-dev at lists.llvm.org
> *Subject:* Re: [cfe-dev] icecream - what should /proc/cpuinfo contain on
> remote machines?
>
>
>
>
>
>
>
> On Wed, May 10, 2017 at 1:01 PM, Reid Kleckner <rnk at google.com> wrote:
>
> This seems like an LLVM bug. I'd rather LLVM didn't look in /proc, but I
> can't find an alternative. In any case, we shouldn't print to stderr for a
> successful build. Silently falling back to std::thread::hardware_concurrency()
> seems fine, which is what the code does.
>
>
>
> Agreed, we should at least return -1 if we can't find /proc/cpuinfo (which
> will result in fallback to std::thread::hardware_concurrency()). The
> question I have is whether it should be silent or not - my understanding
> was that this file should exist in x86_64 Linux.
>
>
>
> But I am also wondering how you hit this, since reading through the github
> thread you linked indicated that this is not LTO or ThinLTO. Oh, I see now
> where it is being used to initialize the value of an option...if this is a
> file that will not always exist for Linux x86_64 then it needs to fall back
> silently.
>
>
>
> Teresa
>
>
>
>
>
> On Wed, May 10, 2017 at 12:21 PM, Miller Henry via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>
>
> Greetings, I’m the maintainer of icecream, and I’m trying to solve a bug
> some of my users are seeing: when using clang  4.0 their build fails
> because the file /proc/cpuinfo does not exist.  I’m trying to figure out
> what the right solution to this is. See https://github.com/icecc/
> icecream/issues/176
>
> From what we can tell this came in from: https://reviews.llvm.org/D25564.
>
>
>
>
> A summary for those who don’t want to read the above: icecream is a
> distributed build tool that allows you to use other networked computers to
> speed up your builds.  The local build machine where you run “make” ships
> the compiler to remote machines which then run the compiler in a chroot
> (this ensures that if the remote has a different version of clang installed
> we do not use that).
>
>
>
> We see three options:
>
>
>
> Place an empty /proc/cpuinfo in the chroot. This seems to work and it is
> easy.  From what we can tell this file is only used at link time and
> icecream does not link on remote machines.  However if clang starts to use
> /proc/cpuinfo for something else (asserts on valid contents) we are broke
> again.
>
>
>
> When building the compiler package add /proc/cpuinfo from whatever machine
> is building the environment.  /proc/cpuinfo may be bogus but at least it
> exists. This is easy to do, but it is a coincidence if the file is actually
> correct.
>
>
>
> Copy /proc/cpuinfo into the chroot on each build machine.  This is the
> most difficult to implement (test), and I’m not sure if it gains anything
> or not. Note in particular that icecream tries hard to use as many cores on
> the remote machines as possible without the user of the remote machine
> noticing a performance problem, thus clang using /proc/cpuinfo to spawn
> additional threads is probably something we do not want.
>
>
>
> I’m leaning to the first because it is easy, but others have different
> opinions. After looking at all the responses I’ve concluded that so far
> nobody actually knows enough about what clang is doing (much less planning
> to do) to make a useful decision.  Thus I’m reaching out to the clang
> community: what considerations are we not aware of, and is there a good
> reason to choose any plan?
>
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
>
>
>
>
>
> --
>
> Teresa Johnson |
>
>  Software Engineer |
>
>  tejohnson at google.com |
>
>  408-460-2413 <(408)%20460-2413>
>
>
>
>
>
>
>
> --
>
> Teresa Johnson |
>
>  Software Engineer |
>
>  tejohnson at google.com |
>
>  408-460-2413 <(408)%20460-2413>
>
>
>

-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |  408-460-2413
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170511/799870e9/attachment.html>