[llvm-dev] Need help with code generation

Hal Finkel via llvm-dev llvm-dev at lists.llvm.org
Tue Mar 22 13:44:36 PDT 2016 

----- Original Message -----

> From: "Rui Ueyama" <ruiu at google.com>
> To: "David Blaikie" <dblaikie at gmail.com>
> Cc: "Hal Finkel" <hfinkel at anl.gov>, "llvm-dev"
> <llvm-dev at lists.llvm.org>, "Bruce Hoult" <bruce at hoult.org>
> Sent: Tuesday, March 22, 2016 3:29:34 PM
> Subject: Re: [llvm-dev] Need help with code generation

> On Tue, Mar 22, 2016 at 9:19 PM, David Blaikie < dblaikie at gmail.com >
> wrote:

> > On Tue, Mar 22, 2016 at 1:15 PM, Rui Ueyama < ruiu at google.com >
> > wrote:
> 

> > > On Tue, Mar 22, 2016 at 9:00 PM, David Blaikie <
> > > dblaikie at gmail.com
> > > >
> > > wrote:
> > 
> 

> > > > On Tue, Mar 22, 2016 at 12:36 PM, Rui Ueyama < ruiu at google.com
> > > > >
> > > > wrote:
> > > 
> > 
> 

> > > > > On Tue, Mar 22, 2016 at 7:36 PM, Rui Ueyama < ruiu at google.com
> > > > > >
> > > > > wrote:
> > > > 
> > > 
> > 
> 

> > > > > > I have a question. If there is a ELF verifier function that
> > > > > > walks
> > > > > > every part of an ELF file to verify that the file is sane,
> > > > > > and
> > > > > > if
> > > > > > you can call that before calling LLD's function, are you
> > > > > > guys
> > > > > > happy
> > > > > > with that?
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > I'd like to get you guys opinion on this question.
> > > > 
> > > 
> > 
> 
> > > > I'd still find it problematic that lld itself would consider
> > > > crash-on-invalid "not a bug" to the point of not
> > > > reviewing/approving
> > > > patches to fix such issues. That's what I'm concerned about in
> > > > this
> > > > thread.
> > > 
> > 
> 
> > > That's one way to see that. The other view is it as a whole has a
> > > boolean option IsInputTrustworthy and works accordingly. What
> > > matters most is what we provide to the users as a guarantee. You
> > > have an opinion that that should be implemented within LLD, but
> > > that
> > > would now an internal design choice. Hypothetically we had such
> > > pass
> > > to verify inputs, and if you send a patch to "fix" crash bug of
> > > LLD,
> > > we wouldn't probably reject that but instead argue that that
> > > needs
> > > to be addressed in the verifier pass instead. This is about "how"
> > > something should be implemented and usual design choice
> > > discussion,
> > > no?
> > 
> 
> > OK, sorry - some confusion. I assume that you wouldn't run this
> > verifier pass by default in lld-the-command-line-tool, right? (I
> > would guess it wouldn't meet your performance criteria)
> 

> Correct.

> > So from that perspective, lld-the-command-line-tool would still be
> > crashing-by-design on certain inputs in its
> > default/normal/user-facing mode & that would be seem problematic to
> > me.
> 

> I disagree. You are in almost all case handling valid object files
> created by compilers, and if omitting some error checks for really
> pathetic case would make code simpler and improve performance, I
> think having that option is worth it (and I believe that's the case,
> at least those who are actually writing code seems to take that
> stance.) Also, if you give broken object files, you wouldn't get an
> output anyways. The only difference that a user can observe is
> whether it dies with an error message or not. We could even catch an
> segfault and run the verifier on the input again to print out an
> error after something goes wrong.
An important point, however, is that not all buffer overruns will cause segfaults; some will just corrupt the output. Even running under ASAN won't catch everything, because some buffer overruns will simply cause garbage to appear within a valid output buffer (or data later copied into one). If we could be sure that all such behavior would definitely and immediately cause a segfault, I'd feel differently. 

-Hal 

> > If you're proposing having this verifier run by default - sure,
> > then
> > I can't construct an input that crashes the linker, it'll fail with
> > an error message an exit. Yes, that would be fine by me - the way
> > the feature is implemented is not something I mean to imply
> > constraints on.
> 

> > - David
> 

> > > > > > On Tue, Mar 22, 2016 at 6:39 PM, Hal Finkel via llvm-dev <
> > > > > > llvm-dev at lists.llvm.org > wrote:
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > From: "David Blaikie via llvm-dev" <
> > > > > > > > llvm-dev at lists.llvm.org
> > > > > > > > >
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > To: "Rafael Espíndola" < rafael.espindola at gmail.com >
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > Cc: "llvm-dev" < llvm-dev at lists.llvm.org >, "Bruce
> > > > > > > > Hoult"
> > > > > > > > <
> > > > > > > > bruce at hoult.org >
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > Sent: Tuesday, March 22, 2016 10:18:03 AM
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > Subject: Re: [llvm-dev] Need help with code generation
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > On Tue, Mar 22, 2016 at 4:27 AM, Rafael Espíndola <
> > > > > > > > llvm-dev at lists.llvm.org > wrote:
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > > Maybe not, but it's not impossible either -
> > > > > > > > > > browsers
> > > > > > > > > > manage
> > > > > > > > > > to
> > > > > > > > > > harden themselves against malicious input and they
> > > > > > > > > > operate
> > > > > > > > > > in
> > > > > > > > > > a
> > > > > > > > > > far hostile environment with many more input
> > > > > > > > > > formats
> > > > > > > > > > than
> > > > > > > > > > we
> > > > > > > > > > do.
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > It is important to note how different they are. Both
> > > > > > > > > Firefox
> > > > > > > > > and
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > Chromium have people working just to try to make them
> > > > > > > > > more
> > > > > > > > > secure.
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > Compare that with LLVM: One week ago I pointed out
> > > > > > > > > that
> > > > > > > > > your
> > > > > > > > > patch
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > (r263521) introduces a crash. It still hasn't been
> > > > > > > > > reverted
> > > > > > > > > or
> > > > > > > > > even
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > acknowledge yet.
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > > I'm not trying to shift your personal goal, or to
> > > > > > > > > > direct
> > > > > > > > > > the
> > > > > > > > > > features that you choose to put your time into, but
> > > > > > > > > > I
> > > > > > > > > > am
> > > > > > > > > > interested in project policy.
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > Why do you care about policy that is not followed? A
> > > > > > > > > policy
> > > > > > > > > saying
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > llvm should not crash on any input is as relevant as
> > > > > > > > > one
> > > > > > > > > that
> > > > > > > > > says
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > that clang should keep bootstrapping in under one
> > > > > > > > > second.
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > It's pretty different when you say, essentially, that
> > > > > > > > patches
> > > > > > > > to
> > > > > > > > address these things are unlikely to be accepted. It
> > > > > > > > doesn't
> > > > > > > > seem
> > > > > > > > surprising that people wouldn't try to provide those
> > > > > > > > patches
> > > > > > > > and
> > > > > > > > would choose not to use the project if that's the
> > > > > > > > expressed
> > > > > > > > policy
> > > > > > > > of the developers on the project and doesn't line up
> > > > > > > > with
> > > > > > > > the
> > > > > > > > needs
> > > > > > > > of other people.
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > +1
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > -Hal
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > So, if we stick to reality, what we have is that lld
> > > > > > > > > (ELF
> > > > > > > > > and
> > > > > > > > > COFF)
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > are already the most reliable parts of the toolchain.
> > > > > > > > > If
> > > > > > > > > not
> > > > > > > > > for
> > > > > > > > > Rui
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > and I being upfront about it most people would not
> > > > > > > > > even
> > > > > > > > > know
> > > > > > > > > that
> > > > > > > > > you
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > could crash it. So please, just let us keep working
> > > > > > > > > on
> > > > > > > > > the
> > > > > > > > > most
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > reliable part of the toolchain.
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > > Cheers,
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > Rafael
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > _______________________________________________
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > LLVM Developers mailing list
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > llvm-dev at lists.llvm.org
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > > _______________________________________________
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > LLVM Developers mailing list
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > llvm-dev at lists.llvm.org
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > --
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > Hal Finkel
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Assistant Computational Scientist
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Leadership Computing Facility
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > Argonne National Laboratory
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

> > > > > > > _______________________________________________
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > LLVM Developers mailing list
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > llvm-dev at lists.llvm.org
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 
> > > > > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> 

-- 

Hal Finkel 
Assistant Computational Scientist 
Leadership Computing Facility 
Argonne National Laboratory 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160322/81f01636/attachment.html>

More information about the llvm-dev mailing list