[llvm-dev] Need help with code generation

Tue Mar 22 13:53:58 PDT 2016

On Tue, Mar 22, 2016 at 9:44 PM, Hal Finkel <hfinkel at anl.gov> wrote:

>
>
> ------------------------------
>
> *From: *"Rui Ueyama" <ruiu at google.com>
> *To: *"David Blaikie" <dblaikie at gmail.com>
> *Cc: *"Hal Finkel" <hfinkel at anl.gov>, "llvm-dev" <llvm-dev at lists.llvm.org>,
> "Bruce Hoult" <bruce at hoult.org>
> *Sent: *Tuesday, March 22, 2016 3:29:34 PM
> *Subject: *Re: [llvm-dev] Need help with code generation
>
> On Tue, Mar 22, 2016 at 9:19 PM, David Blaikie <dblaikie at gmail.com> wrote:
>
>>
>>
>> On Tue, Mar 22, 2016 at 1:15 PM, Rui Ueyama <ruiu at google.com> wrote:
>>
>>> On Tue, Mar 22, 2016 at 9:00 PM, David Blaikie <dblaikie at gmail.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Mar 22, 2016 at 12:36 PM, Rui Ueyama <ruiu at google.com> wrote:
>>>>
>>>>> On Tue, Mar 22, 2016 at 7:36 PM, Rui Ueyama <ruiu at google.com> wrote:
>>>>>
>>>>>> I have a question. If there is a ELF verifier function that walks
>>>>>> every part of an ELF file to verify that the file is sane, and if you can
>>>>>> call that before calling LLD's function, are you guys happy with that?
>>>>>>
>>>>>
>>>>> I'd like to get you guys opinion on this question.
>>>>>
>>>>
>>>> I'd still find it problematic that lld itself would consider
>>>> crash-on-invalid "not a bug" to the point of not reviewing/approving
>>>> patches to fix such issues. That's what I'm concerned about in this thread.
>>>>
>>>
>>> That's one way to see that. The other view is it as a whole has a
>>> boolean option *IsInputTrustworthy* and works accordingly. What matters
>>> most is what we provide to the users as a guarantee.  You have an opinion
>>> that that should be implemented within LLD, but that would now an internal
>>> design choice. Hypothetically we had such pass to verify inputs, and if you
>>> send a patch to "fix" crash bug of LLD, we wouldn't probably reject that
>>> but instead argue that that needs to be addressed in the verifier pass
>>> instead. This is about "how" something should be implemented and usual
>>> design choice discussion, no?
>>>
>>
>> OK, sorry - some confusion. I assume that you wouldn't run this verifier
>> pass by default in lld-the-command-line-tool, right? (I would guess it
>> wouldn't meet your performance criteria)
>>
>
> Correct.
>
>
>> So from that perspective, lld-the-command-line-tool would still be
>> crashing-by-design on certain inputs in its default/normal/user-facing mode
>> & that would be seem problematic to me.
>>
>
> I disagree. You are in almost all case handling valid object files created
> by compilers, and if omitting some error checks for really pathetic case
> would make code simpler and improve performance, I think having that option
> is worth it (and I believe that's the case, at least those who are actually
> writing code seems to take that stance.) Also, if you give broken object
> files, you wouldn't get an output anyways. The only difference that a user
> can observe is whether it dies with an error message or not. We could even
> catch an segfault and run the verifier on the input again to print out an
> error after something goes wrong.
>
> An important point, however, is that not all buffer overruns will cause
> segfaults; some will just corrupt the output. Even running under ASAN won't
> catch everything, because some buffer overruns will simply cause garbage to
> appear within a valid output buffer (or data later copied into one). If we
> could be sure that all such behavior would definitely and immediately cause
> a segfault, I'd feel differently.
>

Any linker may create corrupted outputs that wouldn't run if you give
corrupted inputs because not all errors are detectable by the linker. There
is always a possibility to silently fail if you pass corrupted file no
matter which linker you are using.

>  -Hal
>
>
> If you're proposing having this verifier run by default - sure, then I
>> can't construct an input that crashes the linker, it'll fail with an error
>> message an exit. Yes, that would be fine by me - the way the feature is
>> implemented is not something I mean to imply constraints on.
>>
>> - David
>>
>>
>>>
>>>
>>>>>
>>>>>> On Tue, Mar 22, 2016 at 6:39 PM, Hal Finkel via llvm-dev <
>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> *From: *"David Blaikie via llvm-dev" <llvm-dev at lists.llvm.org>
>>>>>>> *To: *"Rafael Espíndola" <rafael.espindola at gmail.com>
>>>>>>> *Cc: *"llvm-dev" <llvm-dev at lists.llvm.org>, "Bruce Hoult" <
>>>>>>> bruce at hoult.org>
>>>>>>> *Sent: *Tuesday, March 22, 2016 10:18:03 AM
>>>>>>> *Subject: *Re: [llvm-dev] Need help with code generation
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Mar 22, 2016 at 4:27 AM, Rafael Espíndola <
>>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>>
>>>>>>>> > Maybe not, but it's not impossible either - browsers manage to
>>>>>>>> harden themselves against malicious input and they operate in a far hostile
>>>>>>>> environment with many more input formats than we do.
>>>>>>>>
>>>>>>>> It is important to note how different they are. Both Firefox and
>>>>>>>> Chromium have people working just to try to make them more secure.
>>>>>>>> Compare that with LLVM: One week ago I pointed out that your patch
>>>>>>>> (r263521) introduces a crash. It still hasn't been reverted or even
>>>>>>>> acknowledge yet.
>>>>>>>>
>>>>>>>>
>>>>>>>> > I'm not trying to shift your personal goal, or to direct the
>>>>>>>> features that you choose to put your time into, but I am interested in
>>>>>>>> project policy.
>>>>>>>>
>>>>>>>> Why do you care about policy that is not followed? A policy saying
>>>>>>>> llvm should not crash on any input is as relevant as one that says
>>>>>>>> that clang should keep bootstrapping in under one second.
>>>>>>>>
>>>>>>>
>>>>>>> It's pretty different when you say, essentially, that patches to
>>>>>>> address these things are unlikely to be accepted. It doesn't seem
>>>>>>> surprising that people wouldn't try to provide those patches and would
>>>>>>> choose not to use the project if that's the expressed policy of the
>>>>>>> developers on the project and doesn't line up with the needs of other
>>>>>>> people.
>>>>>>>
>>>>>>>
>>>>>>> +1
>>>>>>>
>>>>>>>  -Hal
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> So, if we stick to reality, what we have is that lld (ELF and COFF)
>>>>>>>> are already the most reliable parts of the toolchain. If not for Rui
>>>>>>>> and I being upfront about it most people would not even know that
>>>>>>>> you
>>>>>>>> could crash it. So please, just let us keep working on the most
>>>>>>>> reliable part of the toolchain.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Rafael
>>>>>>>> _______________________________________________
>>>>>>>> LLVM Developers mailing list
>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> LLVM Developers mailing list
>>>>>>> llvm-dev at lists.llvm.org
>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Hal Finkel
>>>>>>> Assistant Computational Scientist
>>>>>>> Leadership Computing Facility
>>>>>>> Argonne National Laboratory
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> LLVM Developers mailing list
>>>>>>> llvm-dev at lists.llvm.org
>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160322/8c341c6c/attachment-0001.html>