[llvm-dev] Redefining optnone to help LTO

Mon Jan 16 11:17:23 PST 2017

> On Jan 16, 2017, at 11:02 AM, Sean Silva <chisophugis at gmail.com> wrote:
> 
> 
> 
> On Mon, Jan 16, 2017 at 10:22 AM, Mehdi Amini <mehdi.amini at apple.com <mailto:mehdi.amini at apple.com>> wrote:
> 
>> On Jan 16, 2017, at 10:00 AM, Sean Silva via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>> What is the end goal? If the goal is roughly "if a user passes -O0 when compiling a TU in LTO mode, their final binary should have functions that look like the result of -O0 noLTO compilation", then the frontend can just emit a normal -O0 object file I think. What is the advantage of passing bitcode all the way to the linker only to jump through hoops to prevent LTO code generation from modifying it?
> 
> That’s a good point, that addresses the “I want LTO for optimization”, which seems contradictory to use `-flto -O0`, and could even advocate for forbidding (or warning).
> 
> However LTO has other uses than optimizations: for instance instrumentations that needs full program access.
> One of them is CFI: http://clang.llvm.org/docs/ControlFlowIntegrity.html <http://clang.llvm.org/docs/ControlFlowIntegrity.html>
> If you want to debug and rebuild only part of the program, CFI still requires to use LTO, IIUC.
> 
> That's a good point. I wonder if there are any commonalities of this problem and the "hosted"/"freestanding" issues recently? These all seem to tie into a common theme of "when I use LTO, various per-TU settings don't make it to the LTO code generator"; can we adopt a uniform solution for this class of problems, like always using function attributes or something? This probably won't be the last of such issues, and we should have a "standard solution" for them.

Yes, the optnone thing is indeed part of the work to bring “everything” needed to setup TLI, CodeGen and so on into function attributes.
I was working on the TLI and "pulling strings" I ended-up starting with optnone.

> 
> Taking a step back, consider the "trend" as we try to persist more per-TU options to LTO: we will have more and more attributes (or whatever) telling the optimizer and code generator what to do in greater and greater detail.
> In such a world, what is the role of the frontend setting up the pass pipeline, target info, etc. using calls into the LLVM libraries? If, for LTO, we have to serialize those things anyway, then should frontends prefer to simply add the annotations into the IR instead of making calls into the LLVM libraries to configure the code generation?

Yes, as much as possible. This is inline with the work Eric has been doing about having subtargets selected/configured with function attributes. We’re also trying to get rid of global flags in SelectionDAG or other phases of the codegen in favor of function attributes or individual instruction flags for Fast-Math.

> Things like -mllvm options suggest that we're never really going to persist "everything affecting codegen" into the IR on a per-TU basis during LTO (can't control -mllvm options per-function).

I believe -mllvm options are supposed to be “developer options” only and are not “supported” or supposed to be exposed to the end user. For example I don’t believe the clang docs is referencing the -mllvm, right?

> So is our approach here basically to persist compilation options into the IR on an as-needed (i.e. ad-hoc) basis? E.g. we go out of our way to persist -O0 (using e.g. optnone) but don't do anything special for -O1. Also what is the interaction with --lto-O[0123]? (i.e. linker options controlling the optimization level used during LTO codegen)

The discussion in the revision addressed this, but in particular the last comment of Chandler in the revision should answer this somehow: https://reviews.llvm.org/D28404 <https://reviews.llvm.org/D28404>

"Unlike the differences between -O[123], all of -O0, -Os, and -Oz have non-threshold semantic implications. So with this change, I think we will have *all* the -O flags covered, because I view '-O[123]' as a single semantic space with a threshold modifier that we *don't* need to communicate to LTO. We model that state as the absence of any attribute. And -O0, -Os, and-Oz have dedicated attributes.
If we ever want to really push on -Og, that might indeed require an attribute to distinguish it."

I reproduced above only one sentence, but I encourage to read the full comment :)

> I'm just trying to understand the bigger picture here.

It’s not always totally clear to me either :)
Some options are making sense “per TU” while others can make sense “per-function” as well! 
This can be inherent to what the option describes, or because the user interface is not uniform:
- for some options, the interface is both the command line and the source code attributes like `optnone`, `noinline`.
- for other options only the command line is available, like `freestanding`, `nobuiltin` (not sure for this one?)
- and some options are only available in the source: disabling the sanitizer for example.

It changes over time as well, at some point the subtargets options were a property of the TU, while it has been promoted to a per-function attribute.

Finally, this is also a very “clang-centric” vision of the user-interface, other client of LLVM can bring more into the mix.

I have the impression that function attributes when possible seems the more flexible for all the clients overall.

— 
Mehdi

> 
> Personally, my mental model has always been that the flags that you pass to per-TU compilation are instructions for that compilation, and should not influence things like optimization level for LTO code generation (which will run at a different time in a different program). That's at least easy to document.
> 
> -- Sean Silva
>  
> 
> — 
> Mehdi
> 
> 
> 
>> 
>> -- Sean Silva
>> 
>> On Wed, Jan 11, 2017 at 8:34 AM, Robinson, Paul via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> In D28404, Mehdi wanted to use the 'optnone' attribute as a way to record
>> "I was compiled with -O0" in the IR, because it seems like a good idea to
>> remember that fact in an LTO compilation and there is no way to remember
>> that fact currently.  A couple of people felt it might be better to have
>> this idea discussed on the dev list, where it might get better exposure,
>> so I'm volunteering to get that discussion started.
>> 
>> While 'optnone' does cause lots of optimizations to bypass a function,
>> exactly matching -O0 was not the motivation and never a hard requirement.
>> The implementation makes a distinct effort to get close to the behavior
>> of -O0, but it's not an exact match and for the intended purpose (allowing
>> a given function to be un-optimized to help debugging) it worked fine.
>> 
>> Using 'optnone' to convey -O0 to LTO is something of a redefinition, or
>> at least a re-purposing, of the attribute.  To get there from here, I
>> think we would need a couple of things to happen, separately from the
>> minor grunt work of adding 'optnone' to function IR at -O0.
>> 
>> 1) Update the LangRef definition of 'optnone' to reflect this intent.
>> The current definition doesn't provide a motivation, and the description
>> is (deliberately) a bit vague.  If we want 'optnone' to intentionally
>> match -O0, that should be tightened up.
>> 
>> 2) Make a concerted effort to teach 'optnone' to targets.  Currently
>> I know the X86 target is aware of it, but I'm not so sure about others.
>> 
>> 3) Take another look at what 'optnone' currently does *not* turn off,
>> and see if there is something we can do about that.  In some cases this
>> will not be practical, and we may just have to live with that.
>> 
>> (Okay, we need 3 things to happen.)
>> 
>> I won't say this is blocking Mehdi's work, but it would remove a
>> point of contention and allow the review to proceed more smoothly.
>> --paulr
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170116/37de1357/attachment.html>