[LLVMdev] Should LLVM JIT default to lazy or non-lazy?

Evan Cheng evan.cheng at apple.com
Sun Nov 1 14:27:30 PST 2009 

On Nov 1, 2009, at 12:09 AM, Jeffrey Yasskin wrote:

> A possible patch implementing this is at
> http://codereview.appspot.com/144074
> (http://codereview.appspot.com/download/issue144074_1.diff).
>
> I do NOT think we should accept this patch: It changes a lot of APIs
> and makes users specify the choice in many places, while I think most
> users really just want one choice for their whole app. There's a good
> argument to be made that users may want to decide certain calls should
> be lazily-compiled, but those are boundaries _within_ a transitive
> call tree, not the whole-tree choice this change allows. I'd be happy
> with a forced choice when creating the JIT, or with an optional choice
> defaulted to the safer option, but this patch seems like the wrong
> thing to do.
>
> Evan, you wrote, "there are existing customers out there that are
> using previous llvm release who will not appreciate the API change."
> Who are they? Recall that after my original change they can get back
> to lazy compilation by adding jit->DisableLazyCompilation(false) to
> their code.  They have no need for #ifdefs.

I know of several that are not actively participating in these  
discussions. I don't have knowledge of how they are using LLVM JIT now  
nor do I think that's appropriate for me to say anything. There are  
probably other clients that we don't know about. I guess changing API  
is one way to encourage folks from participating in these discussions.

I honestly don't know what the best solution is. I am leaving it to  
those of you who are actively hacking on the JIT to decide.

Evan

>
> I still think that having a default that's right in nearly all cases
> is the best option (the only concrete counter-example I've seen is
> lli), but I'm happy to write another patch with a global forced choice
> or commit this one if that's what you guys want.
>
> On Thu, Oct 29, 2009 at 11:46 AM, Jeffrey Yasskin  
> <jyasskin at google.com> wrote:
>> Cool, I'll start implementing it.
>>
>> Thanks all for the decision!
>>
>> On Thu, Oct 29, 2009 at 11:03 AM, Evan Cheng <evan.cheng at apple.com>  
>> wrote:
>>> I have no objection to Chris' proposal.
>>>
>>> Evan
>>>
>>> On Oct 29, 2009, at 9:45 AM, Jeffrey Yasskin wrote:
>>>
>>>> Are you objecting to Chris's proposal? I was waiting to implement  
>>>> it
>>>> until you replied so I wouldn't have to implement two things. I
>>>> disagree with a lot of what you wrote below, but it's not worth
>>>> arguing about if there's a compromise we can both live with.
>>>>
>>>> On Thu, Oct 29, 2009 at 12:36 AM, Evan Cheng  
>>>> <evan.cheng at apple.com> wrote:
>>>>> There is nothing here that prevents users from enabling or  
>>>>> disabling lazy
>>>>> JIT. We're talk about flipping the default behavior. That does  
>>>>> not make the
>>>>> API any better or worse. Nor does it fix the inherent thread  
>>>>> safety issue.
>>>>>
>>>>> I can tell you there are no Apple clients that depend on the  
>>>>> lazy JIT.
>>>>> Please do not imply I am being secretive. I am simply busy. My  
>>>>> objection is
>>>>> simple. Changing API simply for the sake of flipping default  
>>>>> behavior does
>>>>> not seem like a good trade off to me. You may not feel the pain,  
>>>>> but there
>>>>> are existing customers out there that are using previous llvm  
>>>>> release who
>>>>> will not appreciate the API change.
>>>>>
>>>>> To me, the decision is simple. People who have done the work in  
>>>>> the past
>>>>> have made their choices. We better respect their choices unless  
>>>>> we are
>>>>> replacing them with something better. Flipping the default isn't  
>>>>> better. If
>>>>> someone wants to sign up to design a new API, that person(s) get  
>>>>> to decide
>>>>> on the default behavior. It's simple as that.
>>>>>
>>>>> For behavior / API change like this, we better extend llvm- 
>>>>> config (or some
>>>>> other means) to expose that information. So the existing clients  
>>>>> can at
>>>>> least ifdef the code.
>>>>>
>>>>> Evan
>>>>>
>>>>> On Oct 28, 2009, at 1:31 PM, Jeffrey Yasskin wrote:
>>>>>
>>>>>> On Wed, Oct 28, 2009 at 12:21 PM, Evan Cheng <evan.cheng at apple.com 
>>>>>> > wrote:
>>>>>>>
>>>>>>> On Oct 28, 2009, at 10:07 AM, Chandler Carruth wrote:
>>>>>>>
>>>>>>>> From where I sit, this boils down to a very simple question  
>>>>>>>> (modulo
>>>>>>>> Chris's point): Either choice will surprise some users. Which  
>>>>>>>> surprise
>>>>>>>> is worse? Personally, I'd always prefer correct but slow  
>>>>>>>> behavior by
>>>>>>>> default, and explicitly enabling dangerous (but in some cases  
>>>>>>>> fast)
>>>>>>>> behavior.
>>>>>>>
>>>>>>> The behavior is only dangerous because people are using it in  
>>>>>>> new and
>>>>>>> different ways.
>>>>>>
>>>>>> I'd point out that reid thought he made the JIT thread-safe in  
>>>>>> r22404
>>>>>> back in 2005. Calling it from multiple threads isn't new and
>>>>>> different, it's just subtly broken. I suggested changing the  
>>>>>> default
>>>>>> because most people who run into this problem don't think they're
>>>>>> doing anything unusual, and in fact their code works fine with  
>>>>>> the
>>>>>> eager JIT. People shouldn't stumble into broken behavior, and  
>>>>>> defaults
>>>>>> are good to prevent stumbling.
>>>>>>
>>>>>> To avoid misconceptions, Unladen Swallow added the line to turn  
>>>>>> off
>>>>>> the lazy jit months ago, and we'll keep that line whatever the
>>>>>> decision is here. We might take advantage of a thread-safe
>>>>>> code-patching facility eventually, but we've been designing  
>>>>>> assuming
>>>>>> nobody else will implement that for us. I favor changing the  
>>>>>> default
>>>>>> because it'll help newbies, not because we need it for anything.
>>>>>>
>>>>>>>> I would also point out that it seems that most of the people  
>>>>>>>> new to
>>>>>>>> the JIT are surprised by the current behavior, where as those  
>>>>>>>> who
>>>>>>>> would be surprised by needing to enable lazy JIT are those long
>>>>>>>> familiar with past behavior. In the OSS world, I always favor  
>>>>>>>> easing
>>>>>>>> adoption over maintaining the status quo.
>>>>>>>
>>>>>>> This argues for better documentation. I'd prefer for EE to  
>>>>>>> abort if user
>>>>>>> is asking for a known dangerous configuration (multi-threaded  
>>>>>>> and lazy).
>>>>>>
>>>>>> I think that'd be a decent fix for the thread-safety problem.  
>>>>>> It'd
>>>>>> require an extra check on each call to runJITOnFunctionUnlocked  
>>>>>> since
>>>>>> laziness is set by a call, not on construction. Or, we could  
>>>>>> use the
>>>>>> JIT lock to assert that it's never entered concurrently. On the  
>>>>>> other
>>>>>> hand, Nicolas Geoffray objected when I suggested that in the bug.
>>>>>>
>>>>>>> The biggest argument I have for staying with lazy is llvm JIT  
>>>>>>> is not a
>>>>>>> light weight JIT. It's designed to do all the codegen  
>>>>>>> optimizations a normal
>>>>>>> static compiler would do. Non-lazy JIT is too slow.
>>>>>>
>>>>>> Óscar used the cost of the JIT as an argument _against_ the  
>>>>>> lazy JIT.
>>>>>> Could you elaborate on why you think it's an argument in favor  
>>>>>> of lazy
>>>>>> JITting?
>>>>>>
>>>>>>> I'd prefer not to change the behavior.
>>>>>>
>>>>>> I'm guessing, based on your vagueness, that there are some  
>>>>>> internal
>>>>>> single-threaded Apple users who think that the lazy JIT is the  
>>>>>> best
>>>>>> performing option for them and who don't want to add a line to  
>>>>>> their
>>>>>> apps overriding the default. But it's hard for me or anyone else
>>>>>> outside Apple to judge whether they ought to drive the default  
>>>>>> without
>>>>>> a little more information about why the lazy JIT is good for  
>>>>>> them.
>>>>>> Could you describe any features of their use that make the lazy  
>>>>>> JIT a
>>>>>> good idea for them?
>>>>>>
>>>>>>> If we want to start using it in new and interesting ways, we  
>>>>>>> should just
>>>>>>> design a new JIT.
>>>>>>
>>>>>>>> My meager 2 cents.
>>>>>>>> -Chandler
>>>>>>>>
>>>>>>>> On Wed, Oct 28, 2009 at 9:41 AM, Jeffrey Yasskin <jyasskin at google.com 
>>>>>>>> >
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> In r85295, in response to the discussion at http://llvm.org/PR5184
>>>>>>>>> (Lazy JIT ain't thread-safe), I changed the default JIT from  
>>>>>>>>> lazy to
>>>>>>>>> non-lazy. It has since come to my attention that this may  
>>>>>>>>> have been
>>>>>>>>> the wrong change, so I wanted to ask you guys.
>>>>>>>>>
>>>>>>>>> A couple reasons to make the default non-lazy compilation:
>>>>>>>>> * The lack of thread-safety surprises new users
>>>>>>>>> * Crashes due to this will be rare and so hard to track down
>>>>>>>>> * The current lazy scheme is almost never the right answer for
>>>>>>>>> performance
>>>>>>>>> * It's only one line of code to turn on lazy compilation  
>>>>>>>>> when it is
>>>>>>>>> the right answer for you.
>>>>>>>>>
>>>>>>>>> And a couple to default to lazy compilation:
>>>>>>>>> * It's safe for single-threaded code.
>>>>>>>>> * There are existing users who have assumed this default.
>>>>>>>>> * PPC and ARM don't support non-lazy compilation yet (the  
>>>>>>>>> tests
>>>>>>>>> currently run the lazy jit).
>>>>>>>>> * Gratuitous changes are bad.
>>>>>>>>>
>>>>>>>>> Thoughts?
>>>>>>>>>
>>>>>>>>> We can choose the default for lli separately from the JIT's  
>>>>>>>>> default if
>>>>>>>>> we want.
>>>>>>>>> _______________________________________________
>>>>>>>>> LLVM Developers mailing list
>>>>>>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> LLVM Developers mailing list
>>>>>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>>

More information about the llvm-dev mailing list