[LLVMdev] Should LLVM JIT default to lazy or non-lazy?

[Jeffrey Yasskin]

On Wed, Oct 28, 2009 at 12:21 PM, Evan Cheng <evan.cheng at apple.com> wrote:
>
> On Oct 28, 2009, at 10:07 AM, Chandler Carruth wrote:
>
>> From where I sit, this boils down to a very simple question (modulo
>> Chris's point): Either choice will surprise some users. Which surprise
>> is worse? Personally, I'd always prefer correct but slow behavior by
>> default, and explicitly enabling dangerous (but in some cases fast)
>> behavior.
>
> The behavior is only dangerous because people are using it in new and different ways.

I'd point out that reid thought he made the JIT thread-safe in r22404
back in 2005. Calling it from multiple threads isn't new and
different, it's just subtly broken. I suggested changing the default
because most people who run into this problem don't think they're
doing anything unusual, and in fact their code works fine with the
eager JIT. People shouldn't stumble into broken behavior, and defaults
are good to prevent stumbling.

To avoid misconceptions, Unladen Swallow added the line to turn off
the lazy jit months ago, and we'll keep that line whatever the
decision is here. We might take advantage of a thread-safe
code-patching facility eventually, but we've been designing assuming
nobody else will implement that for us. I favor changing the default
because it'll help newbies, not because we need it for anything.

>> I would also point out that it seems that most of the people new to
>> the JIT are surprised by the current behavior, where as those who
>> would be surprised by needing to enable lazy JIT are those long
>> familiar with past behavior. In the OSS world, I always favor easing
>> adoption over maintaining the status quo.
>
> This argues for better documentation. I'd prefer for EE to abort if user is asking for a known dangerous configuration (multi-threaded and lazy).

I think that'd be a decent fix for the thread-safety problem. It'd
require an extra check on each call to runJITOnFunctionUnlocked since
laziness is set by a call, not on construction. Or, we could use the
JIT lock to assert that it's never entered concurrently. On the other
hand, Nicolas Geoffray objected when I suggested that in the bug.

> The biggest argument I have for staying with lazy is llvm JIT is not a light weight JIT. It's designed to do all the codegen optimizations a normal static compiler would do. Non-lazy JIT is too slow.

Óscar used the cost of the JIT as an argument _against_ the lazy JIT.
Could you elaborate on why you think it's an argument in favor of lazy
JITting?

> I'd prefer not to change the behavior.

I'm guessing, based on your vagueness, that there are some internal
single-threaded Apple users who think that the lazy JIT is the best
performing option for them and who don't want to add a line to their
apps overriding the default. But it's hard for me or anyone else
outside Apple to judge whether they ought to drive the default without
a little more information about why the lazy JIT is good for them.
Could you describe any features of their use that make the lazy JIT a
good idea for them?

> If we want to start using it in new and interesting ways, we should just design a new JIT.

>> My meager 2 cents.
>> -Chandler
>>
>> On Wed, Oct 28, 2009 at 9:41 AM, Jeffrey Yasskin <jyasskin at google.com> wrote:
>>> In r85295, in response to the discussion at http://llvm.org/PR5184
>>> (Lazy JIT ain't thread-safe), I changed the default JIT from lazy to
>>> non-lazy. It has since come to my attention that this may have been
>>> the wrong change, so I wanted to ask you guys.
>>>
>>> A couple reasons to make the default non-lazy compilation:
>>> * The lack of thread-safety surprises new users
>>> * Crashes due to this will be rare and so hard to track down
>>> * The current lazy scheme is almost never the right answer for performance
>>> * It's only one line of code to turn on lazy compilation when it is
>>> the right answer for you.
>>>
>>> And a couple to default to lazy compilation:
>>> * It's safe for single-threaded code.
>>> * There are existing users who have assumed this default.
>>> * PPC and ARM don't support non-lazy compilation yet (the tests
>>> currently run the lazy jit).
>>> * Gratuitous changes are bad.
>>>
>>> Thoughts?
>>>
>>> We can choose the default for lli separately from the JIT's default if we want.
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>