[LLVMdev] MultiSource/Applications/lemon slow in JIT

[Török Edwin]

Evan Cheng wrote:
> On May 21, 2008, at 6:09 AM, Matthijs Kooijman wrote:
>
>   
>> Hi,
>>
>> I've been toying around with the testsuite for a bit, and after  
>> recompiling
>> llvm-gcc a bunch of times and fixing a nasty bashism bug, I actually  
>> got it to
>> run and pass most tests.
>>
>> When running, I noticed a very long runtime on the lemon test. At  
>> first I
>> suspected an infinite loop, but it turned out the test simply needed  
>> around
>> 1000 seconds to run. Some investigation turned out that the lemon  
>> program does
>> a lot of forking (it runs every input file 20 times, so that's  
>> around 200
>> forks). However, since the forking happens quite early in the main  
>> function,
>> it happens before most of the program is JITted. So, every one of  
>> these forked
>> process is taking a few seconds to compile the other needed  
>> functions, over
>> and over again.
>>
>> A solution for this problem seems to be to run the lemon test with the
>> -no-lazy option, so things get compiled before forking. In my case,  
>> this
>> reduced runtime from 1000+ seconds to only 16 seconds.
>>
>> Intrestingly enough, I can't seem to find similar behaviour in the  
>> test
>> results on llvm.org/nightlytest (though I only checked two machines  
>> [since the
>> results take ages to load...] of which
>> only one actually ran the test). Am I the only one that is  
>> experiencing this?
>>
>> Anyhow, the below patch fixes the problem for me, good to commit?
>>     
>
> Ouch. I didn't notice this. I don't think passing this option is the  
> right fix though. Any test that forks a lot is simply not a good  
> compiler benchmark. The right thing to do is to eliminate the forking.  
> Is that cannot be easily done. We should remove it from our test suite.
>   

The forking was introduced because the runtime was too small; the
program keeps global state, so there is no easy way to reset state.
I would prefer to remove the forking (and get short runtimes again),
rather than removing it completely from the testsuite.
What do you think?

Best regards.
--Edwin