[LLVMdev] LLVM versus Intel's PIN tool

[duraid at octopus.com.au]

Hi Vasanth,

Can you be a little more precise about what you mean by "dynamic
optimization"? In one sense, neither LLVM nor Pin are dynamic optimization
tools per se - LLVM is more of a compiler(-building) suite while Pin is an
instrumentation toolkit.

> 1. I should be able to run all the Spec 2000 and Spec 95 floating point
> and integer benchmarks.

LLVM will do this - if it doesn't (on your particular platform) then
that's a bug that can and will be fixed. :) Pin just runs whatever
binaries you have managed to build (perhaps even using LLVM.)

> 2. I should be able to instrument the beginning and end of specific
> program regions (e.g., functions, loops) with calls to subroutines that I
> write.

OK. You could do this with the LLVM JIT - easily if you want to patch the
heads and tails of functions, a little more work (but not too much) to get
at specific loops. With Pin you won't have any issues, but either way
you'll be writing a bit of code.

> 3. I need to be able to patch the binary code of an executing program.
> Specifically, I want to be able to insert NOPs (into functions and
> loops) at compile time that I patch into calls to subroutines at runtime.

Do you really need to insert NOPs and then write over them? Can't you just
replace calls to functions with calls to new functions that in turn call
whatever subroutines you like?

> My questions are:
>
> A. The testresults at http://llvm.cs.uiuc.edu/testresults/X86/ show that
> several benchmarks (swim, applu, tomcatv, su2cor...) in the Spec CFP 2000
> and 95 suites failed when run under the JIT. Why did they fail and what
> are the limitations in running Spec benchmarks?

I'm not sure why these don't run in the JIT, actually. They should. 
(Anyone?)

> B. Is what I am suggesting in 3 possible in LLVM?

You should be able to achieve the same effect, but I'm not sure if it will
actually involve overwriting NOPs (if for no other reason than an
optimizing compiler like LLVM doesn't emit NOPs lightly! ;)

> C. Has anyone here used both LLVM and Intel's PIN? If so, can you
> comment on the drawbacks of using one over the other? I am especially
> interested in any comparisons of their performance and ease-of-use.

I have used LLVM and the old (Itanium-only) version of Pin. It's hard to
compare these directly as they're quite different things. One can't really
compare their performance as they do different things: LLVM is an
optimizing compiler, Pin is an instrumentation tool. As for ease-of-use,
I'd say they're both reasonably easy to use, but it really depends on what
you want to do. If by "dynamic optimization" you are actually thinking of
*optimizing* (higher-level) code, LLVM is probably the way to go. If you
are more interested in *binary* reoptimization, however, you could
certainly use Pin to help you find and characterize interesting hot spots
but beyond that you'll be on your own.

       Hope this helps,

       Duraid

P.S. If you are feeling insane, you can probably figure out some fun way
to use both LLVM *and* Pin, e.g. you could turn LLVM into a Pin tool with
callbacks that trigger aggressive LLVM optimizations/register
allocators/whatever on hot functions.