[LLVMdev] How to translate library functions into LLVM IR bitcode?

Liwei Yang yangliwei.uestc at gmail.com
Sat Sep 20 19:45:55 PDT 2014 

Hi Johannes,

After reconsideration, I think your guess that the dependencies should be
handled manually is reasonable. Because otherwise, if the dependencies are
resolved automatically as I previously hoped, there might be an issue of
redefinition of functions when multiple top functions share the same
callees from library.

So, forget about the automatic dependence resolving stuff. I think this
question is done and can be closed now.
Again, thanks for the helpful script, Johannes.
Cheers;)

On Sat, Sep 20, 2014 at 5:06 PM, Johannes Doerfert <
doerfert at cs.uni-saarland.de> wrote:

> Hey Liwei,
>
> I inlined two comments but in short: I don't have a solution for your
> automatic dependence gathering problem.
>
> On 09/20, Liwei Yang wrote:
> > Hi Johannes,
> >
> > Actually, I'm working in the same scenario, i.e. configure + make of a
> > benchmark/program/library like you said. I've got your point of using
> this
> > script as a replacement to generate .bc files instead of a executable.
> > That's truly helpful and has already answered my original question.
> >
> > Now I'm actually moving a step further. Take the same example in your
> > reply, say, if I have a bunch of .c files, where a.c calls functions
> > defined in b.c, b.c calls functions defined in c.c and so on. I know
> that I can
> > compile them into a .bc file by using this command:
> > ~/llvm_link -I <HEADER_DIR> a.c b.c c.c d.c -emit-llvm -c -o linked.bc
> >
> > But the problem is all of the files which define the callees need to be
> > specified manually. Is it possible that the source files in which the
> > callees are defined can be automatically inferred, compiled and linked?
> If
> > so, then we can just simply use this command below to include all the
> > callees required by in a.c:
> > ~/llvm_link -I <HEADER_DIR> a.c -emit-llvm -c -o linked.bc
> I don't think this is in general even decidable and I'm not aware of any
> implemented approximation to that problem. If such a thing would exist we
> wouldn't need to write build dependences in Makefiles(.in) by hand any
> more ;)
>
> > Though we can define the dependency for the source files defining callees
> > in makefile, it's still a manual specification there. So that's why I'm
> > looking into this further step.
> My guess is you have to go the extra mile and define all dependences in
> a Makefile yourself.
>
> I hope you'll find a way that works for you,
>   Johannes
>
> > On Sat, Sep 20, 2014 at 10:53 AM, Johannes Doerfert <
> > doerfert at cs.uni-saarland.de> wrote:
> >
> > > Hey Liwei,
> > >
> > > the script was "designed" to act as a compile during "the usual"
> > > configure + make of a benchmark/program/library. If you have such a
> setup
> > > (a configure script which will produce the Makefile file), you can do
> > > something similar to:
> > >
> > > ln -s ~/llvm_link.py ~/llvm_link
> > > ln -s ~/llvm_link.py ~/llvm_link++
> > >
> > > export CC=~/llvm_link
> > > export CXX=~/llvm_link++
> > >
> > > cd <SRC>
> > > ./configure
> > > make
> > >
> > > and it should result in a .bc file where otherwise an e.g., an
> > > executable would have been produced.
> > >
> > > If you just have a bunch of .c files which can be compiled into an
> > > executable with:
> > >
> > > clang -I <HEADER_DIR> a.c b.c c.c d.c -o ex
> > >
> > > you can replace clang by the script and it should give you a ex.bc in
> > > the end.
> > >
> > > Does that help?
> > >
> > > Best regards,
> > >   Johannes
> > >
> > > On 09/20, Liwei Yang wrote:
> > > > Hi Johannes,
> > > >
> > > > By following your directions, I can use your script as is to produce
> the
> > > > .bc file now. Here's my command line for compiling s_sin.c into
> s_sin.bc
> > > > file and the output:
> > > > command line:
> > > > ~/Downloads/newlib-2.1.0/newlib/libm/mathfp ยป python ~/llvm_link.py
> > > s_sin.c
> > > > -I../common/ -I../../libc/include/ -o s_sin.bc
> > > >
> > > > output:
> > > > Initiate CLANG (/path-to-clang):
> > > >      Options: 's_sin.c -I../common/ -I../../libc/include/ -o s_sin.bc
> > > > -emit-llvm -c'
> > > > In file included from s_sin.c:18:
> > > > ./zmath.h:7:9: warning: 'NAN' macro redefined
> > > > #define NAN 2
> > > >         ^
> > > > ../../libc/include/math.h:57:11: note: previous definition is here
> > > > #  define NAN (__builtin_nanf(""))
> > > >           ^
> > > > 1 warning generated.
> > > >      Retcode: 0
> > > >
> > > > I don't know why this warning gets generated since in
> > > > ../../libc/include/math.h:57 the macro NAN is wrapped by ifndef as
> shown
> > > > below, but that's not important.
> > > > # ifndef NAN
> > > > #  define NAN (__builtin_nanf(""))
> > > > # endif
> > > >
> > > >
> > > > I llvm-dis this s_sin.bc file and get a s_sin.ll file as follows.
> > > > ; Function Attrs: nounwind uwtable
> > > > define double @sin(double %x) #0 {
> > > > entry:
> > > >   %x.addr = alloca double, align 8
> > > >   store double %x, double* %x.addr, align 8
> > > >   %0 = load double* %x.addr, align 8
> > > >   %call = call double @sine(double %0, i32 0)
> > > >   ret double %call
> > > > }
> > > >
> > > > declare double @sine(double, i32) #1
> > > >
> > > > Now here comes the point. In this bitcode file, the callee of sin()
> > > > function, i.e. sine(), does not have a function body. I know that the
> > > > definition of sine() is in another .c file, which is s_sine.c in the
> same
> > > > folder. So essentially, I'm wondering if the script can help to
> > > > automatically compile and link all the callees recursively required
> by
> > > > sin() without manually specifying every source file of the callees.
> > > >
> > > > Sorry for coming back late on this topic and thank you so much
> Johannes,
> > > > for sharing your helper script.
> > > >
> > > >
> > > > On Tue, Sep 16, 2014 at 9:00 PM, Johannes Doerfert <
> > > > doerfert at cs.uni-saarland.de> wrote:
> > > >
> > > > > Hey Liwei,
> > > > >
> > > > > I attached a script I used some time back to compile multiple
> source
> > > > > files (of a benchmark) into one bitcode file (instead of an
> > > executable).
> > > > > The script is very simple but worked surprisingly well. It checks
> the
> > > > > command line options for indicators what kind of output is
> expected and
> > > > > produces bitcode files with the appropriate names. In order to use
> it
> > > > > you just need to put/link the script on your path and use it as
> your
> > > > > standard compiler (e.g., export CC=<script_name>) instead of clang.
> > > > > However, clang and llvm-link need to be on the path. If the name
> of the
> > > > > executed script is <script_name>++ (note the ++ in the end) then
> > > clang++
> > > > > will be used to compile the source files, otherwise clang. Also
> note
> > > that
> > > > > the script will remove some command line options as they are not
> > > supported
> > > > > or desired when creating bitcode instead of object code files.
> > > > >
> > > > > It should also be able to pass a usual autoconf configure run by
> > > > > detecting it and simply redirecting the complete command line to
> > > clang(++).
> > > > > But I never used it to link libraries though.
> > > > >
> > > > > I'm not sure if you can use this script as is or as a starting
> point to
> > > > > create your own but maybe you can.
> > > > >
> > > > >
> > > > > Best regards,
> > > > >   Johannes
> > > > >
> > > > >
> > > > >
> > > > > On 09/15, Liwei Yang wrote:
> > > > > > Good tips. Although I have used llvm-link to merge .bc files
> > > together, I
> > > > > > guess -flto could optimize the resultant .bc file further.
> > > > > >
> > > > > > As for the assembly, yes it is an issue. Anyway, I'll try to
> address
> > > > > those
> > > > > > sources which are available for being translated into .bc first.
> > > > > >
> > > > > > Thanks for your advice, Tim.
> > > > > >
> > > > > > On Mon, Sep 15, 2014 at 2:55 PM, Tim Northover <
> > > t.p.northover at gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > > If there's any automated way to infer about all the
> subroutines
> > > that
> > > > > one
> > > > > > > > function needs, clang them into .bc file and link them into a
> > > > > stand-alone
> > > > > > > > .bc library, that will be more than appreciated:-)
> > > > > > >
> > > > > > > If manually compiling the files is working for you, you could
> try
> > > > > > > building the entire library with "-flto" for Link-time
> > > optimisation.
> > > > > > > The output of that will be LLVM IR (if you can convince the
> build
> > > > > > > system to do it for you).
> > > > > > >
> > > > > > > The issue is that parts of the standard library are
> > > > > > > performance-critical and often written in assembly. If the
> entire
> > > file
> > > > > > > is assembly you won't be able to produce IR very easily at all.
> > > > > > >
> > > > > > > Cheers.
> > > > > > >
> > > > > > > Tim.
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best Regards
> > > > > > Liwei
> > > > >
> > > > > > _______________________________________________
> > > > > > LLVM Developers mailing list
> > > > > > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> > > > > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Johannes Doerfert
> > > > > Researcher / PhD Student
> > > > >
> > > > > Compiler Design Lab (Prof. Hack)
> > > > > Saarland University, Computer Science
> > > > > Building E1.3, Room 4.26
> > > > >
> > > > > Tel. +49 (0)681 302-57521 : doerfert at cs.uni-saarland.de
> > > > > Fax. +49 (0)681 302-3065  :
> > > http://www.cdl.uni-saarland.de/people/doerfert
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best Regards
> > > > Liwei
> > >
> > > --
> > >
> > > Johannes Doerfert
> > > Researcher / PhD Student
> > >
> > > Compiler Design Lab (Prof. Hack)
> > > Saarland University, Computer Science
> > > Building E1.3, Room 4.26
> > >
> > > Tel. +49 (0)681 302-57521 : doerfert at cs.uni-saarland.de
> > > Fax. +49 (0)681 302-3065  :
> http://www.cdl.uni-saarland.de/people/doerfert
> > >
> >
> >
> >
> > --
> > Best Regards
> > Liwei
>
> --
>
> Johannes Doerfert
> Researcher / PhD Student
>
> Compiler Design Lab (Prof. Hack)
> Saarland University, Computer Science
> Building E1.3, Room 4.26
>
> Tel. +49 (0)681 302-57521 : doerfert at cs.uni-saarland.de
> Fax. +49 (0)681 302-3065  : http://www.cdl.uni-saarland.de/people/doerfert
>



-- 
Best Regards
Liwei
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140921/4209c296/attachment.html>

More information about the llvm-dev mailing list