[LLVMdev] PATH and LD_LIBRARY_PATH

[Reid Spencer]

All,

After reviewing the feedback on this email (thanks!) and discussing it
on IRC we've decided to opt for flexibility but still provide some help,
as follows:

     1. Environment variables are the developer's business and we'll
        dictate no requirements here. Building llvm will work regardless
        of your environment variables.
     2. The "make install" approach will be supported throughout. If you
        want to roll your own environments, you can always do that.
        Several people preferred this approach and whatever we do you
        will still be able to "make install" and set your path
        accordingly.
     3. We will be providing a "support" module (its nearly ready) that
        will contain fundamental configuration (autoconf) scripts,
        system and support libraries, and the build system (makefiles).
        This will work in such a way that no environment variables are
        required. Where paths are needed, there will be configure script
        options to specify them.
     4. The "llvm-top" module is also nearly ready for use.  This module
        is aimed at making it "dead simple" to checkout, build and
        install llvm software. It also sets up all the llvm modules in a
        single directory making it easier to create single directory
        environments. While its use will always be optional, it is also
        strongly recommended, especially for novices.
     5. We will provide some shell script functions to make switching
        your llvm-top based environments easier. These are entirely
        optional.

Thanks for the feedback.  I'll keep you posted on the progress of this
stuff. Hopefully, LLVM gets much easier to deal with in the near future.

Reid.


On Thu, 2007-07-19 at 15:09 -0700, Reid Spencer wrote:
> All,
> 
> With the pending reorganization of the software, I have some questions
> about how developers set their PATH and LD_LIBRARY_PATH variables when
> working with LLVM. This is a bit long winded, but bear with me.
> 
> We're planning to break the "llvm" module up into three modules:
> 
>       * support - lib/Support, lib/System, autoconf, make support,
>         utilities
>       * core - VMCore, Asm, Bitcode and the essential IR tools (llvm-as,
>         etc.)
>       * opt (not sure that's the final name) - everything else:
>         Analysis, Transforms, CodeGen, Target, etc
> 
> Additionally, there are new modules such as "hlvm", "cfe",
> "llvm-gcc-4.2" and undoubtedly more to come in the future. 
> 
> We haven't decided the final architecture so don't quibble about what
> goes in what module (yet). The point is, there will be several modules
> instead of everything being in "llvm". With this situation we can no
> longer just put llvm/Debug/bin in PATH and llvm/Debug/lib in
> LD_LIBRARY_PATH and just have things work. Build products would be
> places in the Debug/bin and Debug/lib directories for each module.
> 
> We'd like to better support multiple LLVM environments on a given
> machine.  An environment being "a directory and the associated
> environment variable settings to support software development". I
> personally have worked this way for about a year now. It helps keep
> various works in progress separated. Obviously, having multiple
> environments requires a certain amount of bookkeeping to keep things
> straight.  I use shell functions like "llvmco" and "llvmenv" to checkout
> and switch to particular environments. The functions take care of the
> details.  
> 
> However, even with only a single checkout (environment) of llvm
> software, there are details to be taken care of. We would like to
> support this better, but the question is how.
> 
> Here are some of the issues:
>       * On some platforms you set SHLIB_PATH or SHOBJ_PATH, etc.
>       * With more modules the PATH and LD_LIBRARY_PATH become long (one
>         entry per module). Having every module's Debug/bin in PATH and
>         Debug/lib in LD_LIBRARY_PATH gets hard to maintain when there's
>         multiple environments. Furthermore, the paths need to change
>         when you switch to a release or release+asserts or release
>         +expensive_checks build.
>       * There are inter-dependencies between modules which may affect
>         the relative ordering of the PATH and LD_LIBRARY_PATH component
>         paths.
>       * Building things can be affected because if you put the wrong
>         directory in your LD_LIBRARY_PATH you can end up linking against
>         libraries built by the compiler instead of your platform's
>         native compiler, which will ultimately fail (very late too).
>       * Having two llvm-gcc versions (4.0 and 4.2) in separate modules
>         could lead to conflicts.
>       * Upstream projects like hlvm and cfe will have several
>         dependencies so getting the paths straight is important for
>         successful building.  Additionally, users will have their own
>         project directories, at the top of the food chain, which are
>         dependent on everything.
>       * We want to treat each module, as much as possible, as a separate
>         entity (very loose coupling), but they are API locked anyway and
>         we can't do much about that. The dependencies are real.
>       * There are utilities that we want in the paths (like llvm/utils)
>         as well as utilities like TableGen that might eventually be
>         needed across projects (e.g. "core" would need TableGen for the
>         intrinsic functions but the module containing the targets also
>         needs it). 
>       * Does every module need its own "llvm-config" program?
>       * Some of us have multiple things going on at the same time and so
>         work with multiple LLVM environments. For example, you could be
>         working on an involved bug fix, your normal development work,
>         quickie fixes, a branch for some side work, etc. In each of
>         these cases you want a separate checkout and the associated
>         environment variable settings for that directory. I call this an
>         "environment". It is basically just a way to keep various works
>         in progress separated. How can multiple environment be best
>         supported?
> 
> So, the question is .. what do you want to do about all this?
> 
> Here are some options to be discussed:
> 
>      1. Punt - Let each developer/user figure this out for the
>         themselves.
>      2. Install - That is, set your PATH and LD_LIBRARY_PATH to one
>         place and "make install" the build results into that directory.
>      3. Shell - Provide some shell functions and aliases to manage
>         setting the environment   correctly. This could even use the
>         ModuleInfo.txt file to glean dependencies. For example,the
>         llvm-top module could have a "setenv.sh" scrip that is invoked
>         with ". ./setenv.sh"to set the environment for whatever is
>         checked out in that llvm-top. We'd need one for each type of
>         shell and users would have to remember to run it.
>      4. Other - got any other ideas?
> 
> I need help with #4 but I'm also looking for general feedback on solving the issues raised.
> 
> Reid.
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev