[lldb-dev] Difficulty in using target.source-map to map source files

karnajit wangkhem via lldb-dev lldb-dev at lists.llvm.org
Mon Aug 28 22:10:34 PDT 2017


Thanks for the info. I will keep that in mind.

On Mon, Aug 28, 2017 at 11:05 PM, Jim Ingham <jingham at apple.com> wrote:

> That will be great!  Last time we were more aggressive about normalizing
> paths we caused a decent performance regression by normalizing all the
> paths every time we did a file-name compare for setting breakpoints.  If
> you look at FileSpec::Equal we added some "is it worth doing this work"
> preparatory work.  I don't know where you'll find the additional places
> this needed to be done, but it's worth keeping an eye out for not putting
> GetNormalizedPath on a hot path.
>
> Jim
>
>
>
> > On Aug 28, 2017, at 1:19 AM, karnajit wangkhem <karnajitw at gmail.com>
> wrote:
> >
> > Thanks Jim for the help. I would definitely like to work on this PR soon.
> >
> > Filed the PR here
> > https://bugs.llvm.org/show_bug.cgi?id=34341
> >
> > Regards,
> > Karan
> >
> > On Thu, Aug 24, 2017 at 11:17 PM, Jim Ingham <jingham at apple.com> wrote:
> >
> > > On Aug 24, 2017, at 10:06 AM, karnajit wangkhem <karnajitw at gmail.com>
> wrote:
> > >
> > > Hi Jim,
> > >
> > > Thanks for the valuable reply. Please find my comments inline.
> > >
> > > Regards,
> > > Karan
> > >
> > > On Wed, Aug 23, 2017 at 12:32 AM, Jim Ingham <jingham at apple.com>
> wrote:
> > >
> > > > On Aug 21, 2017, at 12:02 PM, karnajit wangkhem via lldb-dev <
> lldb-dev at lists.llvm.org> wrote:
> > > >
> > > > Hi All,
> > > >
> > > > Below is a sample example, where target.source-map seems to have a
> limitation. The limitation seems to be because
> > > > 1. lldb does not have an equivalent command like directory in gdb
> > >
> > > The gdb "dir" command seemed always to be more annoying than useful in
> real situations.  If your project had any complexity in the directory
> structure you ended up having to add dir commands for all the
> subdirectories, which got tedious quickly.  Since you pretty much always
> move your sources around rigidly, "source maps" are a more natural way to
> specify this.
> > >
> > > Note, we could add a recursive "dir" command, but you can't do the
> simpleminded recursive search or you'll mess up when the sources have same
> named files in different directories.  Because of this, again the source
> maps seem more suited.
> > >
> > >
> > > -- Really a good info to know. Thanks!
> > >
> > > > 2. target.source-map does not accept multiple mapping entries. I
> checked this only with freebsd.
> > > >
> > > > (lldb) settings show target.source-map
> > > > target.source-map (path-map) =
> > > > [0] "/home/karnajitw/lldb_test_execs/test_source_line1" ->
> "/u/test/test_source_line1"
> > >
> > > That is not correct:
> > >
> > > (lldb) settings set target.source-map /some/source/path /tmp
> /some/other/source/path /tmp
> > > (lldb) settings show target.source-map
> > > target.source-map (path-map) =
> > > [0] "/some/source/path" -> "/tmp"
> > > [1] "/some/other/source/path" -> "/tmp"
> > >
> > > or
> > >
> > > (lldb) set set target.source-map /some/source/path /tmp
> > > (lldb) set append target.source-map /some/other/source/path /tmp
> > > (lldb) set show target.source-map
> > > target.source-map (path-map) =
> > > [0] "/some/source/path" -> "/tmp"
> > > [1] "/some/other/source/path" -> "/tmp"
> > >
> > > - Thanks! This helped.
> > >
> > > >
> > > > 3. Haven't checked in the code yet, but if we see the mappings of
> scenario 1, they all point to a single real path /home/karnajitw/lldb_test_execs/test_source_line1.
> But looks like the mapping logic only considers strings into account. But,
> at the same time, I am not claiming that they should be interpreted as path
> from a different machine during the mapping.
> > > >
> > >
> > > I'm not sure what you mean here.  lldb does what it can to unwind the
> source path, but it doesn't assume that those paths actually exist locally,
> so the most it can do is remove redundant "/"-s and "/./"-s and unwind
> ".."-s.  We do this before most source path comparisons (it's part of the
> FileSpec Equals method).  We've been tinkering with this over time so make
> sure you are using a recent version of lldb.  If there are places where we
> don't do this, then that's easily fixed.
> > >
> > >
> > > - Your answer is in line with what I wanted to know. My lldb should be
> quite recent. But I used it only for freebsd currently.
> > >
> > > % ./lldb --version
> > > lldb version 5.0.0 (http://llvm.org/svn/llvm-project/lldb/trunk
> revision 305778)
> > >   clang revision 305546
> > >   llvm revision 305548
> > >
> > >
> > > > I want to check on this issue in-depth. But before that, want to
> confirm if this is real issue or there are other ways to deal these
> scenarios which I am not aware of?
> > > >
> > > > I am referring below link for the lldb commands.
> > > > https://lldb.llvm.org/lldb-gdb.html
> > > >
> > > > 1. First scenario: Different souce file path representation
> > > >
> > > > [/home/karnajitw/lldb_test_execs/test_source_line1] $ clang -O0 -g
> /home/karnajitw/lldb_test_execs///test_source_line1/main.c
> /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_source_line1/a/ainc.c
> /home/karnajitw/lldb_test_execs/test_source_line1/a/b/binc.c
> > > >
> > > > Machine 1: /home/karnajitw/lldb_test_execs/test_source_line1 ->
> Machine 2: /u/test/test_source_line1
> > > >
> > > > test_source_line1
> > > > |-- a
> > > > |   |-- ainc.c
> > > > |   |-- ainc.h
> > > > |   `-- b
> > > > |       |-- binc.c
> > > > |       `-- binc.h
> > > > |-- a.out
> > > > `-- main.c
> > > >
> > > > % ./lldb test_source_line1/a.out
> > > >
> > > > (lldb) target create "test_source_line1/a.out"
> > > > Current executable set to 'test_source_line1/a.out' (x86_64).
> > > > (lldb) l main
> > > > File: /home/karnajitw/lldb_test_execs///test_source_line1/main.c
> > > > (lldb) l afn
> > > > File: /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_
> source_line1/a/ainc.c
> > > > (lldb) l bfn
> > > > File: /home/karnajitw/lldb_test_execs/test_source_line1/a/b/binc.c
> > > >
> > > > (lldb) settings set target.source-map /home/karnajitw/lldb_test_execs///test_source_line1
> /u/test/test_source_line1
> > > >
> > > > (lldb) l main
> > > > File: /home/karnajitw/lldb_test_execs///test_source_line1/main.c
> > > >    1    #include "a/ainc.h"
> > > >    2
> > > >    3    int main()
> > > >    4    {
> > > >    5      afn();
> > > >    6
> > > >    7      bfn();
> > > >    8
> > > >    9      return 0;
> > > >    10   }
> > > > (lldb) l afn
> > > > File: /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_
> source_line1/a/ainc.c
> > > > (lldb) l bfn
> > > > File: /home/karnajitw/lldb_test_execs/test_source_line1/a/b/binc.c
> > > >
> > > > (lldb) settings set target.source-map /home/karnajitw/lldb_test_
> execs/../lldb_test_execs/test_source_line1 /u/test/test_source_line1
> > > >
> > > > (lldb) l main
> > > > File: /home/karnajitw/lldb_test_execs///test_source_line1/main.c
> > > > (lldb) l afn
> > > > File: /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_
> source_line1/a/ainc.c
> > > >    1    #include <stdio.h>
> > > >    2    #include "ainc.h"
> > > >    3
> > > >    4    void afn()
> > > >    5    {
> > > >    6      printf("Hello this is afn...\n");
> > > >    7    }
> > > > (lldb) l bfn
> > > > File: /home/karnajitw/lldb_test_execs/test_source_line1/a/b/binc.c
> > > >
> > > > (lldb) settings set target.source-map /home/karnajitw/lldb_test_execs/test_source_line1
> /u/test/test_source_line1
> > > >
> > > > (lldb) l main
> > > > File: /home/karnajitw/lldb_test_execs///test_source_line1/main.c
> > > > (lldb) l afn
> > > > File: /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_
> source_line1/a/ainc.c
> > > > (lldb) l bfn
> > > > File: /home/karnajitw/lldb_test_execs/test_source_line1/a/b/binc.c
> > > >    1    #include <stdio.h>
> > > >    2    #include "binc.h"
> > > >    3
> > > >    4    void bfn()
> > > >    5    {
> > > >    6      printf("Hello this is bfn...\n");
> > > >    7    }
> > >
> > > I can't test the "///" part of this, clang seems to always collapse
> these for me.  But with a recent lldb, I see:
> > >
> > > $ clang -g -O0 -c '/tmp/sources/build/../files/hello.c'
> > > $ dwarfdump hello.o
> > > ...
> > > 0x0000000b: TAG_compile_unit [1] *
> > >              AT_producer( "Apple LLVM version 9.0.0 (clang-900.0.31)" )
> > >              AT_language( DW_LANG_C99 )
> > >              AT_name( "/tmp/sources/build/../files/hello.c" )
> > >
> > > So we did get a compile unit name with .. in it, but:
> > >
> > > lldbrulez:/tmp/sources/build > clang -g -O0 -o hello hello.o
> > > lldbrulez:/tmp/sources/build > lldb
> > > (lldb) file hello
> > > Current executable set to 'hello' (x86_64).
> > > (lldb) source list -n main
> > > File: /tmp/sources/build/../files/hello.c
> > >    1    #include <stdio.h>
> > >    2
> > >    3    int
> > >    4    main()
> > >    5    {
> > >    6      printf ("Hello there.\n");
> > >    7      return 0;
> > >    8    }
> > >
> > > So we do unwind the ..'s in this case.  This example doesn't have
> source-maps, but the same thing works if I move the sources and add just a
> <build-top> -> <debug-top> mapping.  As I say, we have been working these
> ".." aware comparisons through all the file compares recently, so you may
> just need to get a newer lldb.
> > >
> > > - Lets say in my above example, the function is this way
> > > main -> afn  -> bfn
> > >
> > > Now, I breakpoint on bfn, now to get source list on all the 3 frames,
> I will need this below mapping
> > >
> > > (lldb)
> > > target.source-map (path-map) =
> > > [0] "/home/karnajitw/lldb_test_execs///test_source_line1" ->
> "/u/test/test_source_line1"
> > > [1] "/home/karnajitw/lldb_test_execs/../lldb_test_execs/test_source_line1"
> -> "/u/test/test_source_line1"
> > > [2] "/home/karnajitw/lldb_test_execs/test_source_line1" ->
> "/u/test/test_source_line1"
> > >
> > > Like the previous discussion, All the LHS and RHS actually refers to
> the same path. So why we need 3 mappings here? But, as I have mentioned
> previously the problem could be just in the freebsd version also.
> > >
> > >
> > > >
> > > > 2. Scenario 2: Deep directory structure
> > > >
> > > > <top>/obj/a/b/c/d/e/app/sub/../../../../../../../../src/a/b/
> c/d/e/app/sub/file
> > > > <top>/obj/a/b/c/d/e/app/../../../../../../../src/a/b/c/d/e/app/file
> > > >
> > > > - If we carry the copy the source file to machine 2, we cannot
> easily map the source file without creating dummy <top>/obj/a/b/c/d/e/f/g.
> > > >
> > > You should only need to map <top> to <top>.  The only reason you would
> have to start specifying subdirectories is you have ones specified with
> ".."-s, but that problem is properly handled by making the file comparisons
> aware of backup operators.  So if the comparisons are working right (again
> check a more recent lldb) you should not need to deal with all these dots.
> > >
> > >
> > > - Checked with latest lldb build, its the same
> > >
> > > (lldb) version
> > > lldb version 6.0.0 (http://llvm.org/svn/llvm-project/lldb/trunk
> revision 311075)
> > >   clang revision 311072
> > >   llvm revision 311071
> > >
> > > $ build/bin/lldb a.out
> > > (lldb) target create "a.out"
> > > Current executable set to 'a.out' (x86_64).
> > > (lldb) settings set target.source-map /home/karnajitw/lldbtest/obj/
> a/b/c/d/e/app/../../../../../../../src/a/b/c/d/e/app /home/karnajitw/app
> > > (lldb) l main
> > > File: /home/karnajitw/lldbtest/obj/a/b/c/d/e/app/../../../../../.
> ./../src/a/b/c/d/e/app/main.c
> > >    1    #include "sub/subfile.h"
> > >    2
> > >    3    int main()
> > >    4    {
> > >    5      subfn();
> > >    6
> > >    7      return 0;
> > >    8    }
> > >    9
> > > (lldb) l subfn
> > > File: /home/karnajitw/lldbtest/obj/a/b/c/d/e/app/sub/../../../../
> ../../../../src/a/b/c/d/e/app/sub/subfile.c
> > > (lldb) settings append target.source-map /home/karnajitw/lldbtest/obj/
> a/b/c/d/e/app/sub/../../../../../../../../src/a/b/c/d/e/app
> /home/karnajitw/app
> > > (lldb) l subfn
> > > File: /home/karnajitw/lldbtest/obj/a/b/c/d/e/app/sub/../../../../
> ../../../../src/a/b/c/d/e/app/sub/subfile.c
> > >    1    #include <stdio.h>
> > >    2    #include "subfile.h"
> > >    3
> > >    4    void subfn()
> > >    5    {
> > >    6      printf("This is subfn...\n");
> > >    7    }
> > >    8
> > >
> > > It still needs both the mappings
> > > (lldb) settings show target.source-map
> > > target.source-map (path-map) =
> > > [0] "/home/karnajitw/lldbtest/obj/a/b/c/d/e/app/../../../../../../../src/a/b/c/d/e/app"
> -> "/home/karnajitw/app"
> > > [1] "/home/karnajitw/lldbtest/obj/a/b/c/d/e/app/sub/../../../../../../../../src/a/b/c/d/e/app"
> -> "/home/karnajitw/app"
> >
> > That does seem wrong.  Please file a PR about this.
> >
> > There's a method in FileSpec (GetNormalizedPath) that canonicalizes
> paths by doing the obvious text substitution (unwinding ..'s and removing
> /./ and /// -> /, etc.)  Somewhere in the comparison of the source and
> build paths we aren't using this when we should do.  There's also a
> FileSpec::Equal method that allows a canonicalized comparison.  It might be
> that there's just some obvious place we should have been using that.
> >
> > I'm in the middle of something else right now, but if you have some time
> to dig into this, it would probably be straightforward to track down where
> we aren't using the right comparison.  But if there's a PR, then even if
> you don't have time, somebody else will get to the PR when they have a
> chance.
> >
> > Thanks for chasing this down!
> >
> > Jim
> >
> >
> >
> > >
> > >
> > > > 3. Scenario 3: External libraries
> > > > - I haven't exactly tested this yet. But I belive in scenario too we
> might need to change the source-map.
> > >
> > > External libraries should be no different from your project.  I don't
> see anything different needed here.
> > >
> > >
> > > - Agreed.
> > >
> > > Jim
> > >
> > >
> > > >
> > > > Please look into this and guide me for the same.
> > > >
> > > > Regards,
> > > > Karan
> > > > _______________________________________________
> > > > lldb-dev mailing list
> > > > lldb-dev at lists.llvm.org
> > > > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20170829/47a96834/attachment-0001.html>


More information about the lldb-dev mailing list