[lldb-dev] Difficulty in using target.source-map to map source files
Jim Ingham via lldb-dev
lldb-dev at lists.llvm.org
Mon Aug 28 10:35:44 PDT 2017
That will be great! Last time we were more aggressive about normalizing paths we caused a decent performance regression by normalizing all the paths every time we did a file-name compare for setting breakpoints. If you look at FileSpec::Equal we added some "is it worth doing this work" preparatory work. I don't know where you'll find the additional places this needed to be done, but it's worth keeping an eye out for not putting GetNormalizedPath on a hot path.
Jim
> On Aug 28, 2017, at 1:19 AM, karnajit wangkhem <karnajitw at gmail.com> wrote:
>
> Thanks Jim for the help. I would definitely like to work on this PR soon.
>
> Filed the PR here
> https://bugs.llvm.org/show_bug.cgi?id=34341
>
> Regards,
> Karan
>
> On Thu, Aug 24, 2017 at 11:17 PM, Jim Ingham <jingham at apple.com> wrote:
>
> > On Aug 24, 2017, at 10:06 AM, karnajit wangkhem <karnajitw at gmail.com> wrote:
> >
> > Hi Jim,
> >
> > Thanks for the valuable reply. Please find my comments inline.
> >
> > Regards,
> > Karan
> >
> > On Wed, Aug 23, 2017 at 12:32 AM, Jim Ingham <jingham at apple.com> wrote:
> >
> > > On Aug 21, 2017, at 12:02 PM, karnajit wangkhem via lldb-dev <lldb-dev at lists.llvm.org> wrote:
> > >
> > > Hi All,
> > >
> > > Below is a sample example, where target.source-map seems to have a limitation. The limitation seems to be because
> > > 1. lldb does not have an equivalent command like directory in gdb
> >
> > The gdb "dir" command seemed always to be more annoying than useful in real situations. If your project had any complexity in the directory structure you ended up having to add dir commands for all the subdirectories, which got tedious quickly. Since you pretty much always move your sources around rigidly, "source maps" are a more natural way to specify this.
> >
> > Note, we could add a recursive "dir" command, but you can't do the simpleminded recursive search or you'll mess up when the sources have same named files in different directories. Because of this, again the source maps seem more suited.
> >
> >
> > -- Really a good info to know. Thanks!
> >
> > > 2. target.source-map does not accept multiple mapping entries. I checked this only with freebsd.
> > >
> > > (lldb) settings show target.source-map
> > > target.source-map (path-map) =
> > > [0] "/home/karnajitw/lldb_test_execs/test_source_line1" -> "/u/test/test_source_line1"
> >
> > That is not correct:
> >
> > (lldb) settings set target.source-map /some/source/path /tmp /some/other/source/path /tmp
> > (lldb) settings show target.source-map
> > target.source-map (path-map) =
> > [0] "/some/source/path" -> "/tmp"
> > [1] "/some/other/source/path" -> "/tmp"
> >
> > or
> >
> > (lldb) set set target.source-map /some/source/path /tmp
> > (lldb) set append target.source-map /some/other/source/path /tmp
> > (lldb) set show target.source-map
> > target.source-map (path-map) =
> > [0] "/some/source/path" -> "/tmp"
> > [1] "/some/other/source/path" -> "/tmp"
> >
> > - Thanks! This helped.
> >
> > >
> > > 3. Haven't checked in the code yet, but if we see the mappings of scenario 1, they all point to a single real path /home/karnajitw/lldb_test_execs/test_source_line1. But looks like the mapping logic only considers strings into account. But, at the same time, I am not claiming that they should be interpreted as path from a different machine during the mapping.
> > >
> >
> > I'm not sure what you mean here. lldb does what it can to unwind the source path, but it doesn't assume that those paths actually exist locally, so the most it can do is remove redundant "/"-s and "/./"-s and unwind ".."-s. We do this before most source path comparisons (it's part of the FileSpec Equals method). We've been tinkering with this over time so make sure you are using a recent version of lldb. If there are places where we don't do this, then that's easily fixed.
> >
> >
> > - Your answer is in line with what I wanted to know. My lldb should be quite recent. But I used it only for freebsd currently.
> >
> > % ./lldb --version
> > lldb version 5.0.0 (http://llvm.org/svn/llvm-project/lldb/trunk revision 305778)
> > clang revision 305546
> > llvm revision 305548
> >
> >
> > > I want to check on this issue in-depth. But before that, want to confirm if this is real issue or there are other ways to deal these scenarios which I am not aware of?
> > >
> > > I am referring below link for the lldb commands.
> > > https://lldb.llvm.org/lldb-gdb.html
> > >
> > > 1. First scenario: Different souce file path representation
> > >
> > > [/home/karnajitw/lldb_test_execs/test_source_line1] $ clang -O0 -g /home/karnajitw/lldb_test_execs///test_source_line1/main.c /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_source_line1/a/ainc.c /home/karnajitw/lldb_test_execs/test_source_line1/a/b/binc.c
> > >
> > > Machine 1: /home/karnajitw/lldb_test_execs/test_source_line1 -> Machine 2: /u/test/test_source_line1
> > >
> > > test_source_line1
> > > |-- a
> > > | |-- ainc.c
> > > | |-- ainc.h
> > > | `-- b
> > > | |-- binc.c
> > > | `-- binc.h
> > > |-- a.out
> > > `-- main.c
> > >
> > > % ./lldb test_source_line1/a.out
> > >
> > > (lldb) target create "test_source_line1/a.out"
> > > Current executable set to 'test_source_line1/a.out' (x86_64).
> > > (lldb) l main
> > > File: /home/karnajitw/lldb_test_execs///test_source_line1/main.c
> > > (lldb) l afn
> > > File: /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_source_line1/a/ainc.c
> > > (lldb) l bfn
> > > File: /home/karnajitw/lldb_test_execs/test_source_line1/a/b/binc.c
> > >
> > > (lldb) settings set target.source-map /home/karnajitw/lldb_test_execs///test_source_line1 /u/test/test_source_line1
> > >
> > > (lldb) l main
> > > File: /home/karnajitw/lldb_test_execs///test_source_line1/main.c
> > > 1 #include "a/ainc.h"
> > > 2
> > > 3 int main()
> > > 4 {
> > > 5 afn();
> > > 6
> > > 7 bfn();
> > > 8
> > > 9 return 0;
> > > 10 }
> > > (lldb) l afn
> > > File: /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_source_line1/a/ainc.c
> > > (lldb) l bfn
> > > File: /home/karnajitw/lldb_test_execs/test_source_line1/a/b/binc.c
> > >
> > > (lldb) settings set target.source-map /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_source_line1 /u/test/test_source_line1
> > >
> > > (lldb) l main
> > > File: /home/karnajitw/lldb_test_execs///test_source_line1/main.c
> > > (lldb) l afn
> > > File: /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_source_line1/a/ainc.c
> > > 1 #include <stdio.h>
> > > 2 #include "ainc.h"
> > > 3
> > > 4 void afn()
> > > 5 {
> > > 6 printf("Hello this is afn...\n");
> > > 7 }
> > > (lldb) l bfn
> > > File: /home/karnajitw/lldb_test_execs/test_source_line1/a/b/binc.c
> > >
> > > (lldb) settings set target.source-map /home/karnajitw/lldb_test_execs/test_source_line1 /u/test/test_source_line1
> > >
> > > (lldb) l main
> > > File: /home/karnajitw/lldb_test_execs///test_source_line1/main.c
> > > (lldb) l afn
> > > File: /home/karnajitw/lldb_test_execs/../lldb_test_execs/test_source_line1/a/ainc.c
> > > (lldb) l bfn
> > > File: /home/karnajitw/lldb_test_execs/test_source_line1/a/b/binc.c
> > > 1 #include <stdio.h>
> > > 2 #include "binc.h"
> > > 3
> > > 4 void bfn()
> > > 5 {
> > > 6 printf("Hello this is bfn...\n");
> > > 7 }
> >
> > I can't test the "///" part of this, clang seems to always collapse these for me. But with a recent lldb, I see:
> >
> > $ clang -g -O0 -c '/tmp/sources/build/../files/hello.c'
> > $ dwarfdump hello.o
> > ...
> > 0x0000000b: TAG_compile_unit [1] *
> > AT_producer( "Apple LLVM version 9.0.0 (clang-900.0.31)" )
> > AT_language( DW_LANG_C99 )
> > AT_name( "/tmp/sources/build/../files/hello.c" )
> >
> > So we did get a compile unit name with .. in it, but:
> >
> > lldbrulez:/tmp/sources/build > clang -g -O0 -o hello hello.o
> > lldbrulez:/tmp/sources/build > lldb
> > (lldb) file hello
> > Current executable set to 'hello' (x86_64).
> > (lldb) source list -n main
> > File: /tmp/sources/build/../files/hello.c
> > 1 #include <stdio.h>
> > 2
> > 3 int
> > 4 main()
> > 5 {
> > 6 printf ("Hello there.\n");
> > 7 return 0;
> > 8 }
> >
> > So we do unwind the ..'s in this case. This example doesn't have source-maps, but the same thing works if I move the sources and add just a <build-top> -> <debug-top> mapping. As I say, we have been working these ".." aware comparisons through all the file compares recently, so you may just need to get a newer lldb.
> >
> > - Lets say in my above example, the function is this way
> > main -> afn -> bfn
> >
> > Now, I breakpoint on bfn, now to get source list on all the 3 frames, I will need this below mapping
> >
> > (lldb)
> > target.source-map (path-map) =
> > [0] "/home/karnajitw/lldb_test_execs///test_source_line1" -> "/u/test/test_source_line1"
> > [1] "/home/karnajitw/lldb_test_execs/../lldb_test_execs/test_source_line1" -> "/u/test/test_source_line1"
> > [2] "/home/karnajitw/lldb_test_execs/test_source_line1" -> "/u/test/test_source_line1"
> >
> > Like the previous discussion, All the LHS and RHS actually refers to the same path. So why we need 3 mappings here? But, as I have mentioned previously the problem could be just in the freebsd version also.
> >
> >
> > >
> > > 2. Scenario 2: Deep directory structure
> > >
> > > <top>/obj/a/b/c/d/e/app/sub/../../../../../../../../src/a/b/c/d/e/app/sub/file
> > > <top>/obj/a/b/c/d/e/app/../../../../../../../src/a/b/c/d/e/app/file
> > >
> > > - If we carry the copy the source file to machine 2, we cannot easily map the source file without creating dummy <top>/obj/a/b/c/d/e/f/g.
> > >
> > You should only need to map <top> to <top>. The only reason you would have to start specifying subdirectories is you have ones specified with ".."-s, but that problem is properly handled by making the file comparisons aware of backup operators. So if the comparisons are working right (again check a more recent lldb) you should not need to deal with all these dots.
> >
> >
> > - Checked with latest lldb build, its the same
> >
> > (lldb) version
> > lldb version 6.0.0 (http://llvm.org/svn/llvm-project/lldb/trunk revision 311075)
> > clang revision 311072
> > llvm revision 311071
> >
> > $ build/bin/lldb a.out
> > (lldb) target create "a.out"
> > Current executable set to 'a.out' (x86_64).
> > (lldb) settings set target.source-map /home/karnajitw/lldbtest/obj/a/b/c/d/e/app/../../../../../../../src/a/b/c/d/e/app /home/karnajitw/app
> > (lldb) l main
> > File: /home/karnajitw/lldbtest/obj/a/b/c/d/e/app/../../../../../../../src/a/b/c/d/e/app/main.c
> > 1 #include "sub/subfile.h"
> > 2
> > 3 int main()
> > 4 {
> > 5 subfn();
> > 6
> > 7 return 0;
> > 8 }
> > 9
> > (lldb) l subfn
> > File: /home/karnajitw/lldbtest/obj/a/b/c/d/e/app/sub/../../../../../../../../src/a/b/c/d/e/app/sub/subfile.c
> > (lldb) settings append target.source-map /home/karnajitw/lldbtest/obj/a/b/c/d/e/app/sub/../../../../../../../../src/a/b/c/d/e/app /home/karnajitw/app
> > (lldb) l subfn
> > File: /home/karnajitw/lldbtest/obj/a/b/c/d/e/app/sub/../../../../../../../../src/a/b/c/d/e/app/sub/subfile.c
> > 1 #include <stdio.h>
> > 2 #include "subfile.h"
> > 3
> > 4 void subfn()
> > 5 {
> > 6 printf("This is subfn...\n");
> > 7 }
> > 8
> >
> > It still needs both the mappings
> > (lldb) settings show target.source-map
> > target.source-map (path-map) =
> > [0] "/home/karnajitw/lldbtest/obj/a/b/c/d/e/app/../../../../../../../src/a/b/c/d/e/app" -> "/home/karnajitw/app"
> > [1] "/home/karnajitw/lldbtest/obj/a/b/c/d/e/app/sub/../../../../../../../../src/a/b/c/d/e/app" -> "/home/karnajitw/app"
>
> That does seem wrong. Please file a PR about this.
>
> There's a method in FileSpec (GetNormalizedPath) that canonicalizes paths by doing the obvious text substitution (unwinding ..'s and removing /./ and /// -> /, etc.) Somewhere in the comparison of the source and build paths we aren't using this when we should do. There's also a FileSpec::Equal method that allows a canonicalized comparison. It might be that there's just some obvious place we should have been using that.
>
> I'm in the middle of something else right now, but if you have some time to dig into this, it would probably be straightforward to track down where we aren't using the right comparison. But if there's a PR, then even if you don't have time, somebody else will get to the PR when they have a chance.
>
> Thanks for chasing this down!
>
> Jim
>
>
>
> >
> >
> > > 3. Scenario 3: External libraries
> > > - I haven't exactly tested this yet. But I belive in scenario too we might need to change the source-map.
> >
> > External libraries should be no different from your project. I don't see anything different needed here.
> >
> >
> > - Agreed.
> >
> > Jim
> >
> >
> > >
> > > Please look into this and guide me for the same.
> > >
> > > Regards,
> > > Karan
> > > _______________________________________________
> > > lldb-dev mailing list
> > > lldb-dev at lists.llvm.org
> > > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>
More information about the lldb-dev
mailing list