<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none"><!--P{margin-top:0;margin-bottom:0;} p
{margin-top:0;
margin-bottom:0}--></style>
</head>
<body dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>Hi David,</p>
<p><br>
</p>
<p>I have to notice that I tried to reproduce the same isue for clang-9.0 Zeus but without success.</p>
<p>So seems that it's actual only for Thud clang-7.1.</p>
<p><br>
</p>
<p>I'm not sure if it worth doing but anyway here is how to reproduce:<br>
</p>
<p><br>
</p>
<p>I'm using standalone openembedded-core to reproduce.</p>
<p>- setup workspace <a href="https://www.openembedded.org/wiki/OE-Core_Standalone_Setup">
https://www.openembedded.org/wiki/OE-Core_Standalone_Setup</a></p>
<p>- apped meta-clang layer: git clone https://github.com/kraj/meta-clang.git<br>
</p>
<p>- checkout oe-core, meta-clang with thud and bitbake with 1.40<br>
</p>
<p>- patch clang with <a href="https://github.com/llvm/llvm-project/commit/56acd5a66ea3800a23be0cfdbc422624398cb096">
https://github.com/llvm/llvm-project/commit/56acd5a66ea3800a23be0cfdbc422624398cb096</a>. Actually patch need some manual work, so I attached already modified.<br>
</p>
<p>- add extra option for work-shared:<br>
</p>
<p><span style="font-size:10pt"><span style="font-family:"Courier New",monospace"><br>
</span></span></p>
<p><span style="font-size:10pt"><span style="font-family:"Courier New",monospace">diff --git a/recipes-devtools/clang/compiler-rt_git.bb b/recipes-devtools/clang/compiler-rt_git.bb</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">index 05de71f..b201814 100644</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">--- a/recipes-devtools/clang/compiler-rt_git.bb</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">+++ b/recipes-devtools/clang/compiler-rt_git.bb</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">@@ -25,6 +25,8 @@ DEPENDS_append_class-nativesdk = " clang-native"</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace"> THUMB_TUNE_CCARGS = ""</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace"> #TUNE_CCARGS += "-nostdlib"</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace"> </span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">+DEBUG_PREFIX_MAP += " -fdebug-prefix-map=${S}=. "</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">+</span><br style="font-family:"Courier New",monospace">
</span></p>
<p><br>
</p>
<p>- enable DEBUG_PREFIX_MAP</p>
<p><br style="font-family:"Courier New",monospace; font-size:10pt">
</p>
<p><span style="font-family:"Courier New",monospace; font-size:10pt">diff --git a/classes/clang.bbclass b/classes/clang.bbclass</span><br style="font-family:"Courier New",monospace; font-size:10pt">
<span style="font-family:"Courier New",monospace; font-size:10pt">index 4d20ff0..5b24fe8 100644</span><br style="font-family:"Courier New",monospace; font-size:10pt">
<span style="font-family:"Courier New",monospace; font-size:10pt">--- a/classes/clang.bbclass</span><br style="font-family:"Courier New",monospace; font-size:10pt">
<span style="font-family:"Courier New",monospace; font-size:10pt">+++ b/classes/clang.bbclass</span><br style="font-family:"Courier New",monospace; font-size:10pt">
<span style="font-family:"Courier New",monospace; font-size:10pt">@@ -30,7 +30,7 @@ TOOLCHAIN_class-crosssdk = "gcc"</span><br style="font-family:"Courier New",monospace; font-size:10pt">
<span style="font-family:"Courier New",monospace; font-size:10pt"> TOOLCHAIN_class-cross = "gcc"</span><br style="font-family:"Courier New",monospace; font-size:10pt">
<span style="font-family:"Courier New",monospace; font-size:10pt"> </span><br style="font-family:"Courier New",monospace; font-size:10pt">
<span style="font-family:"Courier New",monospace; font-size:10pt"> # -fmacro-prefix-map does not exist in clang 7.x</span><br style="font-family:"Courier New",monospace; font-size:10pt">
<span style="font-family:"Courier New",monospace; font-size:10pt">-DEBUG_PREFIX_MAP_toolchain-clang = " "</span><br style="font-family:"Courier New",monospace; font-size:10pt">
<span style="font-family:"Courier New",monospace; font-size:10pt">+#DEBUG_PREFIX_MAP_toolchain-clang = " "</span><br style="font-family:"Courier New",monospace; font-size:10pt">
<span style="font-family:"Courier New",monospace; font-size:10pt"> </span><br>
</p>
<p>- switch toolchain to clang: append TOOLCHAIN = "clang" to the conf/local.conf</p>
<p>- build clang-native: bitbake clang-native<br>
</p>
<p>- build compiler-rt: bitbake compiler-rt<br>
</p>
<p>- test for symbols:</p>
<p><br>
</p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">strings <ws>/build/tmp-glibc/work/core2-64-oe-linux/compiler-rt/7.1.0-r0</span></span><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">/build/lib/linux/libclang_rt.fuzzer-x86_64.a
| grep "lambda at" | grep "Iterate"</span></span><br>
</p>
<p><br style="font-size: 10pt; font-family: "Courier New", monospace;">
</p>
<p><span style="font-size: 10pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">void fuzzer::TracePC::IterateInline8bitCounters(CallBack) const [CallBack = (lambda at /home/oobitots/temp/oe-core/build/tmp-glibc/work-shared/llvm-project-source-7.1.0-r0/git/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:77:29)]</span></span><br style="font-size: 10pt; font-family: "Courier New", monospace;">
<span style="font-size: 10pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">void fuzzer::TracePC::IterateInline8bitCounters(CallBack) const [CallBack = (lambda at /home/oobitots/temp/oe-core/build/tmp-glibc/work-shared/llvm-project-source-7.1.0-r0/git/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:85:29)]</span></span><br style="font-size: 10pt; font-family: "Courier New", monospace;">
<span style="font-size: 10pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">void fuzzer::TracePC::IterateInline8bitCounters(CallBack) const [CallBack = (lambda at /home/oobitots/temp/oe-core/build/tmp-glibc/work-shared/llvm-project-source-7.1.0-r0/git/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:99:29)]</span></span><br style="font-size: 10pt; font-family: "Courier New", monospace;">
<span style="font-size: 10pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">void fuzzer::TracePC::IterateInline8bitCounters(CallBack) const [CallBack = (lambda at /home/oobitots/temp/oe-core/build/tmp-glibc/work-shared/llvm-project-source-7.1.0-r0/git/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:206:33)]</span></span><br style="font-size: 10pt; font-family: "Courier New", monospace;">
<span style="font-size: 10pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">void fuzzer::TracePC::IterateCoveredFunctions(CallBack) [CallBack = (lambda at /home/oobitots/temp/oe-core/build/tmp-glibc/work-shared/llvm-project-source-7.1.0-r0/git/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:314:34)]</span></span><br style="font-size: 10pt; font-family: "Courier New", monospace;">
<span style="font-size: 10pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">IterateInline8bitCounters<(lambda at ./compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:77:29)></span></span><br style="font-size: 10pt; font-family: "Courier New", monospace;">
<span style="font-size: 10pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">IterateInline8bitCounters<(lambda at ./compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:85:29)></span></span><br style="font-size: 10pt; font-family: "Courier New", monospace;">
<span style="font-size: 10pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">IterateInline8bitCounters<(lambda at ./compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:99:29)></span></span><br style="font-size: 10pt; font-family: "Courier New", monospace;">
<span style="font-size: 10pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">IterateInline8bitCounters<(lambda at ./compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:206:33)></span></span><br style="font-size: 10pt; font-family: "Courier New", monospace;">
<span style="font-size: 10pt; font-family: "Courier New", monospace;"><span style="font-family: "Courier New", monospace;">IterateCoveredFunctions<(lambda at ./compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:314:34)></span></span><br style="font-size: 10pt; font-family: "Courier New", monospace;">
<br>
</p>
<p><br>
</p>
<p>Regards,</p>
<p>Oleksiy<br>
</p>
<p><br>
</p>
<div style="color:rgb(33,33,33)">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> David Blaikie <dblaikie@gmail.com><br>
<b>Sent:</b> Wednesday, January 13, 2021 22:45<br>
<b>To:</b> Oleksiy Obitotskyi -X (oobitots - GLOBALLOGIC INC at Cisco)<br>
<b>Cc:</b> cfe-dev@lists.llvm.org; Taras Kondratiuk (takondra); Ruslan Bilovol -X (rbilovol - GLOBALLOGIC INC at Cisco)<br>
<b>Subject:</b> Re: [cfe-dev] Questions on reproducibility/deterministic build</font>
<div> </div>
</div>
<div>
<div dir="ltr">Could you provide a small standalone example and precise commands to reproduce the case you're describing?</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, Jan 13, 2021 at 9:06 AM Oleksiy Obitotskyi -X (oobitots - GLOBALLOGIC INC at Cisco) via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr" style="font-size:12pt; color:rgb(0,0,0); background-color:rgb(255,255,255); font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Hi,</p>
<p><br>
</p>
<p>I'm working on reproducibility (or deterministic build) issues with clang.</p>
<p>And I got issue with compiler-rt libraries, some of the debug symbols comes with source absolute path, e.g.:</p>
<p><br>
</p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">strings ./build/lib64/linux/libclang_rt.fuzzer-x86_64.a | grep "lambda at"</span></span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">...</span></span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">---8<--- A</span><br style="font-family:"Courier New",monospace">
</span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">CollectFeatures<(lambda at /compiler-rt/lib/fuzzer/FuzzerDriver.cpp:482:25)></span></span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">...</span><br style="font-family:"Courier New",monospace">
</span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">---8<--- B</span><br style="font-family:"Courier New",monospace">
</span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">void fuzzer::TracePC::IterateInline8bitCounters(CallBack) [CallBack = (lambda at
</span></span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace"><some-path>/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:77:29)]</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">void fuzzer::TracePC::IterateInline8bitCounters(CallBack) [CallBack = (lambda at
</span></span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace"><some-path>/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:85:29)]</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">void fuzzer::TracePC::IterateInline8bitCounters(CallBack) [CallBack = (lambda at
</span></span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace"><some-path>/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:99:29)]</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">void fuzzer::TracePC::IterateInline8bitCounters(CallBack) [CallBack = (lambda at
</span></span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace"><some-path>/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:206:33)]</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">void fuzzer::TracePC::IterateCoveredFunctions(CallBack) [CallBack = (lambda at
</span></span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace"><some-path>/compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:314:34)]</span></span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">---8<--- C</span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">IterateInline8bitCounters<(lambda at /compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:77:29)></span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">IterateInline8bitCounters<(lambda at /compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:85:29)></span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">IterateInline8bitCounters<(lambda at /compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:99:29)></span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">IterateInline8bitCounters<(lambda at /compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:206:33)></span><br style="font-family:"Courier New",monospace">
<span style="font-family:"Courier New",monospace">IterateCoveredFunctions<(lambda at /compiler-rt/lib/fuzzer/FuzzerTracePC.cpp:314:34)></span><br style="font-family:"Courier New",monospace">
</span></p>
<p><span style="font-size:10pt; font-family:"Courier New",monospace"><span style="font-family:"Courier New",monospace">...</span></span></p>
<p><br>
</p>
<p>Conditions:<br>
</p>
<p>- clang version 7 used.<br>
</p>
<p>- flags -ffile-prefix-map/debug-prefix-map/macro-prefix-map <some-path> are set during compilation</p>
<p>- patch backported <a href="https://github.com/llvm/llvm-project/commit/56acd5a66ea3800a23be0cfdbc422624398cb096" target="_blank">
https://github.com/llvm/llvm-project/commit/56acd5a66ea3800a23be0cfdbc422624398cb096</a><br>
</p>
<p><br>
</p>
<p>In section A and C debug symbols for lambda have path correctly remapped with prefix set in -f<file/debug/macro>-prefix-map.</p>
<p>But section B debug symbols for the same lambda code goes without remapping (and it's ok TypePrinter code works properly in this case).</p>
<p><br>
</p>
<p>For section B TypePrinter::printTag called with PrintingPolicy.RemapFilePaths set, but for section A the same function called with different PrintingPolicy and RemapFilePaths is not set.</p>
<p><br>
</p>
<p><br>
</p>
<p>- I'm not sure is it ok that two sections generated in different way and what is the reason for this? The similar code with lambdas from section A produce only one symbol type with remapped path.<br>
</p>
<p><br>
</p>
<p>- Is it possible and is it make sense to pass CGDebugInfo settings to the code Sema::BuildPredefinedExpr to apply
</p>
<p>RemapFilePaths settings to the PrintingPolicy used? As I understand symbols could be generated into different compilation stages/passes and some code generation settings could not be applied directly during other stages.<br>
</p>
<p><br>
</p>
<p>Regards,</p>
<p>Oleksiy<br>
</p>
</div>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br>
</blockquote>
</div>
</div>
</div>
</body>
</html>