[lld][buildbot] Add more buildbots with sanitize/thread
Rick Foos
rfoos at codeaurora.org
Tue Nov 19 13:49:14 PST 2013
All,
With many thanks to Galina for some fixes, and patience from Dmitri
(when the first patch broke his builders) the Address Sanitizer is now
up and running.
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-ubuntu-13.04-sanitize-address
Two questions:
1) The Address Sanitizer fails lit on
Clang::annotate-deep-statements.cpp
<http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-ubuntu-13.04-sanitize-address/builds/104/steps/test/logs/Clang%3A%3Aannotate-deep-statements.cpp>
Is there someone who would look at that?
I didn't hook the buildslave to IRC yet as it would be nagging people
all day.
2) I tried a similar buildbot recipe with Memory Sanitizer, and it fails
to build.
I've added the recommended compile options from clang docs, and built
with clang-3.4.
Is there someone maintaining memory sanitizer?
Thread sanitizer with the updated recipe still fails requiring reboot :)
Thanks,
Rick
--- cmake
cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS="-std=c++11 -Wdocumentation -Wno-documentation-deprecated-sync -fsanitize=memory -fno-omit-frame-pointer -fno-optimize-sibling-calls" -DLLVM_LIT_ARGS="-v" -G Ninja ../llvm.src
--- ninja build failure.
[243/2193] Building CXX object tools/yaml2obj/CMakeFiles/yaml2obj.dir/yaml2coff.cpp.o
[244/2193] Building Intrinsics.gen...
FAILED: cd /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.obj/include/llvm/IR && /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.obj/bin/llvm-tblgen -gen-intrinsic -I /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.src/include/llvm/IR -I /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.src/lib/Target -I /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.src/include /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.src/include/llvm/IR/Intrinsics.td -o /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.obj/include/llvm/IR/Intrinsics.gen.tmp
==63583== WARNING: MemorySanitizer: use-of-uninitialized-value
==63583==WARNING: Trying to symbolize code, but external symbolizer is not initialized!
#0 0x7fb56daa492d (/var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.obj/bin/llvm-tblgen+0x11192d)
#1 0x7fb56da02f50 (/var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.obj/bin/llvm-tblgen+0x6ff50)
SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 ??
Exiting
ninja: build stopped: subcommand failed.
program finished with exit code 1
elapsedTime=35.473355
On 11/04/2013 11:48 AM, Rick Foos wrote:
> On 11/04/2013 11:42 AM, Sean Silva wrote:
>>
>>
>>
>> On Mon, Nov 4, 2013 at 11:57 AM, Rick Foos <rfoos at codeaurora.org
>> <mailto:rfoos at codeaurora.org>> wrote:
>>
>> On 11/04/2013 10:35 AM, Shankar Easwaran wrote:
>>
>> On 10/29/2013 6:46 PM, Dmitri Gribenko wrote:
>>
>> On Tue, Oct 29, 2013 at 4:34 PM, Shankar Easwaran
>> <shankare at codeaurora.org
>> <mailto:shankare at codeaurora.org>> wrote:
>>
>> On 10/29/2013 6:32 PM, Dmitri Gribenko wrote:
>>
>> But that patch adds the builders to the fast
>> schedule, doesn't it?..
>>
>> Yes, can you point me to the schedule that you want
>> to add ?
>>
>> Is there a seperate category of builders where I need
>> to add this ?
>>
>> Sorry, I don't have an in-depth experience of working
>> with zorg
>> scripts to do something non-standard, so I don't think I
>> can give any
>> concrete advice. I remember that when I was setting up
>> the clang_fast
>> schedule, someone on IRC was helping me.
>>
>> What we have been doing over the last week is to run the
>> clang-lld fast build, with sanitize options on the buildbots
>> that we have.
>>
>> They do run fine, except that we see very high cpu usage
>> (load average : 149). We used a *ninja* environment for
>> building and running tests.
>>
>> Do you have any suggestions to get a environment, where the
>> load average is much saner ?
>>
>> Ccing Rick Foos here, who was able to setup the complete
>> environment, if you have any questions.
>>
>> Thanks
>>
>> Shankar Easwaran
>>
>> My thought for config was to use the existing sanitizer category
>> in builders.py, and add an Ubuntu 13.04 slave to run the sanitizers.
>>
>> To stabilize the server, the maximum load of both ninja and ninja
>> running asan needs to be controlled. The sanitizer jobs need to
>> be serialized to avoid resource limits on the server.
>>
>> First removing the nice -10's, which may have limited what the OS
>> could do to reduce the load.
>> Second, ninja -j for the main build.
>>
>>
>> Ninja also has a -l option which will not start new jobs until system
>> load falls below a set target.
>>
>> Third, for asan, defining pools may be a way to control the load,
>> but pools look to be hardcoded rather than tunable for a machine.
>>
>>
>> You can always process the ninja files and add the pools yourself. I
>> wrote a node.js module that makes this sort of thing trivially easy
>> to do <https://github.com/chisophugis/ninja-build-parser> (check out
>> examples/identity.js for a 70-line round-tripping example); just
>> modify the rules in the object stream to add a pool binding to the
>> relevant rules, and add pool declarations at th beginning
>> (examples/toJson.js can be used to inspect the object format;
>> examples/identity.js shows how all the fields relate back to the
>> .ninja file).
>>
>> -- Sean Silva
> Thanks, looking at it now.
> --Rick Foos
>
>>
>> The second problem is to prevent both sanitizers from running at
>> the same time.
>>
>> Is there a scheduler method in zorg to serialize two builders?
>>
>> Any thoughts on this, or suggestions on how to do this better
>> would be appreciated.
>>
>> -rick
>>
>> --
>> Rick Foos
>>
>> Qualcomm Innovation Center, Inc. is a member of Code Aurora
>> Forum, hosted by The Linux Foundation
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu <mailto:llvm-commits at cs.uiuc.edu>
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>
>>
>
>
> --
> Rick Foos
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
--
Rick Foos
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20131119/12afa0ef/attachment.html>
More information about the llvm-commits
mailing list