[lld][buildbot] Add more buildbots with sanitize/thread

Wed Nov 20 06:03:23 PST 2013

Hi,

On Wed, Nov 20, 2013 at 1:49 AM, Rick Foos <rfoos at codeaurora.org> wrote:

>  All,
>
> With many thanks to Galina for some fixes, and patience from Dmitri (when
> the first patch broke his builders) the Address Sanitizer is now up and
> running.
>
> http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-ubuntu-13.04-sanitize-address
>
> Two questions:
> 1) The Address Sanitizer fails lit on Clang::annotate-deep-statements.cpp<http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-ubuntu-13.04-sanitize-address/builds/104/steps/test/logs/Clang%3A%3Aannotate-deep-statements.cpp>
> Is there someone who would look at that?
> I didn't hook the buildslave to IRC yet as it would be nagging people all
> day.
>
> 2) I tried a similar buildbot recipe with Memory Sanitizer, and it fails
> to build.
> I've added the recommended compile options from clang docs, and built with
> clang-3.4.
> Is there someone maintaining memory sanitizer?
>
> Thread sanitizer with the updated recipe still fails requiring reboot :)
>
> Thanks,
> Rick
>
> --- cmake
>
> cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS="-std=c++11 -Wdocumentation -Wno-documentation-deprecated-sync -fsanitize=memory -fno-omit-frame-pointer -fno-optimize-sibling-calls" -DLLVM_LIT_ARGS="-v" -G Ninja ../llvm.src
>
>
I suggest you to use LLVM_USE_SANITIZER CMake option. It accepts "Address",
"Memory" and "MemoryWithOrigins" as values. We used it to setup our
bootstrap bot at:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap

>
>
>
> --- ninja build failure.
>
> [243/2193] Building CXX object tools/yaml2obj/CMakeFiles/yaml2obj.dir/yaml2coff.cpp.o
> [244/2193] Building Intrinsics.gen...
> FAILED: cd /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.obj/include/llvm/IR && /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.obj/bin/llvm-tblgen -gen-intrinsic -I /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.src/include/llvm/IR -I /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.src/lib/Target -I /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.src/include /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.src/include/llvm/IR/Intrinsics.td -o /var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.obj/include/llvm/IR/Intrinsics.gen.tmp
> ==63583== WARNING: MemorySanitizer: use-of-uninitialized-value
> ==63583==WARNING: Trying to symbolize code, but external symbolizer is not initialized!
>     #0 0x7fb56daa492d (/var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.obj/bin/llvm-tblgen+0x11192d)
>     #1 0x7fb56da02f50 (/var/lib/buildbot/slaves/hexagon-build-03-exp/llvm-clang-lld-x86_64-ubuntu-13.04-fsanitize-memory/llvm.obj/bin/llvm-tblgen+0x6ff50)
>
> SUMMARY: MemorySanitizer: use-of-uninitialized-value ??:0 ??
> Exitingninja: build stopped: subcommand failed.program finished with exit code 1
> elapsedTime=35.473355
>
>
>
>
> On 11/04/2013 11:48 AM, Rick Foos wrote:
>
> On 11/04/2013 11:42 AM, Sean Silva wrote:
>
>
>
>
> On Mon, Nov 4, 2013 at 11:57 AM, Rick Foos <rfoos at codeaurora.org> wrote:
>
>> On 11/04/2013 10:35 AM, Shankar Easwaran wrote:
>>
>>> On 10/29/2013 6:46 PM, Dmitri Gribenko wrote:
>>>
>>>> On Tue, Oct 29, 2013 at 4:34 PM, Shankar Easwaran
>>>> <shankare at codeaurora.org> wrote:
>>>>
>>>>> On 10/29/2013 6:32 PM, Dmitri Gribenko wrote:
>>>>>
>>>>>> But that patch adds the builders to the fast schedule, doesn't it?..
>>>>>>
>>>>> Yes, can you point me to the schedule that you want to add ?
>>>>>
>>>>> Is there a seperate category of builders where I need to add this ?
>>>>>
>>>> Sorry, I don't have an in-depth experience of working with zorg
>>>> scripts to do something non-standard, so I don't think I can give any
>>>> concrete advice.  I remember that when I was setting up the clang_fast
>>>> schedule, someone on IRC was helping me.
>>>>
>>> What we have been doing over the last week is to run the clang-lld fast
>>> build, with sanitize options on the buildbots that we have.
>>>
>>> They do run fine, except that we see very high cpu usage (load average :
>>> 149).  We used a *ninja* environment for building and running tests.
>>>
>>> Do you have any suggestions to get a environment, where the load average
>>> is much saner ?
>>>
>>> Ccing Rick Foos here, who was able to setup the complete environment, if
>>> you have any questions.
>>>
>>> Thanks
>>>
>>> Shankar Easwaran
>>>
>>>  My thought for config was to use the existing sanitizer category in
>> builders.py, and add an Ubuntu 13.04 slave to run the sanitizers.
>>
>> To stabilize the server, the maximum load of both ninja and ninja running
>> asan needs to be controlled. The sanitizer jobs need to be serialized to
>> avoid resource limits on the server.
>>
>> First removing the nice -10's, which may have limited what the OS could
>> do to reduce the load.
>> Second, ninja -j for the main build.
>>
>
>  Ninja also has a -l option which will not start new jobs until system
> load falls below a set target.
>
>
>> Third, for asan, defining pools may be a way to control the load, but
>> pools look to be hardcoded rather than tunable for a machine.
>>
>
>  You can always process the ninja files and add the pools yourself. I
> wrote a node.js module that makes this sort of thing trivially easy to do <
> https://github.com/chisophugis/ninja-build-parser> (check out
> examples/identity.js for a 70-line round-tripping example); just modify the
> rules in the object stream to add a pool binding to the relevant rules, and
> add pool declarations at th beginning (examples/toJson.js can be used to
> inspect the object format; examples/identity.js shows how all the fields
> relate back to the .ninja file).
>
>  -- Sean Silva
>
> Thanks, looking at it now.
> --Rick Foos
>
>
>
>>
>> The second problem is to prevent both sanitizers from running at the same
>> time.
>>
>> Is there a scheduler method in zorg to serialize two builders?
>>
>> Any thoughts on this, or suggestions on how to do this better would be
>> appreciated.
>>
>> -rick
>>
>> --
>> Rick Foos
>>
>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted
>> by The Linux Foundation
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>
>
>
>
> --
> Rick Foos
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
>
>
>
> _______________________________________________
> llvm-commits mailing listllvm-commits at cs.uiuc.eduhttp://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
>
>
> --
> Rick Foos
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
>

-- 
Alexey Samsonov, MSK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20131120/7ec7606c/attachment.html>