[cfe-dev] fixing bugs fuzzed out of clang

Dmitry Polukhin via cfe-dev cfe-dev at lists.llvm.org
Mon Jan 25 07:05:43 PST 2016 

BTW, -DCMAKE_BUILD_TYPE=Debug builds llvm/lib/IR/Function.cpp with -O1 and
it hangs (or at least compile time is more that 40 min).
On -DCMAKE_BUILD_TYPE=Release it is -O2 compilation and it works fine.

On Mon, Jan 25, 2016 at 2:15 PM, Yury Gribov <y.gribov at samsung.com> wrote:

> On 01/25/2016 12:16 PM, Dmitry Polukhin via cfe-dev wrote:
>
>> Thank you for the link, debugger tips in
>> https://github.com/google/sanitizers/wiki/AddressSanitizerAndDebugger are
>> very useful but partially outdated. For example, it seems that AsanDie
>> mentioned there is now  __sanitizer::Die. That is why it would be much
>> easier if asan detects attached debugger and does
>> ASAN_OPTIONS=abort_on_error=1 under debugger. I think it will be user
>> friendly behavior and would lower adoption bar :)
>>
>
> I once asked for something like this in ASan group but folks weren't
> interested. You can take a look at https://github.com/yugr/libdebugme
>
>
> On Sat, Jan 23, 2016 at 10:04 AM, Kostya Serebryany <kcc at google.com>
>> wrote:
>>
>>
>>>
>>> On Fri, Jan 22, 2016 at 10:18 PM, Dmitry Polukhin <
>>> dmitry.polukhin at gmail.com> wrote:
>>>
>>> Yes, it was due to using Debug build. With Release build I was able to
>>>> build self build without hang on IR/Function.cpp. It was not a problem
>>>> of using make vs ninja
>>>>
>>>>
>>> My recommendation to use ninja is unrelated to your problem. It's just a
>>> better build tool, imho.
>>>
>>>
>>>   because I got clang command line and run it manually with the same
>>>> result. I do prefer Debug build to be able to run debugger properly
>>>> on sanitized binaries.
>>>>
>>>>
>>> Understood. You can probably build Function.cpp with -O2 manually and
>>> then
>>> build the rest with -O0.
>>>
>>>
>>>
>>> Running GDB on asanified release binaries I found that debug info is weak
>>>>
>>>>
>>> Yep.
>>>
>>>
>>> and stepping over source code very often jumps to the begging on the
>>>> function so it means that some instructions have no source info but I
>>>> don't
>>>> know was it instructions inserted by asan instrumentation or not. Also
>>>> it is very inconvenient that asan terminates execution when finds a
>>>> problem
>>>> instead of breaking into the debugger - it is my kind feature request :)
>>>>
>>>>
>>>
>>> See
>>> https://github.com/google/sanitizers/wiki/AddressSanitizerAndDebugger
>>> (I personally almost never use gdb with asan, so can't share any more
>>> useful tricks)
>>>
>>> --kcc
>>>
>>>
>>>
>>>
>>>>
>>>> On Sat, Jan 23, 2016 at 12:56 AM, Kostya Serebryany <kcc at google.com>
>>>> wrote:
>>>>
>>>>
>>>>>
>>>>> On Fri, Jan 22, 2016 at 6:00 AM, Dmitry Polukhin <
>>>>> dmitry.polukhin at gmail.com> wrote:
>>>>>
>>>>> Hi Kostya,
>>>>>>
>>>>>> I would like to repro and fix some of the issues that fuzzed found
>>>>>>
>>>>>>
>>>>> Nice!
>>>>>
>>>>>
>>>>> but I don't have system Clang on my machine but I do have Clang sources
>>>>>> so I need to make Clang to have ASan and after that use that Clang to
>>>>>> build
>>>>>> Clang one more time but with ASan enabled. It looks like in Clang doc
>>>>>> there
>>>>>> is no info how to do it.
>>>>>>
>>>>>>
>>>>> Mmmm.
>>>>> These pages have some related discussions, but you probably don't need
>>>>> them...
>>>>>
>>>>>
>>>>> https://github.com/google/sanitizers/wiki/MemorySanitizerBootstrappingClang
>>>>> https://github.com/google/sanitizers/wiki/AddressSanitizerHowToBuild
>>>>>
>>>>>
>>>>> I only found LLVM_USE_SANITIZER so I did it like this:
>>>>>>
>>>>>> cd $ROOT
>>>>>> mkdir release
>>>>>> cd release
>>>>>> cmake ../llvm -DCMAKE_BUILD_TYPE=Release
>>>>>> make
>>>>>> cd ..
>>>>>> mkdir build
>>>>>> cd build
>>>>>> CC=$ROOT/release/bin/clang CXX=$ROOT/release/bin/clang++ cmake ../llvm
>>>>>> -DCMAKE_BUILD_TYPE=Debug -DLLVM_USE_SANITIZER=Address
>>>>>> make
>>>>>>
>>>>>> Last build seems to hang on compilation of llvm/lib/IR/Function.cpp.
>>>>>>
>>>>>>
>>>>> I routinely build clang with clang+asan.
>>>>> llvm/lib/IR/Function.cpp indeed takes lots of time, an order of 3
>>>>> minutes,
>>>>> but not even close to what you see.
>>>>> Mostly likely the difference is that you use -DCMAKE_BUILD_TYPE=Debug
>>>>> and I use -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON
>>>>> (in other words, I use -O2).
>>>>> In general, we almost never use asan with -O0 because it'll be too
>>>>> slow.
>>>>>
>>>>> BTW, I encourage you to switch from "make" to "ninja" -- works much
>>>>> faster and nicer
>>>>>
>>>>>
>>>>>
>>>>> I break it after about 30min and 25G RAM consumed.
>>>>>> Without -fsanitize=address compilation takes about 30 sec. Do you
>>>>>> have bot
>>>>>> that checks self build with ASan?
>>>>>>
>>>>>>
>>>>> Yes.
>>>>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap
>>>>> You can check the exact commands e.g. here:
>>>>>
>>>>>
>>>>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/10702/steps/build%20clang%2Fasan/logs/stdio
>>>>> cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON
>>>>> ...
>>>>>
>>>>>
>>>>> Is it known issue?
>>>>>>
>>>>>>
>>>>> I would guess what you see is a manifestation of
>>>>> https://llvm.org/bugs/show_bug.cgi?id=17409,
>>>>> which has been bothering us for years, but I have not checked this
>>>>> particular case.
>>>>>
>>>>> Thanks for doing this!
>>>>>
>>>>> --kcc
>>>>>
>>>>>
>>>>> Thanks,
>>>>>> Dmitry
>>>>>> --
>>>>>> Software Engineer
>>>>>> Intel Compiler Team
>>>>>>
>>>>>> On Tue, Jan 5, 2016 at 1:46 PM, Andrey Bokhanko via cfe-dev <
>>>>>> cfe-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>> We (Intel clang team) will take a look and fix some of these.
>>>>>>>
>>>>>>> Yours,
>>>>>>> Andrey
>>>>>>> ======
>>>>>>> Software Engineer
>>>>>>> Intel Compiler Team
>>>>>>> Intel
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jan 4, 2016 at 9:20 PM, Kostya Serebryany via cfe-dev
>>>>>>> <cfe-dev at lists.llvm.org> wrote:
>>>>>>>
>>>>>>>> Hi Clang devs,
>>>>>>>>
>>>>>>>> In the new year I would like to ask you all to consider fixing clang
>>>>>>>>
>>>>>>> bugs
>>>>>>>
>>>>>>>> found by fuzzing (that includes, but is not limited to,
>>>>>>>> https://llvm.org/bugs/show_bug.cgi?id=23057)
>>>>>>>>
>>>>>>>> The existing fuzzer bot is reporting known bugs that are not being
>>>>>>>>
>>>>>>> fixed for
>>>>>>>
>>>>>>>> months.
>>>>>>>> E.g.
>>>>>>>>
>>>>>>>>
>>>>>>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/5328/steps/stage2%2Fasan%2Bassertions%20run%20clang-fuzzer/logs/stdio
>>>>>>>
>>>>>>>> This precludes us from treating these bugs as errors and make the
>>>>>>>>
>>>>>>> bot red on
>>>>>>>
>>>>>>>> regressions.
>>>>>>>>
>>>>>>>> Also, these shallow bugs prevent us from finding deeper bugs with
>>>>>>>>
>>>>>>> potential
>>>>>>>
>>>>>>>> security implications, and there are some such.
>>>>>>>> E.g. the bug below means that no one can safely host clang as a web
>>>>>>>>
>>>>>>> service.
>>>>>>>
>>>>>>>>
>>>>>>>> echo "* a ((int () (o W, *&])) 0" | ./bin/clang -x c++ -
>>>>>>>>
>>>>>>>> ==13059==ERROR: AddressSanitizer: heap-use-after-free on address
>>>>>>>> 0x61500000e538 at pc 0x00000081df99 bp 0x7ffdbdcb3630 sp
>>>>>>>>
>>>>>>> 0x7ffdbdcb2de8
>>>>>>>
>>>>>>>>
>>>>>>>> READ of size 20 at 0x61500000e538 thread T0
>>>>>>>>      #0 0x81df98 in __asan_memcpy
>>>>>>>>      #1 0xedcd28f in clang::TokenLexer::Lex(clang::Token&)
>>>>>>>> tools/clang/lib/Lex/TokenLexer.cpp:441:7
>>>>>>>>      #2 0xedb3c47 in clang::Preprocessor::Lex(clang::Token&)
>>>>>>>> tools/clang/lib/Lex/Preprocessor.cpp:731:23
>>>>>>>>      #3 0xa5ad93a in ConsumeParen
>>>>>>>> tools/clang/include/clang/Parse/Parser.h:383:5
>>>>>>>>      #4 0xa5ad93a in
>>>>>>>> clang::Parser::SkipUntil(llvm::ArrayRef<clang::tok::TokenKind>,
>>>>>>>> clang::Parser::SkipUntilFlags) tools/clang/lib/Parse/Parser.cpp:3
>>>>>>>>      #5 0xa78bdb8 in SkipUntil
>>>>>>>> tools/clang/include/clang/Parse/Parser.h:864:12
>>>>>>>>
>>>>>>>> 0x61500000e538 is located 312 bytes inside of 456-byte region
>>>>>>>> [0x61500000e400,0x61500000e5c8)
>>>>>>>> freed by thread T0 here:
>>>>>>>>      #0 0x8350db in __interceptor_free
>>>>>>>>      #1 0xa838c02 in ~SmallVectorImpl
>>>>>>>>
>>>>>>> include/llvm/ADT/SmallVector.h:374:7
>>>>>>>
>>>>>>>>      #2 0xa838c02 in
>>>>>>>>
>>>>>>>>
>>>>>>> clang::Parser::ParseCXXAmbiguousParenExpression(clang::Parser::ParenParseOption&,
>>>>>>>
>>>>>>>> clang::OpaquePtr<clang::QualType>&, clang::Bala
>>>>>>>>      #3 0xa7ac905 in
>>>>>>>>
>>>>>>>> clang::Parser::ParseParenExpression(clang::Parser::ParenParseOption&,
>>>>>>> bool,
>>>>>>>
>>>>>>>> bool, clang::OpaquePtr<clang::QualType>&, clang::Sour
>>>>>>>>      #4 0xa794e83 in clang::Parser::ParseCastExpression(bool, bool,
>>>>>>>>
>>>>>>> bool&,
>>>>>>>
>>>>>>>> clang::Parser::TypeCastState)
>>>>>>>>
>>>>>>> tools/clang/lib/Parse/ParseExpr.cpp:709:11
>>>>>>>
>>>>>>>>      #5 0xa77c21c in ParseCastExpression
>>>>>>>> tools/clang/lib/Parse/ParseExpr.cpp:465:20
>>>>>>>>
>>>>>>>>
>>>>>>>> --kcc
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> cfe-dev mailing list
>>>>>>>> cfe-dev at lists.llvm.org
>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>> cfe-dev mailing list
>>>>>>> cfe-dev at lists.llvm.org
>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160125/0958cb45/attachment.html>

More information about the cfe-dev mailing list