[cfe-dev] fixing bugs fuzzed out of clang

Kostya Serebryany via cfe-dev cfe-dev at lists.llvm.org
Fri Jan 22 23:04:09 PST 2016 

On Fri, Jan 22, 2016 at 10:18 PM, Dmitry Polukhin <dmitry.polukhin at gmail.com
> wrote:

> Yes, it was due to using Debug build. With Release build I was able to
> build self build without hang on IR/Function.cpp. It was not a problem of
> using make vs ninja
>

My recommendation to use ninja is unrelated to your problem. It's just a
better build tool, imho.


>  because I got clang command line and run it manually with the same
> result. I do prefer Debug build to be able to run debugger properly
> on sanitized binaries.
>

Understood. You can probably build Function.cpp with -O2 manually and then
build the rest with -O0.



> Running GDB on asanified release binaries I found that debug info is weak
>

Yep.


> and stepping over source code very often jumps to the begging on the
> function so it means that some instructions have no source info but I don't
> know was it instructions inserted by asan instrumentation or not. Also
> it is very inconvenient that asan terminates execution when finds a problem
> instead of breaking into the debugger - it is my kind feature request :)
>


See https://github.com/google/sanitizers/wiki/AddressSanitizerAndDebugger
(I personally almost never use gdb with asan, so can't share any more
useful tricks)

--kcc



>
>
> On Sat, Jan 23, 2016 at 12:56 AM, Kostya Serebryany <kcc at google.com>
> wrote:
>
>>
>>
>> On Fri, Jan 22, 2016 at 6:00 AM, Dmitry Polukhin <
>> dmitry.polukhin at gmail.com> wrote:
>>
>>> Hi Kostya,
>>>
>>> I would like to repro and fix some of the issues that fuzzed found
>>>
>>
>> Nice!
>>
>>
>>> but I don't have system Clang on my machine but I do have Clang sources
>>> so I need to make Clang to have ASan and after that use that Clang to build
>>> Clang one more time but with ASan enabled. It looks like in Clang doc there
>>> is no info how to do it.
>>>
>>
>> Mmmm.
>> These pages have some related discussions, but you probably don't need
>> them...
>>
>> https://github.com/google/sanitizers/wiki/MemorySanitizerBootstrappingClang
>> https://github.com/google/sanitizers/wiki/AddressSanitizerHowToBuild
>>
>>
>>> I only found LLVM_USE_SANITIZER so I did it like this:
>>>
>>> cd $ROOT
>>> mkdir release
>>> cd release
>>> cmake ../llvm -DCMAKE_BUILD_TYPE=Release
>>> make
>>> cd ..
>>> mkdir build
>>> cd build
>>> CC=$ROOT/release/bin/clang CXX=$ROOT/release/bin/clang++ cmake ../llvm
>>> -DCMAKE_BUILD_TYPE=Debug -DLLVM_USE_SANITIZER=Address
>>> make
>>>
>>> Last build seems to hang on compilation of llvm/lib/IR/Function.cpp.
>>>
>>
>> I routinely build clang with clang+asan.
>> llvm/lib/IR/Function.cpp indeed takes lots of time, an order of 3
>> minutes,
>> but not even close to what you see.
>> Mostly likely the difference is that you use -DCMAKE_BUILD_TYPE=Debug
>> and I use -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON
>> (in other words, I use -O2).
>> In general, we almost never use asan with -O0 because it'll be too slow.
>>
>> BTW, I encourage you to switch from "make" to "ninja" -- works much
>> faster and nicer
>>
>>
>>
>>> I break it after about 30min and 25G RAM consumed.
>>> Without -fsanitize=address compilation takes about 30 sec. Do you have bot
>>> that checks self build with ASan?
>>>
>>
>> Yes.
>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap
>> You can check the exact commands e.g. here:
>>
>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/10702/steps/build%20clang%2Fasan/logs/stdio
>> cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON ...
>>
>>
>>> Is it known issue?
>>>
>>
>> I would guess what you see is a manifestation of
>> https://llvm.org/bugs/show_bug.cgi?id=17409,
>> which has been bothering us for years, but I have not checked this
>> particular case.
>>
>> Thanks for doing this!
>>
>> --kcc
>>
>>
>>> Thanks,
>>> Dmitry
>>> --
>>> Software Engineer
>>> Intel Compiler Team
>>>
>>> On Tue, Jan 5, 2016 at 1:46 PM, Andrey Bokhanko via cfe-dev <
>>> cfe-dev at lists.llvm.org> wrote:
>>>
>>>> We (Intel clang team) will take a look and fix some of these.
>>>>
>>>> Yours,
>>>> Andrey
>>>> ======
>>>> Software Engineer
>>>> Intel Compiler Team
>>>> Intel
>>>>
>>>>
>>>> On Mon, Jan 4, 2016 at 9:20 PM, Kostya Serebryany via cfe-dev
>>>> <cfe-dev at lists.llvm.org> wrote:
>>>> > Hi Clang devs,
>>>> >
>>>> > In the new year I would like to ask you all to consider fixing clang
>>>> bugs
>>>> > found by fuzzing (that includes, but is not limited to,
>>>> > https://llvm.org/bugs/show_bug.cgi?id=23057)
>>>> >
>>>> > The existing fuzzer bot is reporting known bugs that are not being
>>>> fixed for
>>>> > months.
>>>> > E.g.
>>>> >
>>>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/5328/steps/stage2%2Fasan%2Bassertions%20run%20clang-fuzzer/logs/stdio
>>>> > This precludes us from treating these bugs as errors and make the bot
>>>> red on
>>>> > regressions.
>>>> >
>>>> > Also, these shallow bugs prevent us from finding deeper bugs with
>>>> potential
>>>> > security implications, and there are some such.
>>>> > E.g. the bug below means that no one can safely host clang as a web
>>>> service.
>>>> >
>>>> > echo "* a ((int () (o W, *&])) 0" | ./bin/clang -x c++ -
>>>> >
>>>> > ==13059==ERROR: AddressSanitizer: heap-use-after-free on address
>>>> > 0x61500000e538 at pc 0x00000081df99 bp 0x7ffdbdcb3630 sp
>>>> 0x7ffdbdcb2de8
>>>> >
>>>> > READ of size 20 at 0x61500000e538 thread T0
>>>> >     #0 0x81df98 in __asan_memcpy
>>>> >     #1 0xedcd28f in clang::TokenLexer::Lex(clang::Token&)
>>>> > tools/clang/lib/Lex/TokenLexer.cpp:441:7
>>>> >     #2 0xedb3c47 in clang::Preprocessor::Lex(clang::Token&)
>>>> > tools/clang/lib/Lex/Preprocessor.cpp:731:23
>>>> >     #3 0xa5ad93a in ConsumeParen
>>>> > tools/clang/include/clang/Parse/Parser.h:383:5
>>>> >     #4 0xa5ad93a in
>>>> > clang::Parser::SkipUntil(llvm::ArrayRef<clang::tok::TokenKind>,
>>>> > clang::Parser::SkipUntilFlags) tools/clang/lib/Parse/Parser.cpp:3
>>>> >     #5 0xa78bdb8 in SkipUntil
>>>> > tools/clang/include/clang/Parse/Parser.h:864:12
>>>> >
>>>> > 0x61500000e538 is located 312 bytes inside of 456-byte region
>>>> > [0x61500000e400,0x61500000e5c8)
>>>> > freed by thread T0 here:
>>>> >     #0 0x8350db in __interceptor_free
>>>> >     #1 0xa838c02 in ~SmallVectorImpl
>>>> include/llvm/ADT/SmallVector.h:374:7
>>>> >     #2 0xa838c02 in
>>>> >
>>>> clang::Parser::ParseCXXAmbiguousParenExpression(clang::Parser::ParenParseOption&,
>>>> > clang::OpaquePtr<clang::QualType>&, clang::Bala
>>>> >     #3 0xa7ac905 in
>>>> > clang::Parser::ParseParenExpression(clang::Parser::ParenParseOption&,
>>>> bool,
>>>> > bool, clang::OpaquePtr<clang::QualType>&, clang::Sour
>>>> >     #4 0xa794e83 in clang::Parser::ParseCastExpression(bool, bool,
>>>> bool&,
>>>> > clang::Parser::TypeCastState)
>>>> tools/clang/lib/Parse/ParseExpr.cpp:709:11
>>>> >     #5 0xa77c21c in ParseCastExpression
>>>> > tools/clang/lib/Parse/ParseExpr.cpp:465:20
>>>> >
>>>> >
>>>> > --kcc
>>>> >
>>>> > _______________________________________________
>>>> > cfe-dev mailing list
>>>> > cfe-dev at lists.llvm.org
>>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>> >
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160122/a1523aae/attachment.html>

More information about the cfe-dev mailing list