[cfe-dev] fixing bugs fuzzed out of clang

Fri Jan 22 22:18:25 PST 2016

Yes, it was due to using Debug build. With Release build I was able to
build self build without hang on IR/Function.cpp. It was not a problem of
using make vs ninja because I got clang command line and run it manually
with the same result. I do prefer Debug build to be able to run debugger
properly on sanitized binaries. Running GDB on asanified release binaries I
found that debug info is weak and stepping over source code very often
jumps to the begging on the function so it means that some instructions
have no source info but I don't know was it instructions inserted by
asan instrumentation or not. Also it is very inconvenient that asan
terminates execution when finds a problem instead of breaking into the
debugger - it is my kind feature request :)

On Sat, Jan 23, 2016 at 12:56 AM, Kostya Serebryany <kcc at google.com> wrote:

>
>
> On Fri, Jan 22, 2016 at 6:00 AM, Dmitry Polukhin <
> dmitry.polukhin at gmail.com> wrote:
>
>> Hi Kostya,
>>
>> I would like to repro and fix some of the issues that fuzzed found
>>
>
> Nice!
>
>
>> but I don't have system Clang on my machine but I do have Clang sources
>> so I need to make Clang to have ASan and after that use that Clang to build
>> Clang one more time but with ASan enabled. It looks like in Clang doc there
>> is no info how to do it.
>>
>
> Mmmm.
> These pages have some related discussions, but you probably don't need
> them...
> https://github.com/google/sanitizers/wiki/MemorySanitizerBootstrappingClang
> https://github.com/google/sanitizers/wiki/AddressSanitizerHowToBuild
>
>
>> I only found LLVM_USE_SANITIZER so I did it like this:
>>
>> cd $ROOT
>> mkdir release
>> cd release
>> cmake ../llvm -DCMAKE_BUILD_TYPE=Release
>> make
>> cd ..
>> mkdir build
>> cd build
>> CC=$ROOT/release/bin/clang CXX=$ROOT/release/bin/clang++ cmake ../llvm
>> -DCMAKE_BUILD_TYPE=Debug -DLLVM_USE_SANITIZER=Address
>> make
>>
>> Last build seems to hang on compilation of llvm/lib/IR/Function.cpp.
>>
>
> I routinely build clang with clang+asan.
> llvm/lib/IR/Function.cpp indeed takes lots of time, an order of 3 minutes,
> but not even close to what you see.
> Mostly likely the difference is that you use -DCMAKE_BUILD_TYPE=Debug
> and I use -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON
> (in other words, I use -O2).
> In general, we almost never use asan with -O0 because it'll be too slow.
>
> BTW, I encourage you to switch from "make" to "ninja" -- works much faster
> and nicer
>
>
>
>> I break it after about 30min and 25G RAM consumed.
>> Without -fsanitize=address compilation takes about 30 sec. Do you have bot
>> that checks self build with ASan?
>>
>
> Yes.
> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap
> You can check the exact commands e.g. here:
>
> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/10702/steps/build%20clang%2Fasan/logs/stdio
> cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON ...
>
>
>> Is it known issue?
>>
>
> I would guess what you see is a manifestation of
> https://llvm.org/bugs/show_bug.cgi?id=17409,
> which has been bothering us for years, but I have not checked this
> particular case.
>
> Thanks for doing this!
>
> --kcc
>
>
>> Thanks,
>> Dmitry
>> --
>> Software Engineer
>> Intel Compiler Team
>>
>> On Tue, Jan 5, 2016 at 1:46 PM, Andrey Bokhanko via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>>
>>> We (Intel clang team) will take a look and fix some of these.
>>>
>>> Yours,
>>> Andrey
>>> ======
>>> Software Engineer
>>> Intel Compiler Team
>>> Intel
>>>
>>>
>>> On Mon, Jan 4, 2016 at 9:20 PM, Kostya Serebryany via cfe-dev
>>> <cfe-dev at lists.llvm.org> wrote:
>>> > Hi Clang devs,
>>> >
>>> > In the new year I would like to ask you all to consider fixing clang
>>> bugs
>>> > found by fuzzing (that includes, but is not limited to,
>>> > https://llvm.org/bugs/show_bug.cgi?id=23057)
>>> >
>>> > The existing fuzzer bot is reporting known bugs that are not being
>>> fixed for
>>> > months.
>>> > E.g.
>>> >
>>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/5328/steps/stage2%2Fasan%2Bassertions%20run%20clang-fuzzer/logs/stdio
>>> > This precludes us from treating these bugs as errors and make the bot
>>> red on
>>> > regressions.
>>> >
>>> > Also, these shallow bugs prevent us from finding deeper bugs with
>>> potential
>>> > security implications, and there are some such.
>>> > E.g. the bug below means that no one can safely host clang as a web
>>> service.
>>> >
>>> > echo "* a ((int () (o W, *&])) 0" | ./bin/clang -x c++ -
>>> >
>>> > ==13059==ERROR: AddressSanitizer: heap-use-after-free on address
>>> > 0x61500000e538 at pc 0x00000081df99 bp 0x7ffdbdcb3630 sp 0x7ffdbdcb2de8
>>> >
>>> > READ of size 20 at 0x61500000e538 thread T0
>>> >     #0 0x81df98 in __asan_memcpy
>>> >     #1 0xedcd28f in clang::TokenLexer::Lex(clang::Token&)
>>> > tools/clang/lib/Lex/TokenLexer.cpp:441:7
>>> >     #2 0xedb3c47 in clang::Preprocessor::Lex(clang::Token&)
>>> > tools/clang/lib/Lex/Preprocessor.cpp:731:23
>>> >     #3 0xa5ad93a in ConsumeParen
>>> > tools/clang/include/clang/Parse/Parser.h:383:5
>>> >     #4 0xa5ad93a in
>>> > clang::Parser::SkipUntil(llvm::ArrayRef<clang::tok::TokenKind>,
>>> > clang::Parser::SkipUntilFlags) tools/clang/lib/Parse/Parser.cpp:3
>>> >     #5 0xa78bdb8 in SkipUntil
>>> > tools/clang/include/clang/Parse/Parser.h:864:12
>>> >
>>> > 0x61500000e538 is located 312 bytes inside of 456-byte region
>>> > [0x61500000e400,0x61500000e5c8)
>>> > freed by thread T0 here:
>>> >     #0 0x8350db in __interceptor_free
>>> >     #1 0xa838c02 in ~SmallVectorImpl
>>> include/llvm/ADT/SmallVector.h:374:7
>>> >     #2 0xa838c02 in
>>> >
>>> clang::Parser::ParseCXXAmbiguousParenExpression(clang::Parser::ParenParseOption&,
>>> > clang::OpaquePtr<clang::QualType>&, clang::Bala
>>> >     #3 0xa7ac905 in
>>> > clang::Parser::ParseParenExpression(clang::Parser::ParenParseOption&,
>>> bool,
>>> > bool, clang::OpaquePtr<clang::QualType>&, clang::Sour
>>> >     #4 0xa794e83 in clang::Parser::ParseCastExpression(bool, bool,
>>> bool&,
>>> > clang::Parser::TypeCastState)
>>> tools/clang/lib/Parse/ParseExpr.cpp:709:11
>>> >     #5 0xa77c21c in ParseCastExpression
>>> > tools/clang/lib/Parse/ParseExpr.cpp:465:20
>>> >
>>> >
>>> > --kcc
>>> >
>>> > _______________________________________________
>>> > cfe-dev mailing list
>>> > cfe-dev at lists.llvm.org
>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>> >
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160123/6a871358/attachment.html>