[cfe-dev] fixing bugs fuzzed out of clang

Fri Jan 22 13:56:51 PST 2016

On Fri, Jan 22, 2016 at 6:00 AM, Dmitry Polukhin <dmitry.polukhin at gmail.com>
wrote:

> Hi Kostya,
>
> I would like to repro and fix some of the issues that fuzzed found
>

Nice!

> but I don't have system Clang on my machine but I do have Clang sources so
> I need to make Clang to have ASan and after that use that Clang to build
> Clang one more time but with ASan enabled. It looks like in Clang doc there
> is no info how to do it.
>

Mmmm.
These pages have some related discussions, but you probably don't need
them...
https://github.com/google/sanitizers/wiki/MemorySanitizerBootstrappingClang
https://github.com/google/sanitizers/wiki/AddressSanitizerHowToBuild

> I only found LLVM_USE_SANITIZER so I did it like this:
>
> cd $ROOT
> mkdir release
> cd release
> cmake ../llvm -DCMAKE_BUILD_TYPE=Release
> make
> cd ..
> mkdir build
> cd build
> CC=$ROOT/release/bin/clang CXX=$ROOT/release/bin/clang++ cmake ../llvm
> -DCMAKE_BUILD_TYPE=Debug -DLLVM_USE_SANITIZER=Address
> make
>
> Last build seems to hang on compilation of llvm/lib/IR/Function.cpp.
>

I routinely build clang with clang+asan.
llvm/lib/IR/Function.cpp indeed takes lots of time, an order of 3 minutes,
but not even close to what you see.
Mostly likely the difference is that you use -DCMAKE_BUILD_TYPE=Debug
and I use -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON
(in other words, I use -O2).
In general, we almost never use asan with -O0 because it'll be too slow.

BTW, I encourage you to switch from "make" to "ninja" -- works much faster
and nicer

> I break it after about 30min and 25G RAM consumed.
> Without -fsanitize=address compilation takes about 30 sec. Do you have bot
> that checks self build with ASan?
>

Yes.
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap
You can check the exact commands e.g. here:
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/10702/steps/build%20clang%2Fasan/logs/stdio
cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON ...

> Is it known issue?
>

I would guess what you see is a manifestation of
https://llvm.org/bugs/show_bug.cgi?id=17409,
which has been bothering us for years, but I have not checked this
particular case.

Thanks for doing this!

--kcc

> Thanks,
> Dmitry
> --
> Software Engineer
> Intel Compiler Team
>
> On Tue, Jan 5, 2016 at 1:46 PM, Andrey Bokhanko via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>> We (Intel clang team) will take a look and fix some of these.
>>
>> Yours,
>> Andrey
>> ======
>> Software Engineer
>> Intel Compiler Team
>> Intel
>>
>>
>> On Mon, Jan 4, 2016 at 9:20 PM, Kostya Serebryany via cfe-dev
>> <cfe-dev at lists.llvm.org> wrote:
>> > Hi Clang devs,
>> >
>> > In the new year I would like to ask you all to consider fixing clang
>> bugs
>> > found by fuzzing (that includes, but is not limited to,
>> > https://llvm.org/bugs/show_bug.cgi?id=23057)
>> >
>> > The existing fuzzer bot is reporting known bugs that are not being
>> fixed for
>> > months.
>> > E.g.
>> >
>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/5328/steps/stage2%2Fasan%2Bassertions%20run%20clang-fuzzer/logs/stdio
>> > This precludes us from treating these bugs as errors and make the bot
>> red on
>> > regressions.
>> >
>> > Also, these shallow bugs prevent us from finding deeper bugs with
>> potential
>> > security implications, and there are some such.
>> > E.g. the bug below means that no one can safely host clang as a web
>> service.
>> >
>> > echo "* a ((int () (o W, *&])) 0" | ./bin/clang -x c++ -
>> >
>> > ==13059==ERROR: AddressSanitizer: heap-use-after-free on address
>> > 0x61500000e538 at pc 0x00000081df99 bp 0x7ffdbdcb3630 sp 0x7ffdbdcb2de8
>> >
>> > READ of size 20 at 0x61500000e538 thread T0
>> >     #0 0x81df98 in __asan_memcpy
>> >     #1 0xedcd28f in clang::TokenLexer::Lex(clang::Token&)
>> > tools/clang/lib/Lex/TokenLexer.cpp:441:7
>> >     #2 0xedb3c47 in clang::Preprocessor::Lex(clang::Token&)
>> > tools/clang/lib/Lex/Preprocessor.cpp:731:23
>> >     #3 0xa5ad93a in ConsumeParen
>> > tools/clang/include/clang/Parse/Parser.h:383:5
>> >     #4 0xa5ad93a in
>> > clang::Parser::SkipUntil(llvm::ArrayRef<clang::tok::TokenKind>,
>> > clang::Parser::SkipUntilFlags) tools/clang/lib/Parse/Parser.cpp:3
>> >     #5 0xa78bdb8 in SkipUntil
>> > tools/clang/include/clang/Parse/Parser.h:864:12
>> >
>> > 0x61500000e538 is located 312 bytes inside of 456-byte region
>> > [0x61500000e400,0x61500000e5c8)
>> > freed by thread T0 here:
>> >     #0 0x8350db in __interceptor_free
>> >     #1 0xa838c02 in ~SmallVectorImpl
>> include/llvm/ADT/SmallVector.h:374:7
>> >     #2 0xa838c02 in
>> >
>> clang::Parser::ParseCXXAmbiguousParenExpression(clang::Parser::ParenParseOption&,
>> > clang::OpaquePtr<clang::QualType>&, clang::Bala
>> >     #3 0xa7ac905 in
>> > clang::Parser::ParseParenExpression(clang::Parser::ParenParseOption&,
>> bool,
>> > bool, clang::OpaquePtr<clang::QualType>&, clang::Sour
>> >     #4 0xa794e83 in clang::Parser::ParseCastExpression(bool, bool,
>> bool&,
>> > clang::Parser::TypeCastState) tools/clang/lib/Parse/ParseExpr.cpp:709:11
>> >     #5 0xa77c21c in ParseCastExpression
>> > tools/clang/lib/Parse/ParseExpr.cpp:465:20
>> >
>> >
>> > --kcc
>> >
>> > _______________________________________________
>> > cfe-dev mailing list
>> > cfe-dev at lists.llvm.org
>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>> >
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160122/f56abe7f/attachment.html>