r230061 - Fix merging of << at end of input.

[Kostya Serebryany]

On Sun, Feb 22, 2015 at 5:56 PM, Jacques Pienaar <jpienaar at google.com>
wrote:

> That is weird as I tested that locally and that was one of the cases I
> checked. And it wouldn't have exercised the code path that caused problems
> last time. I tried reproducing locally as follows:
>
> cmake -GNinja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
> -DLLVM_USE_SANITIZER=Address -DLLVM_USE_SANITIZE_COVERAGE=YES
> -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release ../ && ninja -j 10
> clang-format
>
> printf "<<<" | ASAN_OPTIONS=coverage=1 ./bin/clang-format
>

The reproducer found by the fuzzer is slightly different. Attached file
still triggers an assertion (verified on r230238).
tools/clang/include/clang/Rewrite/Core/RewriteRope.h:203: void
clang::RewriteRope::erase(unsigned int, unsigned int): Assertion
`Offset+NumBytes <= size() && "Invalid region to erase!"' failed.



>
> ninja clang-format-fuzzer
>
> ./bin/clang-format-fuzzer corpus/
>
> But none of the tests gave me an error. So if I was doing it correctly,
> then the only other thing I could think of is that the bot was using an
> older revision. I didn't see the revision number in the error report. Is
> this possible?
>
>
> On Sun, Feb 22, 2015 at 1:43 PM, Kostya Serebryany <kcc at google.com> wrote:
>
>> I don't think this is a complete fix.
>> The fuzzer is still unhappy, this time on <<<
>>
>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/442/steps/stage2%2Fasan%20run%20clang-format-fuzzer/logs/stdio
>>
>> On Fri, Feb 20, 2015 at 1:09 PM, Jacques Pienaar <jpienaar at google.com>
>> wrote:
>>
>>> Author: jpienaar
>>> Date: Fri Feb 20 15:09:01 2015
>>> New Revision: 230061
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=230061&view=rev
>>> Log:
>>> Fix merging of << at end of input.
>>>
>>> Commit of review http://reviews.llvm.org/D7766
>>>
>>>
>>> Modified:
>>>     cfe/trunk/lib/Format/Format.cpp
>>>     cfe/trunk/unittests/Format/FormatTest.cpp
>>>
>>> Modified: cfe/trunk/lib/Format/Format.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Format/Format.cpp?rev=230061&r1=230060&r2=230061&view=diff
>>>
>>> ==============================================================================
>>> --- cfe/trunk/lib/Format/Format.cpp (original)
>>> +++ cfe/trunk/lib/Format/Format.cpp Fri Feb 20 15:09:01 2015
>>> @@ -662,33 +662,26 @@ private:
>>>
>>>    bool tryMergeLessLess() {
>>>      // Merge X,less,less,Y into X,lessless,Y unless X or Y is less.
>>> -    if (Tokens.size() < 4) {
>>> -      // Merge <,<,eof to <<,eof
>>> -      if (Tokens.back()->Tok.isNot(tok::eof))
>>> -        return false;
>>> +    if (Tokens.size() < 3)
>>> +      return false;
>>>
>>> -      auto &eof = Tokens.back();
>>> -      Tokens.pop_back();
>>> -      bool LessLessMerged;
>>> -      if ((LessLessMerged = tryMergeTokens({tok::less, tok::less})))
>>> -        Tokens.back()->Tok.setKind(tok::lessless);
>>> -      Tokens.push_back(eof);
>>> -      return LessLessMerged;
>>> -    }
>>> +    bool FourthTokenIsLess = false;
>>> +    if (Tokens.size() > 3)
>>> +      FourthTokenIsLess = (Tokens.end() - 4)[0]->is(tok::less);
>>>
>>> -    auto First = Tokens.end() - 4;
>>> -    if (First[3]->is(tok::less) || First[2]->isNot(tok::less) ||
>>> -        First[1]->isNot(tok::less) || First[0]->is(tok::less))
>>> +    auto First = Tokens.end() - 3;
>>> +    if (First[2]->is(tok::less) || First[1]->isNot(tok::less) ||
>>> +        First[0]->isNot(tok::less) || FourthTokenIsLess)
>>>        return false;
>>>
>>>      // Only merge if there currently is no whitespace between the two
>>> "<".
>>> -    if (First[2]->WhitespaceRange.getBegin() !=
>>> -        First[2]->WhitespaceRange.getEnd())
>>> +    if (First[1]->WhitespaceRange.getBegin() !=
>>> +        First[1]->WhitespaceRange.getEnd())
>>>        return false;
>>>
>>> -    First[1]->Tok.setKind(tok::lessless);
>>> -    First[1]->TokenText = "<<";
>>> -    First[1]->ColumnWidth += 1;
>>> +    First[0]->Tok.setKind(tok::lessless);
>>> +    First[0]->TokenText = "<<";
>>> +    First[0]->ColumnWidth += 1;
>>>      Tokens.erase(Tokens.end() - 2);
>>>      return true;
>>>    }
>>>
>>> Modified: cfe/trunk/unittests/Format/FormatTest.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Format/FormatTest.cpp?rev=230061&r1=230060&r2=230061&view=diff
>>>
>>> ==============================================================================
>>> --- cfe/trunk/unittests/Format/FormatTest.cpp (original)
>>> +++ cfe/trunk/unittests/Format/FormatTest.cpp Fri Feb 20 15:09:01 2015
>>> @@ -9752,6 +9752,7 @@ TEST_F(FormatTest, TripleAngleBrackets)
>>>  }
>>>
>>>  TEST_F(FormatTest, MergeLessLessAtEnd) {
>>> +  verifyFormat("<<");
>>>
>>>  verifyFormat("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
>>>                 "aaallvm::outs() <<");
>>>
>>>  verifyFormat("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
>>>
>>>
>>> _______________________________________________
>>> cfe-commits mailing list
>>> cfe-commits at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20150223/4f8cf3ce/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: crash-3030501594194
Type: application/octet-stream
Size: 5 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20150223/4f8cf3ce/attachment.obj>