r230061 - Fix merging of << at end of input.

[Kostya Serebryany]

On Mon, Feb 23, 2015 at 11:07 AM, Kostya Serebryany <kcc at google.com> wrote:

>
>
> On Sun, Feb 22, 2015 at 5:56 PM, Jacques Pienaar <jpienaar at google.com>
> wrote:
>
>> That is weird as I tested that locally and that was one of the cases I
>> checked. And it wouldn't have exercised the code path that caused problems
>> last time. I tried reproducing locally as follows:
>>
>> cmake -GNinja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
>> -DLLVM_USE_SANITIZER=Address -DLLVM_USE_SANITIZE_COVERAGE=YES
>> -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release ../ && ninja -j 10
>> clang-format
>>
>> printf "<<<" | ASAN_OPTIONS=coverage=1 ./bin/clang-format
>>
>
> The reproducer found by the fuzzer is slightly different. Attached file
> still triggers an assertion (verified on r230238).
> tools/clang/include/clang/Rewrite/Core/RewriteRope.h:203: void
> clang::RewriteRope::erase(unsigned int, unsigned int): Assertion
> `Offset+NumBytes <= size() && "Invalid region to erase!"' failed.
>
>
Or do like this:
printf "\\\\\x0a<<<" | clang-format



>
>
>>
>> ninja clang-format-fuzzer
>>
>> ./bin/clang-format-fuzzer corpus/
>>
>> But none of the tests gave me an error. So if I was doing it correctly,
>> then the only other thing I could think of is that the bot was using an
>> older revision. I didn't see the revision number in the error report. Is
>> this possible?
>>
>>
>> On Sun, Feb 22, 2015 at 1:43 PM, Kostya Serebryany <kcc at google.com>
>> wrote:
>>
>>> I don't think this is a complete fix.
>>> The fuzzer is still unhappy, this time on <<<
>>>
>>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fuzzer/builds/442/steps/stage2%2Fasan%20run%20clang-format-fuzzer/logs/stdio
>>>
>>> On Fri, Feb 20, 2015 at 1:09 PM, Jacques Pienaar <jpienaar at google.com>
>>> wrote:
>>>
>>>> Author: jpienaar
>>>> Date: Fri Feb 20 15:09:01 2015
>>>> New Revision: 230061
>>>>
>>>> URL: http://llvm.org/viewvc/llvm-project?rev=230061&view=rev
>>>> Log:
>>>> Fix merging of << at end of input.
>>>>
>>>> Commit of review http://reviews.llvm.org/D7766
>>>>
>>>>
>>>> Modified:
>>>>     cfe/trunk/lib/Format/Format.cpp
>>>>     cfe/trunk/unittests/Format/FormatTest.cpp
>>>>
>>>> Modified: cfe/trunk/lib/Format/Format.cpp
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Format/Format.cpp?rev=230061&r1=230060&r2=230061&view=diff
>>>>
>>>> ==============================================================================
>>>> --- cfe/trunk/lib/Format/Format.cpp (original)
>>>> +++ cfe/trunk/lib/Format/Format.cpp Fri Feb 20 15:09:01 2015
>>>> @@ -662,33 +662,26 @@ private:
>>>>
>>>>    bool tryMergeLessLess() {
>>>>      // Merge X,less,less,Y into X,lessless,Y unless X or Y is less.
>>>> -    if (Tokens.size() < 4) {
>>>> -      // Merge <,<,eof to <<,eof
>>>> -      if (Tokens.back()->Tok.isNot(tok::eof))
>>>> -        return false;
>>>> +    if (Tokens.size() < 3)
>>>> +      return false;
>>>>
>>>> -      auto &eof = Tokens.back();
>>>> -      Tokens.pop_back();
>>>> -      bool LessLessMerged;
>>>> -      if ((LessLessMerged = tryMergeTokens({tok::less, tok::less})))
>>>> -        Tokens.back()->Tok.setKind(tok::lessless);
>>>> -      Tokens.push_back(eof);
>>>> -      return LessLessMerged;
>>>> -    }
>>>> +    bool FourthTokenIsLess = false;
>>>> +    if (Tokens.size() > 3)
>>>> +      FourthTokenIsLess = (Tokens.end() - 4)[0]->is(tok::less);
>>>>
>>>> -    auto First = Tokens.end() - 4;
>>>> -    if (First[3]->is(tok::less) || First[2]->isNot(tok::less) ||
>>>> -        First[1]->isNot(tok::less) || First[0]->is(tok::less))
>>>> +    auto First = Tokens.end() - 3;
>>>> +    if (First[2]->is(tok::less) || First[1]->isNot(tok::less) ||
>>>> +        First[0]->isNot(tok::less) || FourthTokenIsLess)
>>>>        return false;
>>>>
>>>>      // Only merge if there currently is no whitespace between the two
>>>> "<".
>>>> -    if (First[2]->WhitespaceRange.getBegin() !=
>>>> -        First[2]->WhitespaceRange.getEnd())
>>>> +    if (First[1]->WhitespaceRange.getBegin() !=
>>>> +        First[1]->WhitespaceRange.getEnd())
>>>>        return false;
>>>>
>>>> -    First[1]->Tok.setKind(tok::lessless);
>>>> -    First[1]->TokenText = "<<";
>>>> -    First[1]->ColumnWidth += 1;
>>>> +    First[0]->Tok.setKind(tok::lessless);
>>>> +    First[0]->TokenText = "<<";
>>>> +    First[0]->ColumnWidth += 1;
>>>>      Tokens.erase(Tokens.end() - 2);
>>>>      return true;
>>>>    }
>>>>
>>>> Modified: cfe/trunk/unittests/Format/FormatTest.cpp
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Format/FormatTest.cpp?rev=230061&r1=230060&r2=230061&view=diff
>>>>
>>>> ==============================================================================
>>>> --- cfe/trunk/unittests/Format/FormatTest.cpp (original)
>>>> +++ cfe/trunk/unittests/Format/FormatTest.cpp Fri Feb 20 15:09:01 2015
>>>> @@ -9752,6 +9752,7 @@ TEST_F(FormatTest, TripleAngleBrackets)
>>>>  }
>>>>
>>>>  TEST_F(FormatTest, MergeLessLessAtEnd) {
>>>> +  verifyFormat("<<");
>>>>
>>>>  verifyFormat("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
>>>>                 "aaallvm::outs() <<");
>>>>
>>>>  verifyFormat("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
>>>>
>>>>
>>>> _______________________________________________
>>>> cfe-commits mailing list
>>>> cfe-commits at cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20150223/5b61a802/attachment.html>