[llvm-bugs] [Bug 51407] New: Infinite loop in demangler

[via llvm-bugs]

https://bugs.llvm.org/show_bug.cgi?id=51407

            Bug ID: 51407
           Summary: Infinite loop in demangler
           Product: tools
           Version: trunk
          Hardware: All
                OS: All
            Status: NEW
          Severity: normal
          Priority: P
         Component: llvm-c++filt
          Assignee: unassignedbugs at nondot.org
          Reporter: borisov.mikhail at gmail.com
                CC: llvm-bugs at lists.llvm.org

A libfuzzer run has discovered some inputs for which the demangler does not
terminate.

When minimized, it looks like this: _Zcv1BIRT_EIS1_E

Deciphered:
_Z
cv    - conversion operator

      * result type
 1B   - "B"
 I    - template args begin
  R   - reference type              <.
   T_ - forward template reference   |  *
 E    - template args end            |  |
                                     |  |
      * parameter type               |  |
 I    - template args begin          |  |
  S1_ - substitution #1              * <'
 E    - template args end

The reason is: template-parameter refs in conversion operator result type
create forward-references, while substitutions are instantly resolved via
back-references. Together these can create a reference loop. It causes an
infinite loop in ReferenceType::collapse().

I see three possible ways to avoid these loops:
1. check if resolving a forward reference creates a loop and reject the invalid
input (hard to traverse AST at this point)
2. check if a substitution contains a malicious forward reference and reject
the invalid input (hard to traverse AST at this point; substitutions are quite
common: may affect performance; hard to clearly detect loops at this point)
3. detect loops in ReferenceType::collapse() (cannot reject the input)

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20210808/b2b457c9/attachment.html>