[libcxx-commits] [PATCH] D107712: Fix possible infinite loop in itanium demangler

[Mikhail Borisov via Phabricator via libcxx-commits]

borman created this revision.
borman added a reviewer: libcxx-commits.
borman added a project: libc++abi.
borman requested review of this revision.
Herald added a reviewer: libc++abi.
Herald added a project: LLVM.

A libfuzzer run has discovered some inputs for which the demangler does not terminate.

When minimized, it looks like this: `_Zcv1BIRT_EIS1_E`

Deciphered:

  _Z
  cv    - conversion operator
  
        * result type
   1B   - "B"
   I    - template args begin
    R   - reference type              <.
     T_ - forward template reference   |  *
   E    - template args end            |  |
                                       |  |
        * parameter type               |  |
   I    - template args begin          |  |
    S1_ - substitution #1              * <'
   E    - template args end

The reason is: template-parameter refs in conversion operator result type create forward-references, while substitutions are instantly resolved via back-references. Together these can create a reference loop. It causes an infinite loop in ReferenceType::collapse().

I see three possible ways to avoid these loops:

1. check if resolving a forward reference creates a loop and reject the invalid input (hard to traverse AST at this point)
2. check if a substitution contains a malicious forward reference and reject the invalid input (hard to traverse AST at this point; substitutions are quite common: may affect performance; hard to clearly detect loops at this point)
3. detect loops in `ReferenceType::collapse()` (cannot reject the input)

This patch implements (3) as seemingly the least-impact change. As a side effect, such invalid input strings are not rejected and produce garbage, however there are already similar guards in `if (Printing) return;` checks.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D107712

Files:
  libcxxabi/src/demangle/ItaniumDemangle.h
  libcxxabi/test/test_demangle.pass.cpp
  llvm/include/llvm/Demangle/ItaniumDemangle.h

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D107712.365012.patch
Type: text/x-patch
Size: 17502 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/libcxx-commits/attachments/20210808/cf14784f/attachment-0001.bin>