[cfe-dev] Segfault in ASTContext.cpp

[Artem Dergachev via cfe-dev]

The code definitely does look like it's missing some useful assertions, 
but i'm also wondering how come that we have to do something during 
symbolic execution that doesn't ever happen during "concrete" execution. 
That's usually a good indication that we're doing something wrong - not 
even in SMT API but above it, in the code that calls it.

On 6/28/19 7:43 AM, Mikhail Ramalho via cfe-dev wrote:
> Hello everyone,
>
> As part of the SMT encoding of constraints done in the CSA, we are 
> triggering a segfault when trying to get the integer type order 
> between an enumType and an integer.
>
> The code from ASTContext.cpp, starting in line 5814:
>
> static const Type *getIntegerTypeForEnum(const EnumType *ET) {
>   // Incomplete enum types are not treated as integer types.
>   // FIXME: In C++, enum types are never integer types.
>   if (ET->getDecl()->isComplete() && !ET->getDecl()->isScoped())
>     return ET->getDecl()->getIntegerType().getTypePtr();
> *  return nullptr; // [1]*
> }
>
> /// getIntegerTypeOrder - Returns the highest ranked integer type:
> /// C99 6.3.1.8p1.  If LHS > RHS, return 1.  If LHS == RHS, return 0. If
> /// LHS < RHS, return -1.
> int ASTContext::getIntegerTypeOrder(QualType LHS, QualType RHS) const {
>   const Type *LHSC = getCanonicalType(LHS).getTypePtr();
>   const Type *RHSC = getCanonicalType(RHS).getTypePtr();
>
>   // Unwrap enums to their underlying type.
> *  if (const auto *ET = dyn_cast<EnumType>(LHSC))
>     LHSC = getIntegerTypeForEnum(ET); **// [0]*
>   if (const auto *ET = dyn_cast<EnumType>(RHSC))
>     RHSC = getIntegerTypeForEnum(ET);
>
>   if (LHSC == RHSC) return 0;
>
> *  bool LHSUnsigned = LHSC->isUnsignedIntegerType(); **// [2]*
>   bool RHSUnsigned = RHSC->isUnsignedIntegerType();
>
> The problem happens when LHS (or RHS) is a C++11 enum: 
> getIntegerTypeForEnum [0] is called but returns a nullptr [1], and 
> later we try to call a method from the nullptr [2].
>
> ~
>
> Any suggestion of how we can fix/work around it? It doesn't seem to 
> affect Sema in any way, but the code seems rather fragile...
>
> I hacked around it by doing:
> -  return nullptr;
> +  return ET->getDecl()->getPromotionType().getTypePtr();
>
> But it does not seem right, there is even a comment there that reads:
> // FIXME: In C++, enum types are never integer types.
>
> Thank you,
>
> -- 
>
> Mikhail Ramalho.
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190628/abd0e5f8/attachment.html>