[clang] [analyzer] Model overflow builtins (PR #102602)

[Donát Nagy via cfe-commits]

================
@@ -50,6 +118,75 @@ class BuiltinFunctionChecker : public Checker<eval::Call> {
 
 } // namespace
 
+std::pair<bool, bool>
+BuiltinFunctionChecker::checkOverflow(CheckerContext &C, SVal RetVal,
+                                      QualType Res) const {
+  ProgramStateRef State = C.getState();
+  SValBuilder &SVB = C.getSValBuilder();
+  ASTContext &ACtx = C.getASTContext();
+
+  // Calling a builtin with a non-integer type result produces compiler error.
+  assert(Res->isIntegerType());
+
+  unsigned BitWidth = ACtx.getIntWidth(Res);
+  auto MinVal =
+      llvm::APSInt::getMinValue(BitWidth, Res->isUnsignedIntegerType());
+  auto MaxVal =
+      llvm::APSInt::getMaxValue(BitWidth, Res->isUnsignedIntegerType());
+
+  SVal IsLeMax =
+      SVB.evalBinOp(State, BO_LE, RetVal, nonloc::ConcreteInt(MaxVal), Res);
+  SVal IsGeMin =
+      SVB.evalBinOp(State, BO_GE, RetVal, nonloc::ConcreteInt(MinVal), Res);
+
+  auto [MayNotOverflow, MayOverflow] =
+      State->assume(IsLeMax.castAs<DefinedOrUnknownSVal>());
+  auto [MayNotUnderflow, MayUnderflow] =
+      State->assume(IsGeMin.castAs<DefinedOrUnknownSVal>());
+
+  return {MayOverflow || MayUnderflow, MayNotOverflow && MayNotUnderflow};
+}
+
+void BuiltinFunctionChecker::handleOverflowBuiltin(const CallEvent &Call,
+                                                   CheckerContext &C,
+                                                   BinaryOperator::Opcode Op,
+                                                   QualType ResultType) const {
+  // Calling a builtin with an incorrect argument count produces compiler error.
+  assert(Call.getNumArgs() == 3);
+
+  ProgramStateRef State = C.getState();
+  SValBuilder &SVB = C.getSValBuilder();
+  const Expr *CE = Call.getOriginExpr();
+
+  SVal Arg1 = Call.getArgSVal(0);
+  SVal Arg2 = Call.getArgSVal(1);
+
+  SVal RetValMax = SVB.evalBinOp(State, Op, Arg1, Arg2,
+                                 getSufficientTypeForOverflowOp(C, ResultType));
+  SVal RetVal = SVB.evalBinOp(State, Op, Arg1, Arg2, ResultType);
+
+  auto [Overflow, NotOverflow] = checkOverflow(C, RetValMax, ResultType);
+  if (NotOverflow) {
+    ProgramStateRef StateNoOverflow =
+        State->BindExpr(CE, C.getLocationContext(), SVB.makeTruthVal(false));
+
+    if (auto L = Call.getArgSVal(2).getAs<Loc>()) {
+      StateNoOverflow =
+          StateNoOverflow->bindLoc(*L, RetVal, C.getLocationContext());
+
+      // Propagate taint if any of the argumets were tainted
+      if (isTainted(State, Arg1) || isTainted(State, Arg2))
+        StateNoOverflow = addTaint(StateNoOverflow, *L);
+    }
+
+    C.addTransition(StateNoOverflow);
----------------
NagyDonat wrote:

> As far as I see, markInteresting is called on bug reported object. And there is no bug report object, since there is no bug here, we just do assumtions. I tried to find an example in other checkers, but I don't see it.

Yes, that's a good question -- the trick is that when you create a `NoteTag`, you can specify a callback that takes a `PathSensitiveBugReport` (by reference) and you should use `markInteresting()` within that callback. You could try to follow the example of `InvalidPtrChecker::createEnvInvalidationNote()` within `InvalidPtrChecker.cpp`, which implements something vaguely similar.

> (Like I see manipulations with markInteresting only during construction of bug report).

When a checker creates a bug report, it can use `markInteresting()` to mark the symbols that are _interesting_ (i.e. we used knowledge about them to conclude that the situation is buggy).

However, in addition to creating bug reports, checkers can also create `NoteTag`s when they do something interesting (e.g. make an assumption, model something important etc.) which is not a bug report. There are many trivial `NoteTag`s that just print the same plain text message on every bug report that passes through the execution step where the `NoteTag` was created, but in general the `NoteTag` may have an associated callback which inspects and modifies the `PathSensitiveBugReport` (in addition to emitting a "note" message).

The idea is that you should create a note tag with a callback which:
- checks whether the return value of the `__builtin_X_overflow()` is interesting and returns early if it isn't,
- marks the incoming arguments as interesting (if they're symbolic) to propagate interestingness;
- if both cases ("overflow" and "no overflow") were feasible, then returns a string message which shows the assumption that was selected.
(This is just a rough draft, there may be intricacies e.g. with describing taintedness.)

https://github.com/llvm/llvm-project/pull/102602