[llvm-bugs] [Bug 45712] New: [LTO] -fwhole-program-vtables causes bad generated code in hybrid lto/non-lto build

[via llvm-bugs]

https://bugs.llvm.org/show_bug.cgi?id=45712

            Bug ID: 45712
           Summary: [LTO] -fwhole-program-vtables causes bad generated
                    code in hybrid lto/non-lto build
           Product: libraries
           Version: trunk
          Hardware: PC
                OS: Windows NT
            Status: NEW
          Severity: normal
          Priority: P
         Component: Interprocedural Optimizations
          Assignee: unassignedbugs at nondot.org
          Reporter: sylvain.audi at ubisoft.com
                CC: llvm-bugs at lists.llvm.org, peter at pcc.me.uk,
                    rnk at google.com, tejohnson at google.com

When trying to enable Thin-LTO on one of our games, we found invalid generated
code which caused an infinite loop when we enabled -the
`-fwhole-program-vtables`.
The bug also appeared when we tried LTO.

I mentionned this bug in comments of D55153

Note that we build the game against some libraries that were not built with
LTO/ThinLTO.


Here it is, reduced as a relatively small repro:

$cat main.cpp
#include "inc.h"
extern void UseExternal();
int main() {
  UseExternal();
  return 0;
}

$cat inc.h
class A {
public:
  virtual void f() {
    pNext->f();
  }
  A* pNext = nullptr;
};

class B : public A {
public:
  virtual void f() {}
};

$cat external.cpp
#include "inc.h"
static A s_A;
static B s_B;
void UseExternal() {
  s_A.pNext = &s_B;
  (&s_A)->f();
}

$cat def.cpp
#include "inc.h"
A g_A;

$clang-cl main.cpp def.cpp -c /Z7 /O2 /GR- -flto -fwhole-program-vtables
$clang-cl extern.cpp -c /Z7 /O2 /GR-
$lld-link /NODEFAULTLIB /ENTRY:main /OUT:main.exe def.obj main.obj external.obj

In the generated code, we can see that A::f() is generated as an endless loop.

14000103f: cc       int3
140001040: eb fe    jmp    0x140001040     ; endless loop
140001042: cc       int3 

It looks like it was wrongly devirtualized:
  virtual void f() {
    pNext->f();
  }
  where pNext->f() becomes an endless recursion.

IMPORTANT: The bug appears only if def.obj is the first object in the lld-link
command.

This bug is pretty fragile to reproduce.
The repro above is with clang-cl / lld-link; I didn't manage to repro it under
clang/ld.lld.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20200428/44203faa/attachment.html>