[LLVMdev] Win32: Crash in DLL created by llvm that calls into the "putchar" function

[Daniel Albuschat]

Hello list,

I got a simple test case to reproduce a crash in code that has been
generated by llvm. The code calls into the "putchar" function from
LIBCMT.LIB. It works when creating an .exe file, but not when creating
a .dll file and calling into an exported function.

First, let me show how the code works as an .exe:
> cat callPutchar.ll
; ModuleID = 'test'

declare i32 @putchar(i32)

define i32 @main() {
__entry__:
  %0 = call i32 @putchar(i32 79)
  ret i32 %0
}

> lli callPutchar.ll ; Test, whether the code itself works. Output is as expected an 'O' (ASCII 79):
O

> llc -filetype obj callPutchar.ll ; Create an .obj file and later link it with MSVC's link.exe:

> link.exe /SUBSYSTEM:CONSOLE callPutchar.obj LIBCMT.LIB
Microsoft (R) Incremental Linker Version 11.00.60610.1
Copyright (C) Microsoft Corporation.  All rights reserved.


> callPutchar.exe ; Call the newly created .exe file. Output is as expected an 'O':
O

Now, when I do something very similar, but with a DLL instead of an
EXE file and call into the exported function, I get an access
violation in the generated function:

> cat callPutchar.ll
; ModuleID = 'test'

declare i32 @putchar(i32)

define dllexport i32 @main() {
__entry__:
  %0 = call i32 @putchar(i32 79)
  ret i32 %0
}

> llc -filetype obj callPutchar.ll

> link .exe/DLL /NOENTRY /SUBSYSTEM:CONSOLE callPutchar.obj LIBCMT.LIB
Microsoft (R) Incremental Linker Version 11.00.60610.1
Copyright (C) Microsoft Corporation.  All rights reserved.

   Bibliothek "callPutchar.lib" und Objekt "callPutchar.exp" werden erstellt.

> demoDllCaller.exe ; >> The typical "demoDllCaller.exe stopped working..." Windows crash dialog appears.

demoDllCaller is a small application that I wrote that loads the dll
and calls the main function. Here's the code of demoDllCaller.exe:

#include <Windows.h>

int main ()
{
    HMODULE lib = LoadLibrary (L"callPutchar.dll");
    if (lib != nullptr) {
        typedef int __declspec(cdecl) (*main1_t) ();
        main1_t main1 = (main1_t) GetProcAddress (lib, "main");
        if (main1 != nullptr) {
            int result = (* main1) ();
            result = result;
        }
    }
}

I'm on Windows 8.1 using Visual Studio 2012 and 2013 (crash happens
with both). I compiled llvm with Visual Studio 2013 (exact version is
"12.0.21005.1 REL").
There is no usable stacktrace  that I could show you.

What seems a little odd to me, too, is that the callPutchar.dll is
62MB in size. callPutchar.exe is 70MB. A bit much I guess ;)

You can get all files (callPutchar.ll, .obj, .dll, .exe, .exp and .lib) here:
https://dl.dropboxusercontent.com/u/1913181/Temp/callPutchar.7z

Greetings,
Daniel Albuschat