[cfe-dev] Multiple symbol definition problems when linking clang tools on Windows

[Kenneth MacKenzie via cfe-dev]

Hello,

We've got a fairly large clang tool which builds and runs successfully
on both Linux and OSX, but we're getting linking errors on Windows.
We've tried both Cygwin and MSYS2/MinGW.  The tool uses the
CommonOptionsParser class, and the problems on Cygwin seem to
be related to this (although things are worse with MSYS2: see later).

A small program which demonstrates our problem is attached at the end
of the message, together with a makefile.  All it does is to take some
command line options and print out their values.  This program shows the 
same behaviour as our larger one.

On Cygwin, if we compile the example with gcc then it compiles and links
successfully, but none of our options appear if we run the tool with
-help, and any non-standard options which we supply cause errors.

The behaviour is different if we use clang (both 3.9.1 installed via
the package manager and a recent version of 4.0.0 built from source).
In this case, the program compiles correctly, but the linker fails
with several errors caused by multiple definitions of symbols.  The
full output of the compiler is attached at the end of the message, but
there are essentially two different types of error.  An example of the
first is:

/usr/lib/libLLVMSupport.a(CommandLine.cpp.o): In function 
`llvm::cl::OptionValueCopy<bool>::compare(bool const&) const':
/usr/src/debug/llvm-3.8.1-1/include/llvm/Support/CommandLine.h:420: 
multiple definition of 
`llvm::cl::OptionValueCopy<bool>::compare(llvm::cl::GenericOptionValue 
const&) const'
main.o:(.text[_ZNK4llvm2cl15OptionValueCopyIbE7compareERKNS0_18GenericOptionValueE]+0x0): 
first defined here

Applying nm to main.o gives

0000000000000000 T 
_ZNK4llvm2cl15OptionValueCopyIbE7compareERKNS0_18GenericOptionValueE

and the same for libLLVMSupport.a.

On Linux, the output of nm for this symbol is

0000000000000000 W 
_ZNK4llvm2cl15OptionValueCopyIbE7compareERKNS0_18GenericOptionValueE

and the same for libLLVMSupport.a


If I'm interpreting this correctly, the function definition is
escaping from the object file on Cygwin and clashing with the value in
the library, but on Linux the symbol is weak and the
clash does not occur.


The second type of error message says

/usr/lib/libLLVMSupport.a(CommandLine.cpp.o): In function 
`llvm::cl::parser<char>::anchor()':
/usr/src/debug/llvm-3.8.1-1/lib/Support/CommandLine.cpp:81: multiple 
definition of `vtable for llvm::cl::OptionValueCopy<std::string>'
main.o:(.rdata[_ZTVN4llvm2cl15OptionValueCopyISsEE]+0x0): first defined here

There are comments in CommandLine.h and CommandLine.cpp that say
that the anchor() methods are the to "pin the vtable to this file".
I don't fully understand this, but presumably it's attempting to
ensure that vtables for various template instantiations are
generated only in the object file containing the contents of
CommandLine.cpp; unfortunately, this doesn't seem to be working
on Cygwin, and again we're getting clashes.


If we remove the use of CommonOptionsParser in our tool, then the
whole thing compiles, links, and runs on Cygwin without any problems
Using both clang and gcc).  However, on MSYS2, we still get many
linking errors for duplicate symbols if we use clang, but now involving
classes including ASTConsumer and FrontEndAction, and we can't work
around these.

It's not clear to me exactly where in the compilation process these 
problems are arising, and it's even less clear how we might solve them.
Does anyone have any suggestions?

Thanks,

Kenneth

================= main.cpp =================

#include "clang/Tooling/CommonOptionsParser.h"

#include <iostream>

using namespace clang;
using namespace std;

llvm::cl::OptionCategory TestCategory("Clang option parser link test");

llvm::cl::opt<std::string>
Action(
        "action",
        llvm::cl::desc("action: string parameter"),
        llvm::cl::cat(TestCategory));

llvm::cl::opt<int>
Count(
     "count",
     llvm::cl::desc("count: integer parameter"),
     llvm::cl::cat(TestCategory));

llvm::cl::opt<bool>
Debug(
       "debug",
       llvm::cl::desc("debug: boolean parameter"),
       llvm::cl::cat(TestCategory));


clang::tooling::CommonOptionsParser OptionsParser(argc, argv, TestCategory);

   cout << "action: " << Action << endl;
   cout << "count:  " << Count << endl;
   cout << "debug:  " << Debug << endl;

   return 0;

}

================= Makefile =================
LLVM_CONFIG=llvm-config

PREFIX = $(shell $(LLVM_CONFIG) --prefix)
INCDIR = $(PREFIX)/include

# ^ This should be a directory containing 
clang/Tooling/CommonOptionsParser.h

CXX=clang++

CXXFLAGS = -std=c++11 \
            -fno-rtti \
            -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS \
            -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS

CXXFLAGS += -I$(INCDIR)

LIBS := $(shell $(LLVM_CONFIG) --ldflags --libs --system-libs)

CLANG_LIBS := \
	-Wl,--start-group \
	-lclangAST \
	-lclangAnalysis \
	-lclangBasic \
	-lclangDriver \
	-lclangEdit \
	-lclangFrontend \
	-lclangLex \
	-lclangParse \
	-lclangSema \
	-lclangSerialization \
	-lclangTooling \
	-Wl,--end-group

all: main

OBJS=main.o

main: $(OBJS)
	$(CXX) $(OBJS) $(CLANG_LIBS) $(LIBS) -o $@

main.o: main.cpp

================= Output of buid process =================
clang++ -std=c++11 -fno-rtti -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS 
-D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/usr/include   -c -o 
main.o main.cpp
clang++ main.o -Wl,--start-group -lclangAST -lclangAnalysis -lclangBasic 
-lclangDriver -lclangEdit -lclangFrontend -lclangLex -lclangParse 
-lclangSema -lclangSerialization -lclangTooling -Wl,--end-group 
-L/usr/lib  -lLLVMLTO -lLLVMObjCARCOpts -lLLVMSymbolize 
-lLLVMDebugInfoPDB -lLLVMDebugInfoDWARF -lLLVMMIRParser -lLLVMTableGen 
-lLLVMOrcJIT -lLLVMCppBackendCodeGen -lLLVMCppBackendInfo 
-lLLVMBPFCodeGen -lLLVMBPFDesc -lLLVMBPFInfo -lLLVMBPFAsmPrinter 
-lLLVMNVPTXCodeGen -lLLVMNVPTXDesc -lLLVMNVPTXInfo -lLLVMNVPTXAsmPrinter 
-lLLVMAMDGPUCodeGen -lLLVMAMDGPUAsmParser -lLLVMAMDGPUDesc 
-lLLVMAMDGPUUtils -lLLVMAMDGPUInfo -lLLVMAMDGPUAsmPrinter 
-lLLVMX86Disassembler -lLLVMX86AsmParser -lLLVMX86CodeGen 
-lLLVMSelectionDAG -lLLVMAsmPrinter -lLLVMX86Desc -lLLVMMCDisassembler 
-lLLVMX86Info -lLLVMX86AsmPrinter -lLLVMX86Utils -lLLVMMCJIT 
-lLLVMLibDriver -lLLVMOption -lLLVMLineEditor -lLLVMPasses -lLLVMipo 
-lLLVMVectorize -lLLVMLinker -lLLVMIRReader -lLLVMAsmParser 
-lLLVMDebugInfoCodeView -lLLVMInterpreter -lLLVMExecutionEngine 
-lLLVMRuntimeDyld -lLLVMCodeGen -lLLVMTarget -lLLVMScalarOpts 
-lLLVMInstCombine -lLLVMInstrumentation -lLLVMProfileData -lLLVMObject 
-lLLVMMCParser -lLLVMTransformUtils -lLLVMMC -lLLVMBitWriter 
-lLLVMBitReader -lLLVMAnalysis -lLLVMCore -lLLVMSupport -lrt -ldl 
-lcurses -lpthread -lz -lm -o main

/usr/lib/libLLVMSupport.a(CommandLine.cpp.o): In function 
`llvm::cl::OptionValueCopy<bool>::compare(bool const&) const':
/usr/src/debug/llvm-3.8.1-1/include/llvm/Support/CommandLine.h:420: 
multiple definition of 
`llvm::cl::OptionValueCopy<bool>::compare(llvm::cl::GenericOptionValue 
const&) const'
main.o:(.text[_ZNK4llvm2cl15OptionValueCopyIbE7compareERKNS0_18GenericOptionValueE]+0x0): 
first defined here

/usr/lib/libLLVMSupport.a(CommandLine.cpp.o): In function 
`llvm::cl::parser<char>::anchor()':
/usr/src/debug/llvm-3.8.1-1/lib/Support/CommandLine.cpp:81: multiple 
definition of 
`llvm::cl::OptionValueCopy<int>::compare(llvm::cl::GenericOptionValue 
const&) const'
main.o:(.text[_ZNK4llvm2cl15OptionValueCopyIiE7compareERKNS0_18GenericOptionValueE]+0x0): 
first defined here

/usr/lib/libLLVMSupport.a(CommandLine.cpp.o): In function 
`llvm::cl::parser<char>::anchor()':
/usr/src/debug/llvm-3.8.1-1/lib/Support/CommandLine.cpp:81: multiple 
definition of 
`llvm::cl::OptionValueCopy<std::string>::compare(llvm::cl::GenericOptionValue 
const&) const'
main.o:(.text[_ZNK4llvm2cl15OptionValueCopyISsE7compareERKNS0_18GenericOptionValueE]+0x0): 
first defined here

/usr/lib/libLLVMSupport.a(CommandLine.cpp.o): In function 
`llvm::cl::parser<char>::anchor()':
/usr/src/debug/llvm-3.8.1-1/lib/Support/CommandLine.cpp:81: multiple 
definition of `vtable for llvm::cl::OptionValueCopy<std::string>'
main.o:(.rdata[_ZTVN4llvm2cl15OptionValueCopyISsEE]+0x0): first defined here

/usr/lib/libLLVMSupport.a(CommandLine.cpp.o): In function 
`llvm::cl::parser<char>::anchor()':
/usr/src/debug/llvm-3.8.1-1/lib/Support/CommandLine.cpp:81: multiple 
definition of `vtable for llvm::cl::OptionValue<bool>'
main.o:(.rdata[_ZTVN4llvm2cl11OptionValueIbEE]+0x0): first defined here

/usr/lib/libLLVMSupport.a(CommandLine.cpp.o): In function 
`llvm::cl::parser<char>::anchor()':
/usr/src/debug/llvm-3.8.1-1/lib/Support/CommandLine.cpp:81: multiple 
definition of `vtable for llvm::cl::OptionValue<int>'
main.o:(.rdata[_ZTVN4llvm2cl11OptionValueIiEE]+0x0): first defined here

collect2: error: ld returned 1 exit status
clang-3.8: error: linker (via gcc) command failed with exit code 1 (use 
-v to see invocation)
make: *** [Makefile:43: main] Error 1