[llvm] r211272 - Emit DWARF3 call frame information when DWARF3+ debug info is requested
Dimitry Andric
dimitry at andric.com
Sun Dec 28 11:48:39 PST 2014
Sure, I filed http://llvm.org/PR22046 to track this.
-Dimitry
> On 24 Dec 2014, at 21:51, Oliver Stannard <ostannard at gmail.com> wrote:
>
> I won't be at my development machine for a few weeks, so it's probably better to file a PR.
>
> Oliver
>
> On 23 Dec 2014 20:17, "Dimitry Andric" <dimitry at andric.com> wrote:
> I think I have now found the root cause for this issue, and why it is
> introduced by r211272.
>
> GCC has a number of different ways of doing its 3-stage bootstrapping,
> as described here: https://gcc.gnu.org/install/build.html . In
> particular, it has a --with-build-config=bootstrap-debug setting for its
> configure script, which is described as follows:
>
> "Verifies that the compiler generates the same executable code, whether
> or not it is asked to emit debug information. To this end, this option
> builds stage2 host programs without debug information, and uses
> contrib/compare-debug to compare them with the stripped stage3 object
> files. [...]"
>
> However, if you do not specify any --with-build-config option, but do
> specify --enable-bootstrap, the configure script tries to auto-detect
> whether objects generated with and without -g are identical, after
> stripping:
>
> 7177 case $BUILD_CONFIG in
> 7178 bootstrap-debug)
> 7179 if echo "int f (void) { return 0; }" > conftest.c &&
> 7180 ${CC} -c conftest.c &&
> 7181 mv conftest.o conftest.o.g0 &&
> 7182 ${CC} -c -g conftest.c &&
> 7183 mv conftest.o conftest.o.g &&
> 7184 ${srcdir}/contrib/compare-debug conftest.o.g0 conftest.o.g > /dev/null 2>&1; then
> 7185 :
> 7186 else
> 7187 BUILD_CONFIG=
> 7188 fi
> 7189 rm -f conftest.c conftest.o conftest.o.g0 conftest.o.g
> 7190 ;;
> 7191 esac
>
> So it compiles a very minimal program with and without the -g flag, and
> runs its "compare-debug" script on it. The compare-debug script is
> rather convoluted, but basically it returns success or failure,
> depending on whether stripped versions of the object files to be
> compared are identical.
>
> Note also, that it uses the "host compiler" (e.g. clang) for this test,
> since this is still at the point where there nothing is yet built, not
> even a stage 1 gcc.
>
> In case of clang before r211272, objects compiled with or without -g
> are identical after stripping:
>
> $ cat build-config-test.c
> int f (void) { return 0; }
> $ ~/obj/llvm-r211271/bin/clang -c build-config-test.c -o test-r211271-without-g.o
> $ ~/obj/llvm-r211271/bin/clang -c -g build-config-test.c -o test-r211271-with-g.o
> $ strip test-r211271-without-g.o
> $ strip test-r211271-with-g.o
> $ md5 test-r211271-without-g.o test-r211271-with-g.o
> MD5 (test-r211271-without-g.o) = e8d1caf0e37db51c305939993f8c1e36
> MD5 (test-r211271-with-g.o) = e8d1caf0e37db51c305939993f8c1e36
>
> But clang r211272 changes this behavior:
>
> $ ~/obj/llvm-r211272/bin/clang -c build-config-test.c -o test-r211272-without-g.o
> $ ~/obj/llvm-r211272/bin/clang -c -g build-config-test.c -o test-r211272-with-g.o
> $ strip test-r211272-without-g.o
> $ strip test-r211272-with-g.o
> $ md5 test-r211272-without-g.o test-r211272-with-g.o
> MD5 (test-r211272-without-g.o) = 1d86d9d6073cfcb6f1228dcca27152cc
> MD5 (test-r211272-with-g.o) = 640e2850cf02dfe84340619e3757a49c
>
> So what is the exact difference between those last two stripped object
> files? It is exactly the CIE version emitted in the .eh_frame section:
>
> $ dwarfdump -v -F test-r211272-without-g.o
>
> .eh_frame
>
> fde:
> < 0><0x00000020:0x0000002b><><fde offset 0x00000018 length: 0x0000001c><eh aug data len 0x0>
> 0x00000020: <off cfa=08(r7) > <off r16=-8(cfa) >
> 0x00000021: <off cfa=16(r7) > <off r6=-16(cfa) > <off r16=-8(cfa) >
> 0x00000024: <off cfa=16(r6) > <off r6=-16(cfa) > <off r16=-8(cfa) >
>
> cie:
> < 0> version 3
> cie section offset 0 0x00000000
> augmentation zR
> code_alignment_factor 1
> data_alignment_factor -8
> return_address_register 16
> eh aug data len 0x1 bytes 0x1b
> bytes of initial instructions 7
> cie length 20
> initial instructions
> 0 DW_CFA_def_cfa r7 8
> 3 DW_CFA_offset r16 -8 (1 * -8)
> 5 DW_CFA_nop
> 6 DW_CFA_nop
>
> $ dwarfdump -v -F test-r211272-with-g.o
>
> .eh_frame
>
> fde:
> < 0><0x00000020:0x0000002b><><fde offset 0x00000018 length: 0x0000001c><eh aug data len 0x0>
> 0x00000020: <off cfa=08(r7) > <off r16=-8(cfa) >
> 0x00000021: <off cfa=16(r7) > <off r6=-16(cfa) > <off r16=-8(cfa) >
> 0x00000024: <off cfa=16(r6) > <off r6=-16(cfa) > <off r16=-8(cfa) >
>
> cie:
> < 0> version 1
> cie section offset 0 0x00000000
> augmentation zR
> code_alignment_factor 1
> data_alignment_factor -8
> return_address_register 16
> eh aug data len 0x1 bytes 0x1b
> bytes of initial instructions 7
> cie length 20
> initial instructions
> 0 DW_CFA_def_cfa r7 8
> 3 DW_CFA_offset r16 -8 (1 * -8)
> 5 DW_CFA_nop
> 6 DW_CFA_nop
>
> r211272 changes this, at least for platforms where the default DWARF
> version is 2 (*BSD, Darwin, Solaris), because it now emits CIE version 3
> when -g is not specified on the command line. This is because
> DwarfDebug::DwarfDebug() retrieves the DWARF version from the module,
> and Module::getDwarfVersion() returns 2 when -g is used, but 4 when -g
> is not used.
>
> It looks like CodeGenModule::Release() only sets the module's "Dwarf
> Version" flag, whenever a particular -gdwarf-X option is passed to the
> CompilerInstance, though. Maybe Module::getDwarfVersion() should not
> return dwarf::DWARF_VERSION unconditionally as 4, for platforms that
> default to another version. Or maybe the "Dwarf Version" module flag
> should always be set, I don't know what the best solution is.
>
> Having said all that, is this something to put in a PR, so it can be
> tracked better? Or is the fix obvious now?
>
> -Dimitry
>
> > On 09 Dec 2014, at 20:59, Dimitry Andric <dimitry at andric.com> wrote:
> >
> > No, this is with the binutils 2.24 port installed, which is normally a
> > prerequisite for any of the gcc ports. So intermediate copies of gcc
> > will run a separately compiled instance of /usr/local/bin/as for its
> > assembler work. During the whole build process, I don't see it
> > compiling any .S files explicitly with clang either.
> >
> > One other possibility is that each following stage links in some common
> > code (e.g. libiberty) from the first stage, so "bad" stuff in there
> > would end up in every stage. But I would find that strange, and rather
> > expect gcc to distrust the host compiler and thus recompile everything?
> >
> > Unfortunately I'm not yet familiar enough with the inner workings of
> > gcc's bootstrapping process, so I will have to look into it a bit more.
> >
> > -Dimitry
> >
> >> On 09 Dec 2014, at 14:46, Oliver Stannard <oliver.stannard at arm.com> wrote:
> >>
> >> Do you know if the clang assembler is being used for the stage2 and stage3
> >> builds? According to https://gcc.gnu.org/install/build.html#TOC0, gas will
> >> only be rebuilt at each stage if you have the binutils source alongside the
> >> gcc source, and I assume it would use the clang assembler otherwise.
> >>
> >> The only difference I can see in the object files is that the debug info is
> >> sorted very slightly differently.
> >>
> >> Oliver
> >>
> >>> -----Original Message-----
> >>> From: Dimitry Andric [mailto:dimitry at andric.com]
> >>> Sent: 08 December 2014 21:46
> >>> To: Oliver Stannard
> >>> Cc: Baptiste Daroussin; Gerald Pfeifer; David Chisnall; llvm-commits
> >>> Subject: Re: [llvm] r211272 - Emit DWARF3 call frame information when
> >>> DWARF3+ debug info is requested
> >>>
> >>> And indeed, with just this diff, the stage comparison also succeeds:
> >>>
> >>> Index: contrib/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
> >>> ===================================================================
> >>> --- contrib/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp (revision
> >>> 275623)
> >>> +++ contrib/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp (working copy)
> >>> @@ -210,7 +210,7 @@
> >>> DwarfVersion = DwarfVersionNumber ? DwarfVersionNumber
> >>> : MMI->getModule()-
> >>>> getDwarfVersion();
> >>>
> >>> - Asm->OutStreamer.getContext().setDwarfVersion(DwarfVersion);
> >>> + //Asm->OutStreamer.getContext().setDwarfVersion(DwarfVersion);
> >>>
> >>> {
> >>> NamedRegionTimer T(DbgTimerName, DWARFGroupName,
> >>> TimePassesIsEnabled);
> >>>
> >>>
> >>> Now, as to why this changes behavior so much, I have no idea... :)
> >>>
> >>> -Dimitry
> >>>
> >>>> On 08 Dec 2014, at 21:56, Dimitry Andric <dimitry at andric.com> wrote:
> >>>>
> >>>> So, with just the first part of the commit reverted, the gcc build
> >>>> completes OK, with no stage comparison failures. This comes down to
> >>> the
> >>>> following diff on 3.5.0 sources:
> >>>>
> >>>> --- a/include/llvm/Support/Dwarf.h
> >>>> +++ b/include/llvm/Support/Dwarf.h
> >>>> @@ -57,6 +57,7 @@ enum LLVMConstants : uint32_t {
> >>>> DW_TAG_user_base = 0x1000, // Recommended base for user tags.
> >>>>
> >>>> DWARF_VERSION = 4, // Default dwarf version we output.
> >>>> + DW_CIE_VERSION = 1, // Common frame information version.
> >>>> DW_PUBTYPES_VERSION = 2, // Section version number for
> >>> .debug_pubtypes.
> >>>> DW_PUBNAMES_VERSION = 2, // Section version number for
> >>> .debug_pubnames.
> >>>> DW_ARANGES_VERSION = 2 // Section version number for
> >>> .debug_aranges.
> >>>> --- a/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
> >>>> +++ b/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
> >>>> @@ -210,8 +210,6 @@ DwarfDebug::DwarfDebug(AsmPrinter *A, Module *M)
> >>>> DwarfVersion = DwarfVersionNumber ? DwarfVersionNumber
> >>>> : MMI->getModule()-
> >>>> getDwarfVersion();
> >>>>
> >>>> - Asm->OutStreamer.getContext().setDwarfVersion(DwarfVersion);
> >>>> -
> >>>> {
> >>>> NamedRegionTimer T(DbgTimerName, DWARFGroupName,
> >>> TimePassesIsEnabled);
> >>>> beginModule();
> >>>> --- a/lib/MC/MCDwarf.cpp
> >>>> +++ b/lib/MC/MCDwarf.cpp
> >>>> @@ -1358,10 +1358,7 @@ const MCSymbol
> >>> &FrameEmitterImpl::EmitCIE(MCObject
> >>>>
> >>>> // Version
> >>>> if (verboseAsm) streamer.AddComment("DW_CIE_VERSION");
> >>>> - // For DWARF2, we use CIE version 1
> >>>> - // For DWARF3+, we use CIE version 3
> >>>> - uint8_t CIEVersion = context.getDwarfVersion() <= 2 ? 1 : 3;
> >>>> - streamer.EmitIntValue(CIEVersion, 1);
> >>>> + streamer.EmitIntValue(dwarf::DW_CIE_VERSION, 1);
> >>>>
> >>>> // Augmentation String
> >>>> SmallString<8> Augmentation;
> >>>> @@ -1389,7 +1386,7 @@ const MCSymbol
> >>> &FrameEmitterImpl::EmitCIE(MCObject
> >>>>
> >>>> // Return Address Register
> >>>> if (verboseAsm) streamer.AddComment("CIE Return Address Column");
> >>>> - if (CIEVersion == 1) {
> >>>> + if (context.getDwarfVersion() <= 2) {
> >>>> assert(MRI->getRARegister() <= 255 &&
> >>>> "DWARF 2 encodes return_address_register in one byte");
> >>>> streamer.EmitIntValue(MRI->getDwarfRegNum(MRI->getRARegister(),
> >>> true), 1);
> >>>>
> >>>> I suspect the crucial part is the deletion of the setDwarfVersion()
> >>> call
> >>>> in DwarfDebug::DwarfDebug().
> >>>>
> >>>> Btw, please note that FreeBSD still defaults to emitting DWARF2. (We
> >>>> initially went with the new default of DWARF4, but after much
> >>> grumbling
> >>>> that was reverted.)
> >>>>
> >>>> -Dimitry
> >>>>
> >>>>> On 08 Dec 2014, at 21:04, Dimitry Andric <dimitry at andric.com> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> I just did another gcc build, first reverting the second hunk in
> >>>>> MCDwarf.cpp. This still leads to the same stage comparison
> >>> failures.
> >>>>>
> >>>>> I will now try reverting just the first part of the commit.
> >>>>>
> >>>>> -Dimitry
> >>>>>
> >>>>>> On 08 Dec 2014, at 17:59, Oliver Stannard <oliver.stannard at arm.com>
> >>> wrote:
> >>>>>>
> >>>>>> The commit in question should really have been two separate
> >>> commits: setting
> >>>>>> the CIE version correctly, and using a ULEB128 for the return
> >>> address
> >>>>>> register in DWARF3+. It may be helpful if you could bisect this a
> >>> bit
> >>>>>> further (reverting the second hunk in MCDwarf.cpp should do it).
> >>>>>>
> >>>>>> Your readelf output shows that the relocations apply to different
> >>> places in
> >>>>>> the .debug_info sections, is there any difference in the
> >>> .debug_info
> >>>>>> sections themselves?
> >>>>>>
> >>>>>> Oliver
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Oliver Stannard [mailto:oliver.stannard at arm.com]
> >>>>>>> Sent: 08 December 2014 15:47
> >>>>>>> To: 'Dimitry Andric'
> >>>>>>> Cc: Baptiste Daroussin; Ed Maste; Roman Divacky; Gerald Pfeifer;
> >>> David
> >>>>>>> Chisnall; llvm-commits
> >>>>>>> Subject: RE: [llvm] r211272 - Emit DWARF3 call frame information
> >>> when
> >>>>>>> DWARF3+ debug info is requested
> >>>>>>>
> >>>>>>> +llvm-commits
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I'm not very familiar with GCC's build system, but if I am
> >>>>>>> understanding this correctly the comparison failure is between the
> >>>>>>> debug info in the state-2 and stage-3 GCC objects. These were
> >>> compiled
> >>>>>>> with the stage-1 and stage-2 GCCs, so I can't see how the debug
> >>> info
> >>>>>>> emitted by clang (which would be present in the stage-1 GCC, but
> >>> should
> >>>>>>> not affect it's execution) could be involved in this comparison.
> >>> Have I
> >>>>>>> understood this correctly?
> >>>>>>>
> >>>>>>> Oliver
> >>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Dimitry Andric [mailto:dim at freebsd.org]
> >>>>>>>> Sent: 07 December 2014 20:21
> >>>>>>>> To: Oliver Stannard
> >>>>>>>> Cc: Baptiste Daroussin; Ed Maste; Roman Divacky; Gerald Pfeifer;
> >>>>>>> David
> >>>>>>>> Chisnall
> >>>>>>>> Subject: Re: [llvm] r211272 - Emit DWARF3 call frame information
> >>> when
> >>>>>>>> DWARF3+ debug info is requested
> >>>>>>>>
> >>>>>>>> On 19 Jun 2014, at 17:39, Oliver Stannard
> >>> <oliver.stannard at arm.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Author: olista01
> >>>>>>>>> Date: Thu Jun 19 10:39:33 2014
> >>>>>>>>> New Revision: 211272
> >>>>>>>>>
> >>>>>>>>> URL: http://llvm.org/viewvc/llvm-project?rev=211272&view=rev
> >>>>>>>>> Log:
> >>>>>>>>> Emit DWARF3 call frame information when DWARF3+ debug info is
> >>>>>>>> requested
> >>>>>>>>>
> >>>>>>>>> Currently, llvm always emits a DWARF CIE with a version of 1,
> >>> even
> >>>>>>>> when emitting
> >>>>>>>>> DWARF 3 or 4, which both support CIE version 3. This patch makes
> >>> it
> >>>>>>>> emit the
> >>>>>>>>> newer CIE version when we are emitting DWARF 3 or 4. This will
> >>> not
> >>>>>>>> reduce
> >>>>>>>>> compatibility, as we already emit other DWARF3/4 features, and
> >>> is
> >>>>>>>> worth doing as
> >>>>>>>>> the DWARF3 spec removed some ambiguities in the interpretation
> >>> of
> >>>>>>>> call frame
> >>>>>>>>> information.
> >>>>>>>>>
> >>>>>>>>> It also fixes a minor bug where the "return address" field of
> >>> the
> >>>>>>> CIE
> >>>>>>>> was
> >>>>>>>>> encoded as a ULEB128, which is only valid when the CIE version
> >>> is
> >>>>>>> 3.
> >>>>>>>> There are
> >>>>>>>>> no test changes for this, because (as far as I can tell) none of
> >>>>>>> the
> >>>>>>>> platforms
> >>>>>>>>> that we test have a return address register with a DWARF
> >>> register
> >>>>>>>> number >127.
> >>>>>>>>
> >>>>>>>> Hi Oliver,
> >>>>>>>>
> >>>>>>>> After a lot of experimentation, I think I found that this
> >>> particular
> >>>>>>>> commit causes some trouble when building relatively new versions
> >>> of
> >>>>>>> gcc
> >>>>>>>> with clang.
> >>>>>>>>
> >>>>>>>> I'm currently working on importing clang 3.5.0 into FreeBSD 11.0-
> >>>>>>>> CURRENT. While doing several tests, I noticed that some of the
> >>> gcc
> >>>>>>>> ports,
> >>>>>>>> lang/gcc48 and lang/gcc49, failed to build due to a bootstrap
> >>>>>>>> comparison failure. For example, lang/gcc48 failed with:
> >>>>>>>>
> >>>>>>>> [...]
> >>>>>>>> gmake[5]: Leaving directory
> >>>>>>>> '/usr/work/share/dim/ports/lang/gcc48/work/build'
> >>>>>>>> Comparing stages 2 and 3
> >>>>>>>> warning: gcc/cc1-checksum.o differs
> >>>>>>>> warning: gcc/cc1obj-checksum.o differs
> >>>>>>>> warning: gcc/cc1plus-checksum.o differs
> >>>>>>>> Bootstrap comparison failure!
> >>>>>>>> gcc/cfg.o differs
> >>>>>>>> gcc/coverage.o differs
> >>>>>>>> gcc/asan.o differs
> >>>>>>>> gcc/tree-ssa-threadupdate.o differs
> >>>>>>>> gcc/valtrack.o differs
> >>>>>>>> Makefile:18939: recipe for target 'compare' failed
> >>>>>>>>
> >>>>>>>> If I look at the differing object files, I can see some
> >>> differences
> >>>>>>> in
> >>>>>>>> the .rela.debug_info sections. For example, if I use "readelf -
> >>> a" to
> >>>>>>>> dump gcc/cfg.o, and compare the stage 2 and stage 3 versions, the
> >>>>>>> diff
> >>>>>>>> boils down to:
> >>>>>>>>
> >>>>>>>> Relocation section '.rela.debug_info' at offset 0x34ca0 contains
> >>> 5228
> >>>>>>>> entries:
> >>>>>>>> Offset Info Type Symbol's
> >>>>>>>> Value Symbol's Name + Addend
> >>>>>>>> [...]
> >>>>>>>> 4382,4391c4382,4391
> >>>>>>>> < 000000000000a4f0 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + bfad
> >>>>>>>> < 000000000000a4fc 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 59ab
> >>>>>>>> < 000000000000a502 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + f9c1
> >>>>>>>> < 000000000000a515 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 85bf
> >>>>>>>> < 000000000000a51b 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 1096
> >>>>>>>> < 000000000000a52e 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + b30
> >>>>>>>> < 000000000000a534 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 5ab4
> >>>>>>>> < 000000000000a543 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 417e
> >>>>>>>> < 000000000000a549 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 2217
> >>>>>>>> < 000000000000a558 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + ce88
> >>>>>>>> ---
> >>>>>>>>> 000000000000a4ea 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + bfad
> >>>>>>>>> 000000000000a4f6 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 59ab
> >>>>>>>>> 000000000000a4fc 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + f9c1
> >>>>>>>>> 000000000000a50f 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 85bf
> >>>>>>>>> 000000000000a515 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 1096
> >>>>>>>>> 000000000000a528 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + b30
> >>>>>>>>> 000000000000a52e 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 5ab4
> >>>>>>>>> 000000000000a53d 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 417e
> >>>>>>>>> 000000000000a543 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + 2217
> >>>>>>>>> 000000000000a552 0000003c0000000a R_X86_64_32
> >>>>>>>> 0000000000000000 .debug_str + ce88
> >>>>>>>>
> >>>>>>>> So for just a few of the debug strings, the offsets are slightly
> >>>>>>>> different. Something similar also happens for the other .o
> >>> files.
> >>>>>>>>
> >>>>>>>> To find the cause of this problem, which does not occur with
> >>> clang
> >>>>>>>> 3.4.1, I bisected llvm/clang trunk between the branch points for
> >>> 3.4
> >>>>>>>> and 3.5, and I arrived at your commit r211272, which appears to
> >>> cause
> >>>>>>>> it.
> >>>>>>>>
> >>>>>>>> Do you have any idea if there might be a bug in this commit? I
> >>> know
> >>>>>>> it
> >>>>>>>> is probably very hard to say, but I'm rather at a loss for the
> >>> real
> >>>>>>>> explanation...
> >>>>>>>>
> >>>>>>>> -Dimitry
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> llvm-commits mailing list
> >>>>>> llvm-commits at cs.uiuc.edu
> >>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
> >>>>>
> >>>>
> >>
> >>
> >>
> >>
> >
> > _______________________________________________
> > llvm-commits mailing list
> > llvm-commits at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 194 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20141228/53dd67b1/attachment.sig>
More information about the llvm-commits
mailing list