[llvm-dev] llvm bug 36466 fix
David Blaikie via llvm-dev
llvm-dev at lists.llvm.org
Sun Nov 4 08:11:35 PST 2018
On Sat, Nov 3, 2018 at 1:42 PM Muhui Jiang <jiangmuhui at gmail.com> wrote:
> Hi Dave
>
> What sort of bitcode analysis are you doing? (does this relate to the
> dwarfdump crash - or is this a separate/unrelated issue?)
> ====================================================
> In my llvm pass, I need to extract the intra-procedure cfg and print the
> source level debugging information for every IR. Then I can combine this
> information with the dwarf debugging information to get the binary level
> control flow graph. That is the reason why I need to do bitcode analysis.
> According to my experience, when I use the tool opt to load the llvm pass
> to analysis the bitcode, it took me rather a long time for large programs.
>
Perhaps your analysis is expensive (either high constant cost, or gets very
expensive on larger inputs)? Which is OK, but it's good to understand where
the cost is coming from.
> To conclude, this doesn't related to dwarfdump crash, but if I use another
> version of llvm to compile the source code, I have to repeat this operation
> on all the target binaries to keep the consistent of the experiments.
>
But llvm-dwarfdump can read any DWARF - it doesn't have to be LLVM's, nor
does it have to be LLVM's of the same version (between compiler and
dumper). So you could upgrade your llvm-dwarfdump & run that on the same
inputs and reuse the original IR intra-procedure CFG you generated?
> If you've got an official LLVM release, that should be fine - if you've
> built LLVM yourself, the default is a Debug build (unoptimized, with
> assertions enabled) which will be quite slow. You probably want to build a
> Release build, or Release with Assertions.
> ==========================================
> I build the LLVM by myself. I strictly follow the documentation here
> https://llvm.org/docs/GettingStarted.html
>
At this point in those instructions:
- -DCMAKE_BUILD_TYPE=type — Valid options for *type* are Debug, Release,
RelWithDebInfo, and MinSizeRel. Default is Debug.
Did you pick a non-default value? or did you stick with the default of
"Debug"? If you used the default, LLVM would be substantially slower than
if you used "Release". (though if all the slowness is in your own analysis,
it may not make much of a difference which mode you picked here)
I would like to clarify my worries. For example, I have the source code, I
> use llvm-7 and llvm-8 to compile it with the same compiling commands. Then
> I get two binaries. Any differences on these two binaries?
>
Yes, they may well be different binaries if you build them with different
versions of LLVM (different optimizations may trigger, etc). Assuming
you're optimizing during this compilation (that's more likely to produce
substantially different binaries)
> Especially on the control flow graph.
>
Yep - at least if you're looking at the CFG after LLVM's optimizations have
run - if you're looking at it before, the CFG might not change that much
between LLVM releases.
> Many Thanks
> I think this bug has been fixed in the newest version of llvm, right?
>
I'm not sure - I haven't checked, the bug you linked to didn't look like it
mentioned that anyone had worked on/fixed the issue, I think?
> I am afraid that if I use the llvm-8 to compile it and the dwarfdump crash
> will occur again.
>
Quite possible - though, again, you don't need to recompile the target
program to run the new version of llvm-dwarfdump on it. You can run the
LLVM 8 llvm-dwarfdump on a program produced by LLVM 7.
> Ah, no worries. Reducing the test case may not require that much
> familiarity with the source code - fairly mechanically just removing bits
> of code, compiling it, then running dwarfdump to see if it crashes. Any
> change you make that still succeeds at compiling and crashes dwarfdump is
> good, keep going, keep removing bits of code/simplifying things (notice
> that the program doesn't need to /run/ (either at all, or correctly) since
> you're only testing if dwarfdump crashes).
> ================================
> Got it. However, even if I could reduce the test case, I still cannot
> change it in my experiments since it is the standard benchmark.
>
Yeah, this wouldn't be a workaround for you - it'd be a way to help folks
who might be able to fix the bug, by providing them with more actionable
information/removing (for them) some of the costs involved in investigating
the bug.
> But it would be good for you guys to locate the original problem. I would
> be happy to do it after my deadline.
>
> Many Thanks
> Regards
> Muhui
>
>
> David Blaikie <dblaikie at gmail.com> 于2018年11月4日周日 上午4:28写道：
>
>>
>>
>> On Sat, Nov 3, 2018 at 1:19 PM Muhui Jiang <jiangmuhui at gmail.com> wrote:
>>
>>> Hi Dave
>>>
>>> Sorry, I meant the hardware you're using to compile LLVM - you mentioned
>>> it took you a long time to rebuild it so it would be hard for you to
>>> write/experiment on tests.
>>> =============================
>>> Compiling LLVM doesn't take me too much time(less than 2 hours). The
>>> hardware is good enough and I am using interl E5 CPU.
>>>
>>
>> Ah, OK
>>
>>
>>> What I mean is that it took me a long time to analysis the bitcode.
>>>
>>
>> What sort of bitcode analysis are you doing? (does this relate to the
>> dwarfdump crash - or is this a separate/unrelated issue?)
>>
>>
>>> If I recompile the binary, I need to re-analysis the bitcode. To make
>>> everything consistent, I need to re-compile and re-analysis the whole SPEC
>>> CPU. Are there any differences on the binaries if I use different kinds of
>>> version LLVM?
>>>
>>
>> If you've got an official LLVM release, that should be fine - if you've
>> built LLVM yourself, the default is a Debug build (unoptimized, with
>> assertions enabled) which will be quite slow. You probably want to build a
>> Release build, or Release with Assertions.
>>
>>
>>> If not, I may try to download and compile the newest version of llvm and
>>> compile the specific binaries.
>>>
>>> Could you try reducing this test further? Taking the code/compile
>>> commands and try to reduce the remove anything in the source code that
>>> isn't needed to cause llvm-dwarfdump to crash?
>>> =============================
>>> I may not be able to reduce it. CPU SPEC is the standard benchmark and I
>>> am not very familiar to the source code. The target binary is also rather
>>> large. Sorry for that.
>>>
>>
>> Ah, no worries. Reducing the test case may not require that much
>> familiarity with the source code - fairly mechanically just removing bits
>> of code, compiling it, then running dwarfdump to see if it crashes. Any
>> change you make that still succeeds at compiling and crashes dwarfdump is
>> good, keep going, keep removing bits of code/simplifying things (notice
>> that the program doesn't need to /run/ (either at all, or correctly) since
>> you're only testing if dwarfdump crashes).
>>
>> No worries though,
>> - Dave
>>
>>
>>>
>>> Regards
>>> Muhui
>>>
>>>
>>>
>>> David Blaikie <dblaikie at gmail.com> 于2018年11月4日周日 上午4:02写道：
>>>
>>>>
>>>>
>>>> On Sat, Nov 3, 2018 at 12:37 PM Muhui Jiang <jiangmuhui at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Dave
>>>>>
>>>>> I am not going to access any hardware.
>>>>>
>>>>
>>>> Sorry, I meant the hardware you're using to compile LLVM - you
>>>> mentioned it took you a long time to rebuild it so it would be hard for you
>>>> to write/experiment on tests.
>>>>
>>>>
>>>>> I am using clang to analysis the ARM binaries. The binary is
>>>>> 483.xalancbmk in CPU SPEC2006. When I use the optimization O0, no crash
>>>>> will occur. The crash occurs when I set optimization level as O1,O2,O3 and
>>>>> Os.
>>>>>
>>>>
>>>> Could you try reducing this test further? Taking the code/compile
>>>> commands and try to reduce the remove anything in the source code that
>>>> isn't needed to cause llvm-dwarfdump to crash?
>>>>
>>>>
>>>>>
>>>>> If I have to recompile and rerun the tests. What version of llvm is
>>>>> suggested. It would be better if anyone could provide the patch on this
>>>>> bug.
>>>>>
>>>>> Regards
>>>>> Muhui
>>>>>
>>>>> David Blaikie <dblaikie at gmail.com> 于2018年11月4日周日 上午2:58写道：
>>>>>
>>>>>>
>>>>>>
>>>>>> On Sat, Nov 3, 2018 at 6:51 AM Muhui Jiang via llvm-dev <
>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> I come across the following exception when I use the llvm-dwarfdump
>>>>>>> -debug-info target_binary:
>>>>>>>
>>>>>>> llvm-dwarfdump:
>>>>>>> /home/linux/llvm-7/llvm/lib/MC/MCRegisterInfo.cpp:87: int
>>>>>>> llvm::MCRegisterInfo::getLLVMRegNum(unsigned int, bool) const: Assertion `I
>>>>>>> != M+Size && I->FromReg == RegNum && "Invalid RegNum"' failed.
>>>>>>> Stack dump:
>>>>>>> 0. Program arguments:
>>>>>>> /home/linux/llvm-7/llvm/build/bin/llvm-dwarfdump -debug-info
>>>>>>> C++/483.xalancbmk.O1
>>>>>>> #0 0x000000000093ab37 llvm::sys::PrintStackTrace(llvm::raw_ostream&)
>>>>>>> /home/linux/llvm-7/llvm/lib/Support/Unix/Signals.inc:490:0
>>>>>>> #1 0x000000000093abca PrintStackTraceSignalHandler(void*)
>>>>>>> /home/linux/llvm-7/llvm/lib/Support/Unix/Signals.inc:554:0
>>>>>>> #2 0x00000000009388d8 llvm::sys::RunSignalHandlers()
>>>>>>> /home/linux/llvm-7/llvm/lib/Support/Signals.cpp:67:0
>>>>>>> #3 0x000000000093a531 SignalHandler(int)
>>>>>>> /home/linux/llvm-7/llvm/lib/Support/Unix/Signals.inc:353:0
>>>>>>> #4 0x00007f39f47d7390 __restore_rt
>>>>>>> (/lib/x86_64-linux-gnu/libpthread.so.0+0x11390)
>>>>>>> #5 0x00007f39f374d428 gsignal
>>>>>>> /build/glibc-Cl5G7W/glibc-2.23/signal/../sysdeps/unix/sysv/linux/raise.c:54:0
>>>>>>> #6 0x00007f39f374f02a abort
>>>>>>> /build/glibc-Cl5G7W/glibc-2.23/stdlib/abort.c:91:0
>>>>>>> #7 0x00007f39f3745bd7 __assert_fail_base
>>>>>>> /build/glibc-Cl5G7W/glibc-2.23/assert/assert.c:92:0
>>>>>>> #8 0x00007f39f3745c82 (/lib/x86_64-linux-gnu/libc.so.6+0x2dc82)
>>>>>>> #9 0x00000000006e6ba1 llvm::MCRegisterInfo::getLLVMRegNum(unsigned
>>>>>>> int, bool) const /home/linux/llvm-7/llvm/lib/MC/MCRegisterInfo.cpp:88:0
>>>>>>> #10 0x00000000004bd03e
>>>>>>> llvm::prettyPrintRegisterOp(llvm::raw_ostream&, unsigned char, unsigned
>>>>>>> long*, llvm::MCRegisterInfo const*, bool)
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFExpression.cpp:206:0
>>>>>>> #11 0x00000000004bd226
>>>>>>> llvm::DWARFExpression::Operation::print(llvm::raw_ostream&,
>>>>>>> llvm::DWARFExpression const*, llvm::MCRegisterInfo const*, bool)
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFExpression.cpp:237:0
>>>>>>> #12 0x00000000004bd466
>>>>>>> llvm::DWARFExpression::print(llvm::raw_ostream&, llvm::MCRegisterInfo
>>>>>>> const*, bool) const
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFExpression.cpp:264:0
>>>>>>> #13 0x00000000004a8c68 dumpExpression(llvm::raw_ostream&,
>>>>>>> llvm::ArrayRef<char>, bool, unsigned int, llvm::MCRegisterInfo const*)
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFDebugLoc.cpp:38:0
>>>>>>> #14 0x00000000004a8ea5
>>>>>>> llvm::DWARFDebugLoc::LocationList::dump(llvm::raw_ostream&, bool, unsigned
>>>>>>> int, llvm::MCRegisterInfo const*, unsigned long, unsigned int) const
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFDebugLoc.cpp:45:0
>>>>>>> #15 0x00000000004b331d dumpLocation(llvm::raw_ostream&,
>>>>>>> llvm::DWARFFormValue&, llvm::DWARFUnit*, unsigned int, llvm::DIDumpOptions)
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFDie.cpp:115:0
>>>>>>> #16 0x00000000004b4034 dumpAttribute(llvm::raw_ostream&,
>>>>>>> llvm::DWARFDie const&, unsigned int*, llvm::dwarf::Attribute,
>>>>>>> llvm::dwarf::Form, unsigned int, llvm::DIDumpOptions)
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFDie.cpp:243:0
>>>>>>> #17 0x00000000004b5cd2 llvm::DWARFDie::dump(llvm::raw_ostream&,
>>>>>>> unsigned int, llvm::DIDumpOptions) const
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFDie.cpp:521:0
>>>>>>> #18 0x00000000004b5db4 llvm::DWARFDie::dump(llvm::raw_ostream&,
>>>>>>> unsigned int, llvm::DIDumpOptions) const
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFDie.cpp:530:0
>>>>>>> #19 0x00000000004b5db4 llvm::DWARFDie::dump(llvm::raw_ostream&,
>>>>>>> unsigned int, llvm::DIDumpOptions) const
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFDie.cpp:530:0
>>>>>>> #20 0x00000000004b5db4 llvm::DWARFDie::dump(llvm::raw_ostream&,
>>>>>>> unsigned int, llvm::DIDumpOptions) const
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFDie.cpp:530:0
>>>>>>> #21 0x0000000000445cdd
>>>>>>> llvm::DWARFCompileUnit::dump(llvm::raw_ostream&, llvm::DIDumpOptions)
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFCompileUnit.cpp:31:0
>>>>>>> #22 0x00000000004482ad llvm::DWARFContext::dump(llvm::raw_ostream&,
>>>>>>> llvm::DIDumpOptions, std::array<llvm::Optional<unsigned long>,
>>>>>>> 27ul>)::'lambda0'(bool, char const*, llvm::DWARFSection,
>>>>>>> llvm::iterator_range<std::unique_ptr<llvm::DWARFCompileUnit,
>>>>>>> std::default_delete<llvm::DWARFCompileUnit> >*>)::operator()(bool, char
>>>>>>> const*, llvm::DWARFSection,
>>>>>>> llvm::iterator_range<std::unique_ptr<llvm::DWARFCompileUnit,
>>>>>>> std::default_delete<llvm::DWARFCompileUnit> >*>) const
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFContext.cpp:351:0
>>>>>>> #23 0x0000000000448e28 llvm::DWARFContext::dump(llvm::raw_ostream&,
>>>>>>> llvm::DIDumpOptions, std::array<llvm::Optional<unsigned long>, 27ul>)
>>>>>>> /home/linux/llvm-7/llvm/lib/DebugInfo/DWARF/DWARFContext.cpp:355:0
>>>>>>> #24 0x0000000000414af6 dumpObjectFile(llvm::object::ObjectFile&,
>>>>>>> llvm::DWARFContext&, llvm::Twine, llvm::raw_ostream&)
>>>>>>> /home/linux/llvm-7/llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp:424:0
>>>>>>> #25 0x0000000000425d9a std::_Function_handler<bool
>>>>>>> (llvm::object::ObjectFile&, llvm::DWARFContext&, llvm::Twine,
>>>>>>> llvm::raw_ostream&), bool (*)(llvm::object::ObjectFile&,
>>>>>>> llvm::DWARFContext&, llvm::Twine,
>>>>>>> llvm::raw_ostream&)>::_M_invoke(std::_Any_data const&,
>>>>>>> llvm::object::ObjectFile&, llvm::DWARFContext&, llvm::Twine&&,
>>>>>>> llvm::raw_ostream&) /usr/include/c++/5/functional:1857:0
>>>>>>> #26 0x0000000000420f69 std::function<bool
>>>>>>> (llvm::object::ObjectFile&, llvm::DWARFContext&, llvm::Twine,
>>>>>>> llvm::raw_ostream&)>::operator()(llvm::object::ObjectFile&,
>>>>>>> llvm::DWARFContext&, llvm::Twine, llvm::raw_ostream&) const
>>>>>>> /usr/include/c++/5/functional:2268:0
>>>>>>> #27 0x0000000000415377 handleBuffer(llvm::StringRef,
>>>>>>> llvm::MemoryBufferRef, std::function<bool (llvm::object::ObjectFile&,
>>>>>>> llvm::DWARFContext&, llvm::Twine, llvm::raw_ostream&)>, llvm::raw_ostream&)
>>>>>>> /home/linux/llvm-7/llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp:472:0
>>>>>>> #28 0x0000000000415ae2 handleFile(llvm::StringRef,
>>>>>>> std::function<bool (llvm::object::ObjectFile&, llvm::DWARFContext&,
>>>>>>> llvm::Twine, llvm::raw_ostream&)>, llvm::raw_ostream&)
>>>>>>> /home/linux/llvm-7/llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp:506:0
>>>>>>> #29 0x00000000004170a4 main
>>>>>>> /home/linux/llvm-7/llvm/tools/llvm-dwarfdump/llvm-dwarfdump.cpp:623:0
>>>>>>> #30 0x00007f39f3738830 __libc_start_main
>>>>>>> /build/glibc-Cl5G7W/glibc-2.23/csu/../csu/libc-start.c:325:0
>>>>>>> #31 0x00000000004092e9 _start
>>>>>>> (/home/linux/llvm-7/llvm/build/bin/llvm-dwarfdump+0x4092e9)
>>>>>>> Aborted (core dumped)
>>>>>>>
>>>>>>>
>>>>>>> After the google search. It seems that it should be an bug here
>>>>>>> https://www.mail-archive.com/llvm-bugs@lists.llvm.org/msg21648.html
>>>>>>>
>>>>>>> I am using llvm 7.0 and cross compiled the binary with linaro arm
>>>>>>> toolchain(7.3.1). It takes me quiet a long time to compile and analysis the
>>>>>>> bitcode. Are there any solutions on fixing this without re-compiling the
>>>>>>> binary. Many Thanks
>>>>>>>
>>>>>>
>>>>>> Not much you can do to fix the bug if you can't readily recompile and
>>>>>> rerun the tests - what sort of hardware do you have access to? (you might
>>>>>> be able to reproduce this without ARM if your ARM machine has few cores/not
>>>>>> much RAM but you have something else (X86 desktop machine) with more of
>>>>>> both)
>>>>>>
>>>>>> If you can't do that but would like to help move the bug forward
>>>>>> anyway, a useful thing to do might be to try to find a reduced/small test
>>>>>> case that demonstrates the issue to make it easier for someone else to
>>>>>> investigate and fix the issue.
>>>>>>
>>>>>> - Dave
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Regards
>>>>>>> Muhui
>>>>>>> _______________________________________________
>>>>>>> LLVM Developers mailing list
>>>>>>> llvm-dev at lists.llvm.org
>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>
>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181104/1eaabea3/attachment.html>
More information about the llvm-dev
mailing list