[llvm] r204958 - DebugInfo: Support for compressed debug info sections
David Blaikie
dblaikie at gmail.com
Fri Mar 28 10:12:43 PDT 2014
On Fri, Mar 28, 2014 at 9:52 AM, Matheus Almeida
<Matheus.Almeida at imgtec.com> wrote:
> The segmentation fault was probably a consequence of not having zlib installed on the machine I was running the test.
> Now that I'm using a build with assertions and debug info I'm getting the same problem reported by Evgeniy Stepanov.
> assert(Success == zlib::StatusOK);
>
> Given that bad things can happen if zlib is not available, shouldn't the code check if it is enabled ? Support/Compression.cpp has some examples.
We should check, certainly - but not down at the point where we're
doing the compression. The check will need to be/should be up in the
frontends that accept the option to enable compression.
I've disabled the test when zlib isn't available in r205016. Let me
know if that clears up the issue for everyone.
I'll work on error handling in the clients (llvm-mc and clang).
- David
>
> Regards,
> Matheus
>
> -----Original Message-----
> From: llvm-commits-bounces at cs.uiuc.edu [mailto:llvm-commits-bounces at cs.uiuc.edu] On Behalf Of Matheus Almeida
> Sent: 28 March 2014 14:16
> To: David Blaikie
> Cc: llvm-commits
> Subject: RE: [llvm] r204958 - DebugInfo: Support for compressed debug info sections
>
> Hi,
>
> I'm getting a segmentation fault on my local build running the test added by this patch. I can't give you much more information because that build doesn't have debug info but I could get a clean call stack which I've attached to this e-mail.
> In the meantime I'll start another build with debug info and I'll try to replicate this issue.
>
> Regards,
> Matheus
>
> -----Original Message-----
> From: llvm-commits-bounces at cs.uiuc.edu [mailto:llvm-commits-bounces at cs.uiuc.edu] On Behalf Of Evgeniy Stepanov
> Sent: 28 March 2014 08:01
> To: David Blaikie
> Cc: llvm-commits
> Subject: Re: [llvm] r204958 - DebugInfo: Support for compressed debug info sections
>
> With this change we get an assertion failure on the bots:
>
> lib/MC/MCAssembler.cpp:245: const SmallVectorImpl<char>
> &llvm::MCCompressedFragment::getCompressedContents() const: Assertion `Success == zlib::StatusOK' failed.
>
> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/2813/steps/check-llvm%20msan/logs/stdio
>
>
> On Fri, Mar 28, 2014 at 12:45 AM, David Blaikie <dblaikie at gmail.com> wrote:
>> Author: dblaikie
>> Date: Thu Mar 27 15:45:58 2014
>> New Revision: 204958
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=204958&view=rev
>> Log:
>> DebugInfo: Support for compressed debug info sections
>>
>> 1) When creating a .debug_* section and instead create a .zdebug_
>> section.
>> 2) When creating a fragment in a .zdebug_* section, make it a compressed
>> fragment.
>> 3) When computing the size of a compressed section, compress the data
>> and use the size of the compressed data.
>> 4) Emit the compressed bytes.
>>
>> Also, check that only if a section has a compressed fragment, then
>> that is the only fragment in the section.
>>
>> Assert-fail if the fragment's data is modified after it is compressed.
>>
>> Initial review on llvm-commits by Eric Christopher and Rafael Espindola.
>>
>> Added:
>> llvm/trunk/test/MC/ELF/compression.s
>> Modified:
>> llvm/trunk/include/llvm/MC/MCAssembler.h
>> llvm/trunk/lib/MC/MCAsmInfo.cpp
>> llvm/trunk/lib/MC/MCAssembler.cpp
>> llvm/trunk/lib/MC/MCContext.cpp
>> llvm/trunk/lib/MC/MCObjectStreamer.cpp
>> llvm/trunk/tools/llvm-mc/llvm-mc.cpp
>>
>> Modified: llvm/trunk/include/llvm/MC/MCAssembler.h
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/MC/MCAssem
>> bler.h?rev=204958&r1=204957&r2=204958&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/include/llvm/MC/MCAssembler.h (original)
>> +++ llvm/trunk/include/llvm/MC/MCAssembler.h Thu Mar 27 15:45:58 2014
>> @@ -51,6 +51,7 @@ public:
>> enum FragmentType {
>> FT_Align,
>> FT_Data,
>> + FT_Compressed,
>> FT_CompactEncodedInst,
>> FT_Fill,
>> FT_Relaxable,
>> @@ -160,6 +161,7 @@ public:
>> return false;
>> case MCFragment::FT_Relaxable:
>> case MCFragment::FT_CompactEncodedInst:
>> + case MCFragment::FT_Compressed:
>> case MCFragment::FT_Data:
>> return true;
>> }
>> @@ -194,7 +196,8 @@ public:
>>
>> static bool classof(const MCFragment *F) {
>> MCFragment::FragmentType Kind = F->getKind();
>> - return Kind == MCFragment::FT_Relaxable || Kind == MCFragment::FT_Data;
>> + return Kind == MCFragment::FT_Relaxable || Kind == MCFragment::FT_Data ||
>> + Kind == MCFragment::FT_Compressed;
>> }
>> };
>>
>> @@ -213,6 +216,11 @@ class MCDataFragment : public MCEncodedF
>>
>> /// Fixups - The list of fixups in this fragment.
>> SmallVector<MCFixup, 4> Fixups;
>> +protected:
>> + MCDataFragment(MCFragment::FragmentType FType, MCSectionData *SD = 0)
>> + : MCEncodedFragmentWithFixups(FType, SD), HasInstructions(false),
>> + AlignToBundleEnd(false) {}
>> +
>> public:
>> MCDataFragment(MCSectionData *SD = 0)
>> : MCEncodedFragmentWithFixups(FT_Data, SD), @@ -246,10 +254,21 @@
>> public:
>> const_fixup_iterator fixup_end() const override {return
>> Fixups.end();}
>>
>> static bool classof(const MCFragment *F) {
>> - return F->getKind() == MCFragment::FT_Data;
>> + return F->getKind() == MCFragment::FT_Data ||
>> + F->getKind() == MCFragment::FT_Compressed;
>> }
>> };
>>
>> +class MCCompressedFragment: public MCDataFragment {
>> + mutable SmallVector<char, 32> CompressedContents;
>> +public:
>> + MCCompressedFragment(MCSectionData *SD = nullptr)
>> + : MCDataFragment(FT_Compressed, SD) {}
>> + const SmallVectorImpl<char> &getCompressedContents() const;
>> + using MCDataFragment::getContents;
>> + SmallVectorImpl<char> &getContents() override; };
>> +
>> /// This is a compact (memory-size-wise) fragment for holding an
>> encoded /// instruction (non-relaxable) that has no fixups
>> registered. When applicable, /// it can be used instead of
>> MCDataFragment and lead to lower memory
>>
>> Modified: llvm/trunk/lib/MC/MCAsmInfo.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/MC/MCAsmInfo.cpp?re
>> v=204958&r1=204957&r2=204958&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/lib/MC/MCAsmInfo.cpp (original)
>> +++ llvm/trunk/lib/MC/MCAsmInfo.cpp Thu Mar 27 15:45:58 2014
>> @@ -100,6 +100,8 @@ MCAsmInfo::MCAsmInfo() {
>> // architecture basis.
>> // - The target subclasses for AArch64, ARM, and X86 handle these cases
>> UseIntegratedAssembler = false;
>> +
>> + CompressDebugSections = false;
>> }
>>
>> MCAsmInfo::~MCAsmInfo() {
>>
>> Modified: llvm/trunk/lib/MC/MCAssembler.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/MC/MCAssembler.cpp?
>> rev=204958&r1=204957&r2=204958&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/lib/MC/MCAssembler.cpp (original)
>> +++ llvm/trunk/lib/MC/MCAssembler.cpp Thu Mar 27 15:45:58 2014
>> @@ -28,6 +28,9 @@
>> #include "llvm/Support/LEB128.h"
>> #include "llvm/Support/TargetRegistry.h"
>> #include "llvm/Support/raw_ostream.h"
>> +#include "llvm/Support/MemoryBuffer.h"
>> +#include "llvm/Support/Compression.h"
>> +#include "llvm/Support/Host.h"
>>
>> using namespace llvm;
>>
>> @@ -230,6 +233,39 @@ MCEncodedFragmentWithFixups::~MCEncodedF
>>
>> /* *** */
>>
>> +const SmallVectorImpl<char>
>> +&MCCompressedFragment::getCompressedContents() const {
>> + assert(getParent()->size() == 1 &&
>> + "Only compress sections containing a single fragment");
>> + if (CompressedContents.empty()) {
>> + std::unique_ptr<MemoryBuffer> CompressedSection;
>> + zlib::Status Success =
>> + zlib::compress(StringRef(getContents().data(), getContents().size()),
>> + CompressedSection);
>> + (void)Success;
>> + assert(Success == zlib::StatusOK);
>> + CompressedContents.push_back('Z');
>> + CompressedContents.push_back('L');
>> + CompressedContents.push_back('I');
>> + CompressedContents.push_back('B');
>> + uint64_t Size = getContents().size();
>> + if (sys::IsLittleEndianHost)
>> + Size = sys::SwapByteOrder(Size);
>> + CompressedContents.append(reinterpret_cast<char *>(&Size),
>> + reinterpret_cast<char *>(&Size + 1));
>> + CompressedContents.append(CompressedSection->getBuffer().begin(),
>> + CompressedSection->getBuffer().end());
>> + }
>> + return CompressedContents;
>> +}
>> +
>> +SmallVectorImpl<char> &MCCompressedFragment::getContents() {
>> + assert(CompressedContents.empty() &&
>> + "Fragment contents should not be altered after
>> +compression");
>> + return MCDataFragment::getContents(); }
>> +
>> +/* *** */
>> +
>> MCSectionData::MCSectionData() : Section(0) {}
>>
>> MCSectionData::MCSectionData(const MCSection &_Section, MCAssembler
>> *A) @@ -430,6 +466,8 @@ uint64_t MCAssembler::computeFragmentSiz
>> case MCFragment::FT_Relaxable:
>> case MCFragment::FT_CompactEncodedInst:
>> return cast<MCEncodedFragment>(F).getContents().size();
>> + case MCFragment::FT_Compressed:
>> + return
>> + cast<MCCompressedFragment>(F).getCompressedContents().size();
>> case MCFragment::FT_Fill:
>> return cast<MCFillFragment>(F).getSize();
>>
>> @@ -618,6 +656,11 @@ static void writeFragment(const MCAssemb
>> break;
>> }
>>
>> + case MCFragment::FT_Compressed:
>> + ++stats::EmittedDataFragments;
>> + OW->WriteBytes(cast<MCCompressedFragment>(F).getCompressedContents());
>> + break;
>> +
>> case MCFragment::FT_Data:
>> ++stats::EmittedDataFragments;
>> writeFragmentContents(F, OW);
>> @@ -694,6 +737,7 @@ void MCAssembler::writeSectionData(const
>> ie = SD->end(); it != ie; ++it) {
>> switch (it->getKind()) {
>> default: llvm_unreachable("Invalid fragment in virtual
>> section!");
>> + case MCFragment::FT_Compressed:
>> case MCFragment::FT_Data: {
>> // Check that we aren't trying to write a non-zero contents (or fixups)
>> // into a virtual section. This is to support clients which
>> use standard @@ -1021,6 +1065,8 @@ void MCFragment::dump() {
>> switch (getKind()) {
>> case MCFragment::FT_Align: OS << "MCAlignFragment"; break;
>> case MCFragment::FT_Data: OS << "MCDataFragment"; break;
>> + case MCFragment::FT_Compressed:
>> + OS << "MCCompressedFragment"; break;
>> case MCFragment::FT_CompactEncodedInst:
>> OS << "MCCompactEncodedInstFragment"; break;
>> case MCFragment::FT_Fill: OS << "MCFillFragment"; break; @@
>> -1047,6 +1093,7 @@ void MCFragment::dump() {
>> << " MaxBytesToEmit:" << AF->getMaxBytesToEmit() << ">";
>> break;
>> }
>> + case MCFragment::FT_Compressed:
>> case MCFragment::FT_Data: {
>> const MCDataFragment *DF = cast<MCDataFragment>(this);
>> OS << "\n ";
>>
>> Modified: llvm/trunk/lib/MC/MCContext.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/MC/MCContext.cpp?re
>> v=204958&r1=204957&r2=204958&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/lib/MC/MCContext.cpp (original)
>> +++ llvm/trunk/lib/MC/MCContext.cpp Thu Mar 27 15:45:58 2014
>> @@ -37,13 +37,13 @@ typedef std::map<SectionGroupPair, const
>>
>> MCContext::MCContext(const MCAsmInfo *mai, const MCRegisterInfo *mri,
>> const MCObjectFileInfo *mofi, const SourceMgr *mgr,
>> - bool DoAutoReset) :
>> - SrcMgr(mgr), MAI(mai), MRI(mri), MOFI(mofi),
>> - Allocator(), Symbols(Allocator), UsedNames(Allocator),
>> - NextUniqueID(0),
>> - CurrentDwarfLoc(0,0,0,DWARF2_FLAG_IS_STMT,0,0),
>> - DwarfLocSeen(false), GenDwarfForAssembly(false),
>> GenDwarfFileNumber(0),
>> - AllowTemporaryLabels(true), DwarfCompileUnitID(0),
>> AutoReset(DoAutoReset) {
>> + bool DoAutoReset)
>> + : SrcMgr(mgr), MAI(mai), MRI(mri), MOFI(mofi), Allocator(),
>> + Symbols(Allocator), UsedNames(Allocator), NextUniqueID(0),
>> + CurrentDwarfLoc(0, 0, 0, DWARF2_FLAG_IS_STMT, 0, 0), DwarfLocSeen(false),
>> + GenDwarfForAssembly(false), GenDwarfFileNumber(0),
>> + AllowTemporaryLabels(true), DwarfCompileUnitID(0),
>> + AutoReset(DoAutoReset) {
>>
>> error_code EC = llvm::sys::fs::current_path(CompilationDir);
>> if (EC)
>> @@ -251,6 +251,11 @@ getELFSection(StringRef Section, unsigne
>> ELFUniquingMap = new ELFUniqueMapTy();
>> ELFUniqueMapTy &Map = *(ELFUniqueMapTy*)ELFUniquingMap;
>>
>> + SmallString<32> ZDebugName;
>> + if (MAI->compressDebugSections() && Section.startswith(".debug_") &&
>> + Section != ".debug_frame")
>> + Section = (".z" + Section.drop_front(1)).toStringRef(ZDebugName);
>> +
>> // Do the lookup, if we have a hit, return it.
>> std::pair<ELFUniqueMapTy::iterator, bool> Entry = Map.insert(
>> std::make_pair(SectionGroupPair(Section, Group), (MCSectionELF
>> *)0));
>>
>> Modified: llvm/trunk/lib/MC/MCObjectStreamer.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/MC/MCObjectStreamer
>> .cpp?rev=204958&r1=204957&r2=204958&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/lib/MC/MCObjectStreamer.cpp (original)
>> +++ llvm/trunk/lib/MC/MCObjectStreamer.cpp Thu Mar 27 15:45:58 2014
>> @@ -20,6 +20,7 @@
>> #include "llvm/MC/MCSection.h"
>> #include "llvm/MC/MCSymbol.h"
>> #include "llvm/Support/ErrorHandling.h"
>> +#include "llvm/MC/MCSectionELF.h"
>> using namespace llvm;
>>
>> MCObjectStreamer::MCObjectStreamer(MCContext &Context, MCAsmBackend
>> &TAB, @@ -63,7 +64,11 @@ MCDataFragment *MCObjectStreamer::getOrC
>> // When bundling is enabled, we don't want to add data to a fragment that
>> // already has instructions (see MCELFStreamer::EmitInstToData for details)
>> if (!F || (Assembler->isBundlingEnabled() && F->hasInstructions())) {
>> - F = new MCDataFragment();
>> + const auto *Sec = dyn_cast<MCSectionELF>(&getCurrentSectionData()->getSection());
>> + if (Sec && Sec->getSectionName().startswith(".zdebug_"))
>> + F = new MCCompressedFragment();
>> + else
>> + F = new MCDataFragment();
>> insert(F);
>> }
>> return F;
>>
>> Added: llvm/trunk/test/MC/ELF/compression.s
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/test/MC/ELF/compression
>> .s?rev=204958&view=auto
>> ======================================================================
>> ========
>> --- llvm/trunk/test/MC/ELF/compression.s (added)
>> +++ llvm/trunk/test/MC/ELF/compression.s Thu Mar 27 15:45:58 2014
>> @@ -0,0 +1,18 @@
>> +// RUN: llvm-mc -filetype=obj -compress-debug-sections -triple
>> +x86_64-pc-linux-gnu %s -o - | llvm-objdump -s - | FileCheck %s
>> +
>> +// CHECK: Contents of section .zdebug_line:
>> +// Check for the 'ZLIB' file magic at the start of the section //
>> +CHECK-NEXT: ZLIB // We shouldn't compress the debug_frame section,
>> +since it can be relaxed // CHECK: Contents of section .debug_frame //
>> +CHECK-NOT: ZLIB
>> +
>> + .section .debug_line,"", at progbits
>> + .text
>> +foo:
>> + .cfi_startproc
>> + .file 1 "Driver.ii"
>> + .loc 1 2 0
>> + nop
>> + .cfi_endproc
>> + .cfi_sections .debug_frame
>>
>> Modified: llvm/trunk/tools/llvm-mc/llvm-mc.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-mc/llvm-mc.c
>> pp?rev=204958&r1=204957&r2=204958&view=diff
>> ======================================================================
>> ========
>> --- llvm/trunk/tools/llvm-mc/llvm-mc.cpp (original)
>> +++ llvm/trunk/tools/llvm-mc/llvm-mc.cpp Thu Mar 27 15:45:58 2014
>> @@ -50,6 +50,9 @@ static cl::opt<bool> ShowEncoding("show-encoding",
>> cl::desc("Show instruction encodings"));
>>
>> static cl::opt<bool>
>> +CompressDebugSections("compress-debug-sections", cl::desc("Compress
>> +DWARF debug sections"));
>> +
>> +static cl::opt<bool>
>> ShowInst("show-inst", cl::desc("Show internal instruction
>> representation"));
>>
>> static cl::opt<bool>
>> @@ -381,6 +384,9 @@ int main(int argc, char **argv) {
>> std::unique_ptr<MCAsmInfo> MAI(TheTarget->createMCAsmInfo(*MRI, TripleName));
>> assert(MAI && "Unable to create target asm info!");
>>
>> + if (CompressDebugSections)
>> + MAI->setCompressDebugSections(true);
>> +
>> // FIXME: This is not pretty. MCContext has a ptr to MCObjectFileInfo and
>> // MCObjectFileInfo needs a MCContext reference in order to initialize itself.
>> std::unique_ptr<MCObjectFileInfo> MOFI(new MCObjectFileInfo());
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list