[llvm] r204958 - DebugInfo: Support for compressed debug info sections
Evgeniy Stepanov
eugeni.stepanov at gmail.com
Fri Mar 28 08:40:48 PDT 2014
We might miss zlib. Note that the test is failing in asan build, too,
on the same bot.
On Fri, Mar 28, 2014 at 7:31 PM, David Blaikie <dblaikie at gmail.com> wrote:
> On Fri, Mar 28, 2014 at 1:00 AM, Evgeniy Stepanov
> <eugeni.stepanov at gmail.com> wrote:
>> With this change we get an assertion failure on the bots:
>>
>> lib/MC/MCAssembler.cpp:245: const SmallVectorImpl<char>
>> &llvm::MCCompressedFragment::getCompressedContents() const: Assertion
>> `Success == zlib::StatusOK' failed.
>>
>> http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-bootstrap/builds/2813/steps/check-llvm%20msan/logs/stdio
>
> Do you have a command line I could use to reproduce this locally? I've
> not seen this (& not sure why msan would make a difference to this
> outcome)
>
> Perhaps you don't have zlib installed/available? (I should gate this
> feature on the presence of zlib, but hadn't gotten around to that &
> wasn't sure it was a common issue in practice)
>
>>
>>
>> On Fri, Mar 28, 2014 at 12:45 AM, David Blaikie <dblaikie at gmail.com> wrote:
>>> Author: dblaikie
>>> Date: Thu Mar 27 15:45:58 2014
>>> New Revision: 204958
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=204958&view=rev
>>> Log:
>>> DebugInfo: Support for compressed debug info sections
>>>
>>> 1) When creating a .debug_* section and instead create a .zdebug_
>>> section.
>>> 2) When creating a fragment in a .zdebug_* section, make it a compressed
>>> fragment.
>>> 3) When computing the size of a compressed section, compress the data
>>> and use the size of the compressed data.
>>> 4) Emit the compressed bytes.
>>>
>>> Also, check that only if a section has a compressed fragment, then that
>>> is the only fragment in the section.
>>>
>>> Assert-fail if the fragment's data is modified after it is compressed.
>>>
>>> Initial review on llvm-commits by Eric Christopher and Rafael Espindola.
>>>
>>> Added:
>>> llvm/trunk/test/MC/ELF/compression.s
>>> Modified:
>>> llvm/trunk/include/llvm/MC/MCAssembler.h
>>> llvm/trunk/lib/MC/MCAsmInfo.cpp
>>> llvm/trunk/lib/MC/MCAssembler.cpp
>>> llvm/trunk/lib/MC/MCContext.cpp
>>> llvm/trunk/lib/MC/MCObjectStreamer.cpp
>>> llvm/trunk/tools/llvm-mc/llvm-mc.cpp
>>>
>>> Modified: llvm/trunk/include/llvm/MC/MCAssembler.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/MC/MCAssembler.h?rev=204958&r1=204957&r2=204958&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/include/llvm/MC/MCAssembler.h (original)
>>> +++ llvm/trunk/include/llvm/MC/MCAssembler.h Thu Mar 27 15:45:58 2014
>>> @@ -51,6 +51,7 @@ public:
>>> enum FragmentType {
>>> FT_Align,
>>> FT_Data,
>>> + FT_Compressed,
>>> FT_CompactEncodedInst,
>>> FT_Fill,
>>> FT_Relaxable,
>>> @@ -160,6 +161,7 @@ public:
>>> return false;
>>> case MCFragment::FT_Relaxable:
>>> case MCFragment::FT_CompactEncodedInst:
>>> + case MCFragment::FT_Compressed:
>>> case MCFragment::FT_Data:
>>> return true;
>>> }
>>> @@ -194,7 +196,8 @@ public:
>>>
>>> static bool classof(const MCFragment *F) {
>>> MCFragment::FragmentType Kind = F->getKind();
>>> - return Kind == MCFragment::FT_Relaxable || Kind == MCFragment::FT_Data;
>>> + return Kind == MCFragment::FT_Relaxable || Kind == MCFragment::FT_Data ||
>>> + Kind == MCFragment::FT_Compressed;
>>> }
>>> };
>>>
>>> @@ -213,6 +216,11 @@ class MCDataFragment : public MCEncodedF
>>>
>>> /// Fixups - The list of fixups in this fragment.
>>> SmallVector<MCFixup, 4> Fixups;
>>> +protected:
>>> + MCDataFragment(MCFragment::FragmentType FType, MCSectionData *SD = 0)
>>> + : MCEncodedFragmentWithFixups(FType, SD), HasInstructions(false),
>>> + AlignToBundleEnd(false) {}
>>> +
>>> public:
>>> MCDataFragment(MCSectionData *SD = 0)
>>> : MCEncodedFragmentWithFixups(FT_Data, SD),
>>> @@ -246,10 +254,21 @@ public:
>>> const_fixup_iterator fixup_end() const override {return Fixups.end();}
>>>
>>> static bool classof(const MCFragment *F) {
>>> - return F->getKind() == MCFragment::FT_Data;
>>> + return F->getKind() == MCFragment::FT_Data ||
>>> + F->getKind() == MCFragment::FT_Compressed;
>>> }
>>> };
>>>
>>> +class MCCompressedFragment: public MCDataFragment {
>>> + mutable SmallVector<char, 32> CompressedContents;
>>> +public:
>>> + MCCompressedFragment(MCSectionData *SD = nullptr)
>>> + : MCDataFragment(FT_Compressed, SD) {}
>>> + const SmallVectorImpl<char> &getCompressedContents() const;
>>> + using MCDataFragment::getContents;
>>> + SmallVectorImpl<char> &getContents() override;
>>> +};
>>> +
>>> /// This is a compact (memory-size-wise) fragment for holding an encoded
>>> /// instruction (non-relaxable) that has no fixups registered. When applicable,
>>> /// it can be used instead of MCDataFragment and lead to lower memory
>>>
>>> Modified: llvm/trunk/lib/MC/MCAsmInfo.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/MC/MCAsmInfo.cpp?rev=204958&r1=204957&r2=204958&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/MC/MCAsmInfo.cpp (original)
>>> +++ llvm/trunk/lib/MC/MCAsmInfo.cpp Thu Mar 27 15:45:58 2014
>>> @@ -100,6 +100,8 @@ MCAsmInfo::MCAsmInfo() {
>>> // architecture basis.
>>> // - The target subclasses for AArch64, ARM, and X86 handle these cases
>>> UseIntegratedAssembler = false;
>>> +
>>> + CompressDebugSections = false;
>>> }
>>>
>>> MCAsmInfo::~MCAsmInfo() {
>>>
>>> Modified: llvm/trunk/lib/MC/MCAssembler.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/MC/MCAssembler.cpp?rev=204958&r1=204957&r2=204958&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/MC/MCAssembler.cpp (original)
>>> +++ llvm/trunk/lib/MC/MCAssembler.cpp Thu Mar 27 15:45:58 2014
>>> @@ -28,6 +28,9 @@
>>> #include "llvm/Support/LEB128.h"
>>> #include "llvm/Support/TargetRegistry.h"
>>> #include "llvm/Support/raw_ostream.h"
>>> +#include "llvm/Support/MemoryBuffer.h"
>>> +#include "llvm/Support/Compression.h"
>>> +#include "llvm/Support/Host.h"
>>>
>>> using namespace llvm;
>>>
>>> @@ -230,6 +233,39 @@ MCEncodedFragmentWithFixups::~MCEncodedF
>>>
>>> /* *** */
>>>
>>> +const SmallVectorImpl<char> &MCCompressedFragment::getCompressedContents() const {
>>> + assert(getParent()->size() == 1 &&
>>> + "Only compress sections containing a single fragment");
>>> + if (CompressedContents.empty()) {
>>> + std::unique_ptr<MemoryBuffer> CompressedSection;
>>> + zlib::Status Success =
>>> + zlib::compress(StringRef(getContents().data(), getContents().size()),
>>> + CompressedSection);
>>> + (void)Success;
>>> + assert(Success == zlib::StatusOK);
>>> + CompressedContents.push_back('Z');
>>> + CompressedContents.push_back('L');
>>> + CompressedContents.push_back('I');
>>> + CompressedContents.push_back('B');
>>> + uint64_t Size = getContents().size();
>>> + if (sys::IsLittleEndianHost)
>>> + Size = sys::SwapByteOrder(Size);
>>> + CompressedContents.append(reinterpret_cast<char *>(&Size),
>>> + reinterpret_cast<char *>(&Size + 1));
>>> + CompressedContents.append(CompressedSection->getBuffer().begin(),
>>> + CompressedSection->getBuffer().end());
>>> + }
>>> + return CompressedContents;
>>> +}
>>> +
>>> +SmallVectorImpl<char> &MCCompressedFragment::getContents() {
>>> + assert(CompressedContents.empty() &&
>>> + "Fragment contents should not be altered after compression");
>>> + return MCDataFragment::getContents();
>>> +}
>>> +
>>> +/* *** */
>>> +
>>> MCSectionData::MCSectionData() : Section(0) {}
>>>
>>> MCSectionData::MCSectionData(const MCSection &_Section, MCAssembler *A)
>>> @@ -430,6 +466,8 @@ uint64_t MCAssembler::computeFragmentSiz
>>> case MCFragment::FT_Relaxable:
>>> case MCFragment::FT_CompactEncodedInst:
>>> return cast<MCEncodedFragment>(F).getContents().size();
>>> + case MCFragment::FT_Compressed:
>>> + return cast<MCCompressedFragment>(F).getCompressedContents().size();
>>> case MCFragment::FT_Fill:
>>> return cast<MCFillFragment>(F).getSize();
>>>
>>> @@ -618,6 +656,11 @@ static void writeFragment(const MCAssemb
>>> break;
>>> }
>>>
>>> + case MCFragment::FT_Compressed:
>>> + ++stats::EmittedDataFragments;
>>> + OW->WriteBytes(cast<MCCompressedFragment>(F).getCompressedContents());
>>> + break;
>>> +
>>> case MCFragment::FT_Data:
>>> ++stats::EmittedDataFragments;
>>> writeFragmentContents(F, OW);
>>> @@ -694,6 +737,7 @@ void MCAssembler::writeSectionData(const
>>> ie = SD->end(); it != ie; ++it) {
>>> switch (it->getKind()) {
>>> default: llvm_unreachable("Invalid fragment in virtual section!");
>>> + case MCFragment::FT_Compressed:
>>> case MCFragment::FT_Data: {
>>> // Check that we aren't trying to write a non-zero contents (or fixups)
>>> // into a virtual section. This is to support clients which use standard
>>> @@ -1021,6 +1065,8 @@ void MCFragment::dump() {
>>> switch (getKind()) {
>>> case MCFragment::FT_Align: OS << "MCAlignFragment"; break;
>>> case MCFragment::FT_Data: OS << "MCDataFragment"; break;
>>> + case MCFragment::FT_Compressed:
>>> + OS << "MCCompressedFragment"; break;
>>> case MCFragment::FT_CompactEncodedInst:
>>> OS << "MCCompactEncodedInstFragment"; break;
>>> case MCFragment::FT_Fill: OS << "MCFillFragment"; break;
>>> @@ -1047,6 +1093,7 @@ void MCFragment::dump() {
>>> << " MaxBytesToEmit:" << AF->getMaxBytesToEmit() << ">";
>>> break;
>>> }
>>> + case MCFragment::FT_Compressed:
>>> case MCFragment::FT_Data: {
>>> const MCDataFragment *DF = cast<MCDataFragment>(this);
>>> OS << "\n ";
>>>
>>> Modified: llvm/trunk/lib/MC/MCContext.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/MC/MCContext.cpp?rev=204958&r1=204957&r2=204958&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/MC/MCContext.cpp (original)
>>> +++ llvm/trunk/lib/MC/MCContext.cpp Thu Mar 27 15:45:58 2014
>>> @@ -37,13 +37,13 @@ typedef std::map<SectionGroupPair, const
>>>
>>> MCContext::MCContext(const MCAsmInfo *mai, const MCRegisterInfo *mri,
>>> const MCObjectFileInfo *mofi, const SourceMgr *mgr,
>>> - bool DoAutoReset) :
>>> - SrcMgr(mgr), MAI(mai), MRI(mri), MOFI(mofi),
>>> - Allocator(), Symbols(Allocator), UsedNames(Allocator),
>>> - NextUniqueID(0),
>>> - CurrentDwarfLoc(0,0,0,DWARF2_FLAG_IS_STMT,0,0),
>>> - DwarfLocSeen(false), GenDwarfForAssembly(false), GenDwarfFileNumber(0),
>>> - AllowTemporaryLabels(true), DwarfCompileUnitID(0), AutoReset(DoAutoReset) {
>>> + bool DoAutoReset)
>>> + : SrcMgr(mgr), MAI(mai), MRI(mri), MOFI(mofi), Allocator(),
>>> + Symbols(Allocator), UsedNames(Allocator), NextUniqueID(0),
>>> + CurrentDwarfLoc(0, 0, 0, DWARF2_FLAG_IS_STMT, 0, 0), DwarfLocSeen(false),
>>> + GenDwarfForAssembly(false), GenDwarfFileNumber(0),
>>> + AllowTemporaryLabels(true), DwarfCompileUnitID(0),
>>> + AutoReset(DoAutoReset) {
>>>
>>> error_code EC = llvm::sys::fs::current_path(CompilationDir);
>>> if (EC)
>>> @@ -251,6 +251,11 @@ getELFSection(StringRef Section, unsigne
>>> ELFUniquingMap = new ELFUniqueMapTy();
>>> ELFUniqueMapTy &Map = *(ELFUniqueMapTy*)ELFUniquingMap;
>>>
>>> + SmallString<32> ZDebugName;
>>> + if (MAI->compressDebugSections() && Section.startswith(".debug_") &&
>>> + Section != ".debug_frame")
>>> + Section = (".z" + Section.drop_front(1)).toStringRef(ZDebugName);
>>> +
>>> // Do the lookup, if we have a hit, return it.
>>> std::pair<ELFUniqueMapTy::iterator, bool> Entry = Map.insert(
>>> std::make_pair(SectionGroupPair(Section, Group), (MCSectionELF *)0));
>>>
>>> Modified: llvm/trunk/lib/MC/MCObjectStreamer.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/MC/MCObjectStreamer.cpp?rev=204958&r1=204957&r2=204958&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/MC/MCObjectStreamer.cpp (original)
>>> +++ llvm/trunk/lib/MC/MCObjectStreamer.cpp Thu Mar 27 15:45:58 2014
>>> @@ -20,6 +20,7 @@
>>> #include "llvm/MC/MCSection.h"
>>> #include "llvm/MC/MCSymbol.h"
>>> #include "llvm/Support/ErrorHandling.h"
>>> +#include "llvm/MC/MCSectionELF.h"
>>> using namespace llvm;
>>>
>>> MCObjectStreamer::MCObjectStreamer(MCContext &Context, MCAsmBackend &TAB,
>>> @@ -63,7 +64,11 @@ MCDataFragment *MCObjectStreamer::getOrC
>>> // When bundling is enabled, we don't want to add data to a fragment that
>>> // already has instructions (see MCELFStreamer::EmitInstToData for details)
>>> if (!F || (Assembler->isBundlingEnabled() && F->hasInstructions())) {
>>> - F = new MCDataFragment();
>>> + const auto *Sec = dyn_cast<MCSectionELF>(&getCurrentSectionData()->getSection());
>>> + if (Sec && Sec->getSectionName().startswith(".zdebug_"))
>>> + F = new MCCompressedFragment();
>>> + else
>>> + F = new MCDataFragment();
>>> insert(F);
>>> }
>>> return F;
>>>
>>> Added: llvm/trunk/test/MC/ELF/compression.s
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/MC/ELF/compression.s?rev=204958&view=auto
>>> ==============================================================================
>>> --- llvm/trunk/test/MC/ELF/compression.s (added)
>>> +++ llvm/trunk/test/MC/ELF/compression.s Thu Mar 27 15:45:58 2014
>>> @@ -0,0 +1,18 @@
>>> +// RUN: llvm-mc -filetype=obj -compress-debug-sections -triple x86_64-pc-linux-gnu %s -o - | llvm-objdump -s - | FileCheck %s
>>> +
>>> +// CHECK: Contents of section .zdebug_line:
>>> +// Check for the 'ZLIB' file magic at the start of the section
>>> +// CHECK-NEXT: ZLIB
>>> +// We shouldn't compress the debug_frame section, since it can be relaxed
>>> +// CHECK: Contents of section .debug_frame
>>> +// CHECK-NOT: ZLIB
>>> +
>>> + .section .debug_line,"", at progbits
>>> + .text
>>> +foo:
>>> + .cfi_startproc
>>> + .file 1 "Driver.ii"
>>> + .loc 1 2 0
>>> + nop
>>> + .cfi_endproc
>>> + .cfi_sections .debug_frame
>>>
>>> Modified: llvm/trunk/tools/llvm-mc/llvm-mc.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-mc/llvm-mc.cpp?rev=204958&r1=204957&r2=204958&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/tools/llvm-mc/llvm-mc.cpp (original)
>>> +++ llvm/trunk/tools/llvm-mc/llvm-mc.cpp Thu Mar 27 15:45:58 2014
>>> @@ -50,6 +50,9 @@ static cl::opt<bool>
>>> ShowEncoding("show-encoding", cl::desc("Show instruction encodings"));
>>>
>>> static cl::opt<bool>
>>> +CompressDebugSections("compress-debug-sections", cl::desc("Compress DWARF debug sections"));
>>> +
>>> +static cl::opt<bool>
>>> ShowInst("show-inst", cl::desc("Show internal instruction representation"));
>>>
>>> static cl::opt<bool>
>>> @@ -381,6 +384,9 @@ int main(int argc, char **argv) {
>>> std::unique_ptr<MCAsmInfo> MAI(TheTarget->createMCAsmInfo(*MRI, TripleName));
>>> assert(MAI && "Unable to create target asm info!");
>>>
>>> + if (CompressDebugSections)
>>> + MAI->setCompressDebugSections(true);
>>> +
>>> // FIXME: This is not pretty. MCContext has a ptr to MCObjectFileInfo and
>>> // MCObjectFileInfo needs a MCContext reference in order to initialize itself.
>>> std::unique_ptr<MCObjectFileInfo> MOFI(new MCObjectFileInfo());
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list