<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Oct 6, 2016 at 11:42 PM, Zachary Turner <span dir="ltr"><<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Ok I think I understand this more now.  I don't think we should expose this method, because for one thing just writing the data to this stream is not sufficient to link it up to the PDB.  It has to be referenced from the DBI stream, and is one of the "optional debug streams" that comes at the end of the DBI stream.<div><br></div><div>So instead, how about adding something to DbiStreamBuilder.  It could be called</div><div><br></div><div>void setDbgStreamBytes(<wbr>DbgHeaderType Type, ArrayRef<uint8_t> Bytes);</div></div></blockquote><div><br></div><div>I tried this but I think I don't like the result. The problem is that this method has to add a new stream to a PDB file, so DbiStreamBuilder is now builds not only a DBI stream but other streams. I'd instead add this method to PDBFileBuilder so that new streams are owned by a PDBFileBuilder. PDBFileBuilder is now responsible to keep the debug indices consistent.<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Then you call this as</div><div><br></div><div>setDbgStreamBytes(<wbr>DbgHeaderType::SectionHdr, SectionHeaderBytes);</div><div><br></div><div>Then, in DbiStreamBuilder::commit(), it does the right thing for each optional debug info stream that has data (i.e. goes through the MsfLayout to create a new indexed stream, etc).  Also in DbiStreamBuilder::commit() you will need to write the optional debug info data at the very end of the dbi stream.</div></div><div class="gmail-HOEnZb"><div class="gmail-h5"><br><div class="gmail_quote"><div dir="ltr">On Thu, Oct 6, 2016 at 8:52 PM Zachary Turner <<a href="mailto:zturner@google.com" target="_blank">zturner@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Ahh, i see the other patch now.<br class="gmail-m_3504198211282225599gmail_msg"><br class="gmail-m_3504198211282225599gmail_msg">Wouldn't it be possible to create SectionHeaderStreamBuilder with a well defined interface, then call MsfBuilder.<wbr>getSectionHeaderBuilder()?<br class="gmail-m_3504198211282225599gmail_msg"><div class="gmail_quote gmail-m_3504198211282225599gmail_msg"><div dir="ltr" class="gmail-m_3504198211282225599gmail_msg">On Thu, Oct 6, 2016 at 8:50 PM Zachary Turner <<a href="mailto:zturner@google.com" class="gmail-m_3504198211282225599gmail_msg" target="_blank">zturner@google.com</a>> wrote:<br class="gmail-m_3504198211282225599gmail_msg"></div><blockquote class="gmail_quote gmail-m_3504198211282225599gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">What will this be used for?  I tried to approach this with the mindset of making it impossible to construct invalid pdb data, because your only interface to writing is via a closed api.  Allowing writing of arbitrary bytes breaks this.  What use did you have in mind?<br class="gmail-m_3504198211282225599gmail_msg"><br class="gmail-m_3504198211282225599gmail_msg"><br class="gmail-m_3504198211282225599gmail_msg"><div class="gmail_quote gmail-m_3504198211282225599gmail_msg"><div dir="ltr" class="gmail-m_3504198211282225599gmail_msg">On Thu, Oct 6, 2016 at 8:25 PM Rui Ueyama <<a href="mailto:ruiu@google.com" class="gmail-m_3504198211282225599gmail_msg" target="_blank">ruiu@google.com</a>> wrote:<br class="gmail-m_3504198211282225599gmail_msg"></div><blockquote class="gmail_quote gmail-m_3504198211282225599gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">ruiu created this revision.<br class="gmail-m_3504198211282225599gmail_msg">
ruiu added a reviewer: zturner.<br class="gmail-m_3504198211282225599gmail_msg">
ruiu added a subscriber: llvm-commits.<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
Previously, there is no way to create a stream other than<br class="gmail-m_3504198211282225599gmail_msg">
pre-defined special stream such as DBI or IPI. This patch<br class="gmail-m_3504198211282225599gmail_msg">
adds a new method, addStream, to add a stream of arbitrary<br class="gmail-m_3504198211282225599gmail_msg">
bytes to a PDB file. The new function takes a byte array<br class="gmail-m_3504198211282225599gmail_msg">
and returns the index of the new stream.<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
<a href="https://reviews.llvm.org/D25356" rel="noreferrer" class="gmail-m_3504198211282225599gmail_msg" target="_blank">https://reviews.llvm.org/<wbr>D25356</a><br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
Files:<br class="gmail-m_3504198211282225599gmail_msg">
  include/llvm/DebugInfo/PDB/<wbr>Raw/DbiStreamBuilder.h<br class="gmail-m_3504198211282225599gmail_msg">
  include/llvm/DebugInfo/PDB/<wbr>Raw/PDBFileBuilder.h<br class="gmail-m_3504198211282225599gmail_msg">
  lib/DebugInfo/PDB/Raw/<wbr>DbiStreamBuilder.cpp<br class="gmail-m_3504198211282225599gmail_msg">
  lib/DebugInfo/PDB/Raw/<wbr>PDBFileBuilder.cpp<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
Index: lib/DebugInfo/PDB/Raw/<wbr>PDBFileBuilder.cpp<br class="gmail-m_3504198211282225599gmail_msg">
==============================<wbr>==============================<wbr>=======<br class="gmail-m_3504198211282225599gmail_msg">
--- lib/DebugInfo/PDB/Raw/<wbr>PDBFileBuilder.cpp<br class="gmail-m_3504198211282225599gmail_msg">
+++ lib/DebugInfo/PDB/Raw/<wbr>PDBFileBuilder.cpp<br class="gmail-m_3504198211282225599gmail_msg">
@@ -66,6 +66,16 @@<br class="gmail-m_3504198211282225599gmail_msg">
   return *Ipi;<br class="gmail-m_3504198211282225599gmail_msg">
 }<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
+Expected<uint32_t> PDBFileBuilder::addStream(<wbr>ArrayRef<uint8_t> Data) {<br class="gmail-m_3504198211282225599gmail_msg">
+  auto ExpectedIndex = Msf->addStream(Data.size());<br class="gmail-m_3504198211282225599gmail_msg">
+  if (!ExpectedIndex)<br class="gmail-m_3504198211282225599gmail_msg">
+    return ExpectedIndex.takeError();<br class="gmail-m_3504198211282225599gmail_msg">
+  uint32_t Index = std::move(*ExpectedIndex);<br class="gmail-m_3504198211282225599gmail_msg">
+<br class="gmail-m_3504198211282225599gmail_msg">
+  Streams.emplace_back(Data, Index);<br class="gmail-m_3504198211282225599gmail_msg">
+  return *ExpectedIndex;<br class="gmail-m_3504198211282225599gmail_msg">
+}<br class="gmail-m_3504198211282225599gmail_msg">
+<br class="gmail-m_3504198211282225599gmail_msg">
 Expected<msf::MSFLayout> PDBFileBuilder::<wbr>finalizeMsfLayout() const {<br class="gmail-m_3504198211282225599gmail_msg">
   if (Info) {<br class="gmail-m_3504198211282225599gmail_msg">
     if (auto EC = Info->finalizeMsfLayout())<br class="gmail-m_3504198211282225599gmail_msg">
@@ -189,5 +199,14 @@<br class="gmail-m_3504198211282225599gmail_msg">
       return EC;<br class="gmail-m_3504198211282225599gmail_msg">
   }<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
+  for (auto &Pair : Streams) {<br class="gmail-m_3504198211282225599gmail_msg">
+    ArrayRef<uint8_t> Data = Pair.first;<br class="gmail-m_3504198211282225599gmail_msg">
+    uint32_t Idx = Pair.second;<br class="gmail-m_3504198211282225599gmail_msg">
+    auto Stream = WritableMappedBlockStream::<wbr>createIndexedStream(Layout, Buffer, Idx);<br class="gmail-m_3504198211282225599gmail_msg">
+    StreamWriter Writer(*Stream);<br class="gmail-m_3504198211282225599gmail_msg">
+    if (auto EC = Writer.writeArray(Data))<br class="gmail-m_3504198211282225599gmail_msg">
+      return EC;<br class="gmail-m_3504198211282225599gmail_msg">
+  }<br class="gmail-m_3504198211282225599gmail_msg">
+<br class="gmail-m_3504198211282225599gmail_msg">
   return Buffer.commit();<br class="gmail-m_3504198211282225599gmail_msg">
 }<br class="gmail-m_3504198211282225599gmail_msg">
Index: lib/DebugInfo/PDB/Raw/<wbr>DbiStreamBuilder.cpp<br class="gmail-m_3504198211282225599gmail_msg">
==============================<wbr>==============================<wbr>=======<br class="gmail-m_3504198211282225599gmail_msg">
--- lib/DebugInfo/PDB/Raw/<wbr>DbiStreamBuilder.cpp<br class="gmail-m_3504198211282225599gmail_msg">
+++ lib/DebugInfo/PDB/Raw/<wbr>DbiStreamBuilder.cpp<br class="gmail-m_3504198211282225599gmail_msg">
@@ -44,6 +44,10 @@<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
 void DbiStreamBuilder::<wbr>setMachineType(PDB_Machine M) { MachineType = M; }<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
+void DbiStreamBuilder::<wbr>setDebugStreamIndex(<wbr>DbgHeaderType Type, uint32_t Index) {<br class="gmail-m_3504198211282225599gmail_msg">
+  DbgStreams[(int)Type] = Index;<br class="gmail-m_3504198211282225599gmail_msg">
+}<br class="gmail-m_3504198211282225599gmail_msg">
+<br class="gmail-m_3504198211282225599gmail_msg">
 uint32_t DbiStreamBuilder::<wbr>calculateSerializedLength() const {<br class="gmail-m_3504198211282225599gmail_msg">
   // For now we only support serializing the header.<br class="gmail-m_3504198211282225599gmail_msg">
   return sizeof(DbiStreamHeader) + calculateFileInfoSubstreamSize<wbr>() +<br class="gmail-m_3504198211282225599gmail_msg">
Index: include/llvm/DebugInfo/PDB/<wbr>Raw/PDBFileBuilder.h<br class="gmail-m_3504198211282225599gmail_msg">
==============================<wbr>==============================<wbr>=======<br class="gmail-m_3504198211282225599gmail_msg">
--- include/llvm/DebugInfo/PDB/<wbr>Raw/PDBFileBuilder.h<br class="gmail-m_3504198211282225599gmail_msg">
+++ include/llvm/DebugInfo/PDB/<wbr>Raw/PDBFileBuilder.h<br class="gmail-m_3504198211282225599gmail_msg">
@@ -44,6 +44,9 @@<br class="gmail-m_3504198211282225599gmail_msg">
   TpiStreamBuilder &getTpiBuilder();<br class="gmail-m_3504198211282225599gmail_msg">
   TpiStreamBuilder &getIpiBuilder();<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
+  // Add given bytes as a new stream. Returns the stream index.<br class="gmail-m_3504198211282225599gmail_msg">
+  Expected<uint32_t> addStream(ArrayRef<uint8_t> Data);<br class="gmail-m_3504198211282225599gmail_msg">
+<br class="gmail-m_3504198211282225599gmail_msg">
   Expected<std::unique_ptr<<wbr>PDBFile>><br class="gmail-m_3504198211282225599gmail_msg">
   build(std::unique_ptr<msf::<wbr>WritableStream> PdbFileBuffer);<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
@@ -59,6 +62,7 @@<br class="gmail-m_3504198211282225599gmail_msg">
   std::unique_ptr<<wbr>DbiStreamBuilder> Dbi;<br class="gmail-m_3504198211282225599gmail_msg">
   std::unique_ptr<<wbr>TpiStreamBuilder> Tpi;<br class="gmail-m_3504198211282225599gmail_msg">
   std::unique_ptr<<wbr>TpiStreamBuilder> Ipi;<br class="gmail-m_3504198211282225599gmail_msg">
+  std::vector<std::pair<<wbr>ArrayRef<uint8_t>, uint32_t>> Streams;<br class="gmail-m_3504198211282225599gmail_msg">
 };<br class="gmail-m_3504198211282225599gmail_msg">
 }<br class="gmail-m_3504198211282225599gmail_msg">
 }<br class="gmail-m_3504198211282225599gmail_msg">
Index: include/llvm/DebugInfo/PDB/<wbr>Raw/DbiStreamBuilder.h<br class="gmail-m_3504198211282225599gmail_msg">
==============================<wbr>==============================<wbr>=======<br class="gmail-m_3504198211282225599gmail_msg">
--- include/llvm/DebugInfo/PDB/<wbr>Raw/DbiStreamBuilder.h<br class="gmail-m_3504198211282225599gmail_msg">
+++ include/llvm/DebugInfo/PDB/<wbr>Raw/DbiStreamBuilder.h<br class="gmail-m_3504198211282225599gmail_msg">
@@ -44,6 +44,7 @@<br class="gmail-m_3504198211282225599gmail_msg">
   void setPdbDllRbld(uint16_t R);<br class="gmail-m_3504198211282225599gmail_msg">
   void setFlags(uint16_t F);<br class="gmail-m_3504198211282225599gmail_msg">
   void setMachineType(PDB_Machine M);<br class="gmail-m_3504198211282225599gmail_msg">
+  void setDebugStreamIndex(<wbr>DbgHeaderType Type, uint32_t Index);<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
   uint32_t calculateSerializedLength() const;<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
<br class="gmail-m_3504198211282225599gmail_msg">
</blockquote></div></blockquote></div></blockquote></div>
</div></div></blockquote></div><br></div></div>