[PATCH] D20654: [pdb] Try super hard to conserve memory in llvm-pdbdump

[Rui Ueyama via llvm-commits]

ruiu added a comment.

A more radical idea would be to call mmap multiple times with explicit "map-to" addresses to map parts of a PDB file to stitch the blocks that consist a stream together in the virtual memory space. Then you could use regular StringRef and ArrayRef, and it would probably be very fast. But this wouldn't work if your machine's page size is larger than the file's page size. So I don't know if it's feasible.


================
Comment at: include/llvm/DebugInfo/CodeView/StreamArray.h:23
@@ +22,3 @@
+/// be a file on disk, or it could be a PDB stream where bytes are stored as
+/// discontiguous blocks on a filesystem.  Usually it is desirable to treat
+/// arrays as contiguous blocks of memory, but doing so with large PDB files,
----------------
Is "discontiguous blocks in a file" better?

================
Comment at: include/llvm/DebugInfo/CodeView/StreamArray.h:96
@@ +95,3 @@
+  StreamView Stream;
+  mutable T TempItem;
+};
----------------
It will construct an object of type T. If T's constructor needs a parameter, it wouldn't compile. You want to use

  mutable uint8_t TempItem[sizeof(T)];

instead for a buffer.

================
Comment at: lib/DebugInfo/CodeView/StreamArray.cpp:20
@@ +19,3 @@
+
+  const CacheItem &Item = RecordOffsets[Index];
+  ArrayRef<uint8_t> Data;
----------------
So RecordOffsets maps each byte position to each byte position, correct? I wonder if it's going to be too much. If you access sequentially from 0 to 100MB using operator[], is it going to insert 100 million items to the hash?

================
Comment at: lib/DebugInfo/CodeView/StreamArray.cpp:51
@@ +50,2 @@
+  }
+}
----------------
Please add newline at end of file.

================
Comment at: lib/DebugInfo/CodeView/StreamString.cpp:1
@@ +1,2 @@
+//===- StreamString.cpp - A string backed by an arbitrary stream ---------===//
+//
----------------
I wouldn't check this file in until I need to. ".h" only should be fine.


http://reviews.llvm.org/D20654