[llvm] [llvm-strings] Use small buffer instead of reading whole file (PR #163073)

[James Henderson via llvm-commits]

================
@@ -88,41 +90,72 @@ static void parseIntArg(const opt::InputArgList &Args, int ID, T &Value) {
   }
 }
 
-static void strings(raw_ostream &OS, StringRef FileName, StringRef Contents) {
-  auto print = [&OS, FileName](unsigned Offset, StringRef L) {
-    if (L.size() < static_cast<size_t>(MinLength))
-      return;
-    if (PrintFileName)
-      OS << FileName << ": ";
-    switch (Radix) {
-    case none:
-      break;
-    case octal:
-      OS << format("%7o ", Offset);
-      break;
-    case hexadecimal:
-      OS << format("%7x ", Offset);
-      break;
-    case decimal:
-      OS << format("%7u ", Offset);
-      break;
+static void strings(raw_ostream &OS, StringRef FileName,
+                    sys::fs::file_t Handle) {
+  SmallString<sys::fs::DefaultReadChunkSize> Buffer;
+  SmallString<DefaultMinLength> Prefix;
----------------
jh7370 wrote:

I made a mistake recommending `SmallString<DefaultMinLength>` yesterday, because for some reason I had in my head that the `Prefix` would generally only have a small number of characters in it, whereas it could of course have any number of characters in, if the unprintable byte that it ends with just so happens to be on the wrong side of a chunk boundary. In the worst case, you could have a string that spans multiple chunks, which might result in the buffer becoming so big that the memory gains you get by doing chunk-based reads are dwarfed.

That leads me to consider an alternative approach entirely, but I have no idea about the performance of it:
1. Read a chunk.
2. Inspect each character one at a time, keeping a note of the start of any candidate string and adjusting when an unprintable character is seen.
3. When a sequence of `MinLength` printable characters is seen, immediately print the radix and file name (if applicable) then the printable characters.
4. Keep printing the characters (rather than storing them) until an unprintable character is seen, then return to step 2.
5. At the end of a chunk, if we are printing, jump straight to step 4 at the start of the next chunk. If we are not currently printing, but we have buffered characters, include them in the set of characters that count towards step 3 and print them once sufficient more characters are encountered.
6. At EOF, we don't need to do anything (since if the file ends with a printable sequence, it would have been printed already).

My concern with the performance of the above is the runtime cost of printing individual bytes. It could be improved by buffering bytes up to a point, then printing them in blocks, but at the cost of memory of course. You'd need to see if there is an optimum buffer size in this case (keeping in mind that different platforms will likely have different performance characteristics).

I'd be inclined to try the above approach and compare its performance versus your current approach.

https://github.com/llvm/llvm-project/pull/163073