[libcxx-commits] [PATCH] D88878: [libcxx][dsl] Cache the result of compilerMacros()

[Alexander Richardson via Phabricator via libcxx-commits]

arichardson created this revision.
arichardson added reviewers: libc++, ldionne.
Herald added subscribers: libcxx-commits, dexonsmith.
Herald added a project: libc++.
Herald added 1 blocking reviewer(s): libc++.
arichardson requested review of this revision.

This function is called about 80 times during test startup and most of
the invokes the same compiler command. Avoiding running a new process and
parsing the output into a python dict by caching the output. This
noticeably speeds up running a single lit test: for me this patch changes
the time it takes to run lit with --no-execute for a single test from
~7 seconds to ~5.5 seconds.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D88878

Files:
  libcxx/utils/libcxx/test/dsl.py


Index: libcxx/utils/libcxx/test/dsl.py
===================================================================
--- libcxx/utils/libcxx/test/dsl.py
+++ libcxx/utils/libcxx/test/dsl.py
@@ -163,6 +163,19 @@
   If the optional `flags` argument (a string) is provided, these flags will
   be added to the compiler invocation when generating the macros.
   """
+  # Since this function is called many times, we cache the output of the compiler macros
+  # for a given command line.
+  # Note: ideally we would cache this based on the full command line, but we don't know it
+  # in this function, so we use the flags argument + the configured substitutions as the
+  # equivalent of the resulting command line.
+  cache = getattr(config, "_cached_macros", None)
+  # Changes in substitutions should also invalidate the cache since they could affect
+  # the commandline used to dump macros.
+  cached_subs = tuple(config.substitutions)  # Needs to be a hashable type, list won't work
+  if cache is not None:
+    result = cache.get((flags, cached_subs), None)
+    if result is not None:
+      return result
   with _makeConfigTest(config) as test:
     unparsedOutput, err, exitCode, timeoutInfo = _executeScriptInternal(test, [
       "%{{cxx}} -xc++ {} -dM -E %{{flags}} %{{compile_flags}} {}".format(os.devnull, flags)
@@ -173,7 +186,11 @@
       line = line[len('#define '):]
       macro, _, value = line.partition(' ')
       parsedMacros[macro] = value
-    return parsedMacros
+  if not hasattr(config, "_cached_macros"):
+    config._cached_macros = {(flags, cached_subs): parsedMacros}
+  else:
+    config._cached_macros[(flags, cached_subs)] = parsedMacros
+  return parsedMacros
 
 def featureTestMacros(config, flags=''):
   """


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D88878.296387.patch
Type: text/x-patch
Size: 1736 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/libcxx-commits/attachments/20201006/e7ae3d46/attachment-0001.bin>