[llvm-bugs] [Bug 33160] New: litsupport: support for test prefix

[via llvm-bugs]

https://bugs.llvm.org/show_bug.cgi?id=33160

            Bug ID: 33160
           Summary: litsupport: support for test prefix
           Product: Test Suite
           Version: trunk
          Hardware: PC
                OS: All
            Status: NEW
          Severity: normal
          Priority: P
         Component: lit
          Assignee: unassignedbugs at nondot.org
          Reporter: llvm at meinersbur.de
                CC: daniel at zuster.org, llvm-bugs at lists.llvm.org

When test-suite program use the prefix argument, such as

    llvm_singlesource(PREFIX prefix)

the data collector in litsupport do not collect the correct data. Prefixes are
not required to end with a "." as the source seems to expect. Sometime a hyphen
was used, which unfortunately introduced some ambiguity as some tests include a
hyphen in their name. Since r286276, no separator is forced anymore.

Unfortunately during this change of PREFIX,
litsupport/compiletime.py
litsupport/stats.py
litsupport/timeit.py
have not been adapted to this change and therefore collect wrong results.

It seems to not be possible to query the PREFIX value from inside python. My
try to get correct values again is shown below (for stats.py). If feels like a
very hacky approach, so I don't submit it as a patch. I would be glad if
collecting data would work again without my local changes.


def _getStats(context):
    # We compile multiple benchmarks in the same directory in SingleSource
    # mode. Only look at compiletime files starting with the name of our test.
    prefix = ""
    if context.config.single_source:
        prefix = "%s." % os.path.basename(context.executable)

    stats = defaultdict(lambda: 0.0)
    dir = os.path.dirname(context.test.getFilePath())

    while True:
      for path, subdirs, files in os.walk(dir):
          for file in files:
              if file.endswith('.stats') and file.startswith(prefix):
                  fullpath = os.path.join(path, file)
                  _mergeStats(stats, fullpath)

      # It is possible that a prefix has been added to the executable; search 
      # again without the prefix. Because we don't know what the prefix is, we
      # peel off one character after the other
      if len(stats) == 0 and prefix:
          prefix = prefix[1:]
          continue

      if len(stats) == 0:
          print("No stats for
'{name}'".format(name=context.test.getFullName()))
          logging.warning("No stats for '%s'", context.test.getFullName())

      if context.config.single_source and len(stats) != 1:
         print("Found {n} stats for single-source target
'{name}'".format(n=len(stats), name=context.test.getFullName()))


    result = dict()
    for key, value in stats.iteritems():
        result[key] = lit.Test.toMetricValue(value)
    return result

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20170524/ad42688d/attachment.html>