[PATCH] [extra] pptrace - preprocessor tracing and testing tool

[Sean Silva]

(sorry for the delay). As per my earlier comment now that it seems like the
issue Kim was seeing is resolved, feel free to check this in.

More comments inline:


On Mon, Oct 28, 2013 at 8:52 PM, John Thompson <
john.thompson.jtsoftware at gmail.com> wrote:

>   Sean,
>
>   Thanks for the excellent review, and the reference to yaml-bench.  It
> caught a few things so far, and I'll have to use it when I write the other
> tests too.  It didn't like the backslashes in Windows file paths, or the
> colons in SourceLocations, so I put in some character replacing and quoting.
>
>   I revised the for loops as pointed out.
>
>   Also, I wanted to ask about the special cases, such as where I use
> values like (null) to represent when a pointer is null, or (invalid) when a
> SourceLocation has the invalid flag set.  In the former case, passing a
> null pointer is expected in some situations.  The YAML parser doesn't seem
> to mind (null), but is this reasonable semantically?
>

I think that YAML's "designated null" is literally `null`, which gets
converted into the appropriate language-specific nullish thing <
http://yaml.org/type/null.html> (and surprisingly not a string, which is
what you would expect... YAML's "you can usually omit the quotes" deal is
convenient, but things like this make it a big pain for writing robust
software). E.g. in Python:

>>> print(yaml.load('a: null'))
{'a': None}
>>> print(yaml.load('a: null_'))
{'a': 'null_'}

I'm not sure how/whether LLVM's YAML parser handles this. In the worst case
it would get turned into a string which is "no harm done".

That link though seems to suggest that the canonical YAML form for this is
just `~` (canonical YAML roughly means "fully marked up"/"desugared") and
also just empty is fine.

>>> print(yaml.load('a: ~'))
{'a': None}
>>> print(yaml.load('a: '))
{'a': None}

If our parser accepts either of these, then they might be simpler (I
especially like the idea of just leaving it empty). I'm not sure what
impact these choices would have on the testing though (e.g. expressing "the
field is empty" in FileCheck (maybe a regex like "foo:$" would work)).

Any of these 3 choices would be fine; I think you're in a better position
to decide as to which is preferable based on your experience working
with/testing the output.


>
>   Also, again semantically, you probably already noticed I'm not trying to
> flatten the full data structures for possible later resurrection, just
> providing what I think is sufficient high-level information to know what
> the preprocessor is up to.
>

Yeah, I wouldn't worry about going too "deep", as long as the tool serves
its purpose. What data structures in particular were you thinking could be
flattened? Being able to reconstruct full macro expansion trees from the
trace would be nifty, but definitely a project for another day :)

-- Sean Silva


>
>   Kim,
>
>   Thanks also for your help. I'll install ninja cmake and give it a shot.
>
>   -John
>
> Hi silvas, kimgr,
>
> http://llvm-reviews.chandlerc.com/D2020
>
> CHANGE SINCE LAST DIFF
>   http://llvm-reviews.chandlerc.com/D2020?vs=5196&id=5210#toc
>
> Files:
>   test/pp-trace/pp-trace-macro.cpp
>   pp-trace/PPTrace.cpp
>   pp-trace/PPCallbacksTracker.h
>   pp-trace/PPCallbacksTracker.cpp
>   pp-trace/CMakeLists.txt
>   pp-trace/Makefile
>   CMakeLists.txt
>   Makefile
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20131031/aa9f0b08/attachment.html>