[LLVMdev] lit: deprecating trailing \ in RUN lines

[Sean Silva]

On Tue, Dec 10, 2013 at 12:40 AM, Alp Toker <alp at nuanti.com> wrote:

>
> On 10/12/2013 04:20, Sean Silva wrote:
>
>> The classic way to do this sort of checking is by hacking into the tool
>> that actually interprets it (i.e. lit in this case). Considering that lit
>> is Python, it should be pretty easy to insert an ad-hoc regex check (or
>> even something substantially more sophisticated). E.g. insert code into
>> TestRunner.py's parseIntegratedTestScript function.
>>
>
> Hi Sean,
>
> Let's try to work on that then. It'll be brilliant if we can make this
> work without churn in the test suite.
>
> I had some progress trying to get that working by building a source line
> table and moving substitutions to later in the processing pipeline so all
> the source information to map changes back are actually available now.
>
> The difficulty is in getting the regular expression engine to use that to
> preserve source locations so we can apply the changes back to the original
> source.
>
> I've been digging into the depths of the Python regex implementation and
> tried inserting line break markers together with the multiline option but
> it didn't work out -- if can help find a way to do that then I think all
> the information we need is there to map back to the original lines and
> apply changes..


My guess is that lit doesn't really do that much on the strings, so you can
probably look at the couple operations that it does, and write a custom
string class that tracks internal state, such as variable evaluation
history, source location, etc. Then immediately upon reading a "raw" line
from the file, stuff it into an instance of the custom string class with
the appropriate source location, and
run the rest as usual.

I'm guessing you can probably get by with implementing:
- concatenation
- substring
- run a regex replace
- evaluate variables

Just implement these operations in a way that tracks the source location in
some internal state (probably very inefficiently, but who cares). Typically
I've seen this done with immutable objects, e.g. CustomString('foo',
line=1) + CustomString('bar', line=2) is a CustomString whose internal
state is "I am a concatenation of this string and that string", with object
references to its components. Or a CustomString after variables have been
evaluated has as its internal state
- a reference to the original CusomString
- a list of locations in that string where variables were substituted, and
with what.

Then just have an __str__ that reconstitutes the actual string value that
the CustomString represents (kind of like llvm::Twine). There will probably
be some places where it is easier to just hack lit's code than write a
truly drop-in str replacement.

Once this is in place, you can just hack in wherever you want in the code
and see the entire history of the string with all source locations
(HistoryString is probably a better name than CustomString).

-- Sean Silva


>
>
> Alp.
>
> --
> http://www.nuanti.com
> the browser experts
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131212/047e02e4/attachment.html>