[LLVMdev] [llvm-commits] [PATCH] Allow per-thread re-direction of outs()/errs()

[Justin Holewinski]

On Sat, Jun 2, 2012 at 8:56 AM, Chris Lattner <clattner at apple.com> wrote:

> On Jun 1, 2012, at 11:13 PM, Justin Holewinski wrote:
>
> In a way, they are.  What if I want a debug trace in a multi-threaded
>> context?  Right now, all threads would just spew to the same stream and the
>> result would be unreadable.  If you allow threads to setup their own outs()
>> and errs() streams (transparently to the rest of LLVM), you can intercept
>> these as you see fit, perhaps dumping each to a separate file.  I
>> acknowledge that you could also enforce single-threaded execution for debug
>> runs, but what if you are in the context of a library with no valid
>> stdout/stderr buffers?
>>
>>
>> This doesn't make any sense.  There is very little sense in trying to use
>> DEBUG (which gets compiled out of optimized builds) in a multithreaded
>> context.  Polluting the purity of outs() and errs() because of this sort of
>> use case doesn't make any sense to me.
>>
>> outs() and errs() are useful independent of how LLVM happens to use them.
>>  The extensions that you are proposing doesn't make any sense give nthe
>> design of raw_ostream.
>>
>
> Is the problem more that outs() and errs() should always point to valid
> stdout/stderr streams, regardless of how the hosting compiler is
> implemented?
>
>
> No.  outs() and errs() are equivalent to stdout and stderr.  Your request
> to change how they work to be thread local is like saying that we should
> change how printf works because LLVM is misusing printf somewhere.
>

Fair enough.  I've always been under the impression that outs() and errs()
were simply convenience wrappers around *some* output mechanism, not
strictly just stdout/stderr.


>
> If so, how do you feel about introducing a new global stream, logs()?
>  This could replace all uses of outs()/errs() in the LLVM libraries, and
> default to errs().  Existing command-line tools could just use this
> default, but compilers that are hosted inside of libraries or other
> non-command line scenarios can attach a custom stream to logs() that
> captures all LLVM output, including DEBUG and any legitimate pass output.
>  Command-line tools can continue to use outs()/errs() just as they do now.
>  Nothing would really change in the way LLVM operates by default
>
>
> In fact, before we had raw_ostream, we had a dbgs() sort of stream, which
> was intended to be used in debug statements.  I'm not strongly opposed to
> this (it would be much better than violating the sanctity of outs() :), but
> I'm still unclear why you care so much about DEBUG output.  If we had a
> dbgs() again, it should *only* be used from within DEBUG macros (for
> example, the linker shouldn't use it).  Would that be enough to achieve
> your goal?
>

It's not so much DEBUG output that I care specifically about, that's just
an example of a place where the LLVM libraries assume an stderr stream
exists and is freely writable.  What I really want is a way to capture
*all* LLVM library output that would normally go to stderr, similar to how
Clang has its Diagnostic functionality that allows a driver to collect all
output messages.

There are legitimate cases where passes may output warnings or even errors
that the compiler should be able to direct to the user.  errs() is fine for
this if you know you're working with a command-line tool, but that's not
always true.

It sounds like dbgs() is not really the solution here, since I want to
capture more than just DEBUG output.  If the module linker wants to output
a warning, I want to capture that.  If an optimization pass wants to emit a
performance warning, I want to capture that too.  Neither of these cases
are DEBUG-only.

I see two possible solutions:

   1. For every pass (or class) that may emit warnings/errors, provide it a
   raw_ostream instance.  This seems to be how some of the utility classes
   work now, but this puts a burden on the compiler to know which passes
   actually need a raw_ostream parameter.  If you want to add a warning output
   to any pass, you need to change its interface (create***Pass and
   constructor) to add the raw_ostream parameter.
   2. Provide a generic mechanism through which passes and any other
   utility classes can write warning/error information.  This gives more
   control to the compiler without introducing any interface changes.

Option (1) is do-able, but option (2) seems more general.  The specifics of
the implementation that I envision are:

   - logs() is added as another stream to raw_ostream.h
   - By default, logs() forwards to errs()  [no change in the way LLVM
   works in the default case]
   - Any pass or utility class that wants to emit warnings/errors should
   write to logs() instead of errs()
   - DEBUG code should use logs() instead of errs()  [again, this will
   default to errs()]
   - An API is provided that allows a client to re-direct logs() to any
   arbitrary raw_ostream, on a per-thread level

You could also envision something along the lines of Clang's Diagnostics
functionality.  You could thread a diagnostics interface through the pass
manager, and passes could output any messages as diagnostics.  These
diagnostics would be collected and the compiler could query for them after
the pass manager completes.


>
> -Chris
>



-- 

Thanks,

Justin Holewinski
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20120602/7bc2256b/attachment.html>