r278882 - If possible, set the stack rlimit to at least 8MiB on cc1 startup, and work

[Joerg Sonnenberger via cfe-commits]

On Sat, Aug 20, 2016 at 12:45:59AM +0000, Chandler Carruth wrote:
> I feel like this thread has two very unrelated concerns and it might be
> useful to separate them.
> 
> 1) Should an application be changing the stack limit, or should the system
> be able to enforce this?
> 
> Fortunately, there are two limits. One an application is allowed to change,
> one it isn't. It seems like the mechanism has been provided for system
> administrators to impose firm limits and neither Clang nor another
> application will perturb them. I don't see any reason to preclude an
> application from changing the one it is explicitly permitted to change.
> That seems to be WAI from a permissions and ACL model.

The only application processes that normally change process limits are
the shells or certain network processes that change the file limit
according to the configured number of connections. The soft limit exists
to allow a user to adjust his/her own limits within bounds. That's why
the first category has appropiate syntax for it. The second category on
the other hand knows pretty precisely what it wants to consume in terms
of ressources and plans accordingly. Clang falls into neither category.
The existing bugs are even evidence of that, e.g. 15560 says 8MB is not
enough either. So this is a patchy band aid at most, it doesn't even solve
the problem.

> 2) Should Clang be using the stack to allocate significant portions of its
> memory, or should it be assiduously keeping allocations on the heap and
> removing recursive algorithms?
> 
> I actually think this is the crux of the issue, and #1 is largely coming up
> as a proxy for "Clang should use less stack space".

I fully agree that there would be no request for fiddling with the stack
limit from within clang without clang blowing up the stack.

> However, I'll point out that this commit does not change how much stack
> space Clang uses. It just changes the nature of errors that occur to make
> the experience of today's Clang more friendly for users. So this commit
> seems like a fine thing even if we decide that Clang should work to lower
> this number over time and reduce what it sets it to. In fact, if you want
> to reduce Clang's stack usage, you almost certainly want it to explicitly
> set a limit so that we consistently hit errors when bugs creep into Clang.
> 
> Past that, while I think Clang's current allocation and recursion pattern
> is fine, I don't actually have a terribly strong opinion. But it seems odd
> to suddenly want to make a dramatic change in the design of Clang now... It
> doesn't seem like this was an accidental or unconsidered design decision.

Correct, this commit does nothing about the reducing the stack use by
clang. But let's look at one of the few bug reports that actually seem
to contain some numbers: 20635. This shows 3200 stack frames consuing
the 2MB default stack on 32bit FreeBSD. Similar defaults limits can be
found on most 32bit platforms, 64bit platforms typically have twice that
limit. This means the average stack frame is 600 Bytes for something
that is *intended* to recurse deeply. It seems pretty obvious to me that
this is the real problem and something that should be fixed. The LLVM
are much more conversative here. Many of the deep recursion places
either use very little stack space or follow the more typical work queue
approach. Setting a higher stack limit is I said earlier is just a
bandaid to paper over implementation issues.

Setting an explicit limit in the test suite would be fine by me. The
only problem in this area is that the kind of limits you want to set can
be difficult to do consistently across platforms, i.e. I would certainly
welcome it if no test case is supposed to require more than 1GB of VA
and 1MB/2MB of stack. That's a slightly different question though.

Joerg