A patch to move codegen includes into public include

[DeadMG]

Yes, I apologize for not being very specific. Since the patch directly
follows on from the cfe-dev discussion I kinda assumed that all involved
would have seen it. Let me take a step back here.

The project that I'm working on right now is an external language which
offers direct interfacing to C++. Here we mean things like, instantiating
templates, implementing methods, calling methods, inheriting from classes,
stuff like that. This really means needing the support of Clang at every
step in the process of handling the C++ code being interfaced with-
lexing/preprocessing, parsing, analysis, and codegen. For example, if my
user requests a field of a CXXRecordDecl, then I need to create IR that
corresponds to that field access. This is also a job that Clang absolutely
must do, so it seems clear to me that simply re-using Clang's existing code
is the smarter play.

I think that we can all agree that a hypothetical future API would be more
abstract than the existing lib/codegen. Effectively, I already have built a
fair chunk of this. I want to continue working on it and I want to produce
something that's useful not just for me, but for anyone with similar needs.
I know that I'm not the only one.

The real issue here is primarily the limitations of my own development
environment. Or to wit, how am I going to construct a new public API
without access to private details to produce an implementation? Some of you
may have seen on IRC how "hilariously" long it takes for me to rebuild
LLVM/Clang, and the need to build from source is a big problem when I want
to use the CI server I have access to, or if another developer wants to
contribute to my work. It's not that I necessarily think that the CodeGen
internals should be public for all time. It's just that if I want to work
in this space with my available environment, to build the API that it seems
like we all agree would be the superior option, then not having them
available is a fairly big problem for me. My CI server was wiped and I'm
looking at not bringing it back online because getting Clang into a point
where it could be used for development in this area was pretty difficult
the first time and the guy I'm bumming CI from changed his OS from the much
more friendly Ubuntu to Gentoo (I'm a Windows dev primarily).

We all seem to agree that we want a lib/ABI rather than a public
lib/CodeGen. It's just that the practicalities of actually *building it*
are seriously impeded for me by developing in-tree rather than out of it.

> Yes, but that's exactly what we're lacking here: a compelling use case
that outlines what API it actually needs in the public headers.

> I'm really not sure why the first step isn't to *design* a reasonable
public API. Throwing header files at the wall and seeing what sticks will
not result in an API we can maintain and support going forward.

It's a bit of a chicken-and-egg problem. I already have quite a bit of work
in this area- certainly not complete but a starting point. But the primary
issue of contributing incrementally is that that implies developing each
increment with still-private APIs- the exact thing I'm trying to *avoid *here
because it's a serious problem for me to keep working that way.
Conceptually, there's certainly no reason why I couldn't simply provide the
work I've done already as a start. The problem is what's gonna happen when
I want to fix bugs, add new features, or change the implementation. I'd be,
relatively speaking, right back to square one, because if I find a better
implementation based on a slightly different CodeGen API, then I'm back to
developing against private internals.

> Digging a bit, it seems like quite a bit got laid out by the OP in the
thread "[cfe-dev] Clang ABI library". However, there hasn't been a clear
connection drawn between the code changes in this patch and the use cases
in that thread.

The APIs in the headers changed in the patch are the ones I've used to
implement those features so far. For a simple example, it's CodeGenModule
that contains the API you need to convert a CXXMethodDecl* into an
llvm::Value* you can call/invoke, or to generate code into your own LLVM
module. It's CodeGenFunction that contains the API you need to define or
call C++ functions. And so on.

There are a few that I know will need changing. For example, Itanium ABI
states that RTTI descriptors should be emitted with the body of a given
method. Understandably, it makes no provision for what should occur if no
C++ compiler will ever see the body of that method because it was
implemented in an external language. I'd need to alter the mechanism for
RTTI emission to handle this case (assuming that an unrelated change didn't
allow a handle for it in 3.5). Another example is that CodeGenTypes doesn't
handle having multiple CodeGenTypes generating into the same module, so you
end up with more than one LLVM type for the same Clang type and the IR no
longer typechecks. But the only reason I even know that an internal change
will be needed is because I already tried with the existing API. But since
developing in-tree is pretty painful for me (as above) I'd rather keep
developing as much as humanly possible externally.

All I'm saying is, moving a few headers seems like a low price to pay for
enabling a much superior development process for the thing we all actually
seem to want in the end (I haven't seen anyone suggest that having a
lib/ABI or similar would be undesirable). It would be less of a problem if
I had a better environment or better hardware, but there's nothing I can do
about that at the current time.

Or alternatively I could just contribute what I've got and then let you
guys finish the job instead of doing it myself.

For the record, if you want to see what I've done so far against 3.4, you
can breeze through the existing tutorials on my website
<http://www.codepuppy.co.uk/>. Obviously you can decide for yourself
whether or not compiling the demonstrated code counts as a compelling use
case.


On 26 August 2014 03:46, Sean Silva <chisophugis at gmail.com> wrote:

>
>
>
> On Mon, Aug 25, 2014 at 7:27 PM, Chandler Carruth <chandlerc at google.com>
> wrote:
>
>>
>> On Mon, Aug 25, 2014 at 7:26 PM, Sean Silva <chisophugis at gmail.com>
>> wrote:
>>
>>> as we get compelling use cases, we bring up the necessary headers into
>>> include/ to satisfy them, making any necessary incremental changes to
>>> better serve the use case.
>>
>>
>> Yes, but that's exactly what we're lacking here: a compelling use case
>> that outlines what API it actually needs in the public headers.
>>
>
> Totally agreed.
>
> Digging a bit, it seems like quite a bit got laid out by the OP in the
> thread "[cfe-dev] Clang ABI library". However, there hasn't been a clear
> connection drawn between the code changes in this patch and the use cases
> in that thread.
>
> DeadMG, maybe you could start a new thread with a more incremental, more
> clearly motivated step attacking a limited subset of your use case, rather
> than your current approach of "These are just the headers I found myself
> to have used so far when grepping my codebase"?
>
> -- Sean Silva
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20140826/2caa06d5/attachment.html>