[llvm-dev] Email list just for front end developers?

[Sean Silva via llvm-dev]

On Thu, May 11, 2017 at 7:48 PM, Perry E. Metzger <perry at piermont.com>
wrote:

> So in writing this, I want to emphasize, in advance, that I'm grateful
> to the LLVM team for having created LLVM, I do not feel I am in any
> way entitled to anyone else's help, etc. The purpose of my original
> post was just to suggest that there be a way for people who are
> struggling with some parts of the APIs to find each other and answer
> each other's questions. If any of the following sounds like I'm
> demanding anything, that's not my intent. This may be longer than it
> should be, too.
>
> On Thu, 11 May 2017 15:51:14 -0700 Sean Silva <chisophugis at gmail.com>
> wrote:
> > I'm still unclear about what sorts of questions/discussion this
> > audience is interested in. Can you provide a few examples of
> > questions that might be asked, or links to discussions/blog posts
> > on the net that exemplify the type of thing that you would want to
> > happen in the "frontend authors using LLVM" medium you are thinking
> > of?
> >
> > For example, looking at the post you linked (
> > http://lists.llvm.org/pipermail/llvm-dev/2017-March/111526.html the
> > author of which is in this thread I think, hi!), there's not much
> > LLVM-specific about it. In order to generate LLVM IR, it is
> > necessary to understand its semantics. The semantics are mostly
> > C-like, and so the answer to questions like those in the post:
> >
> > - How to assign a text to a string variable?
> > - How to access for example the 4th element in a heap-allocated
> > array?
> > - How to create local variables which are not part of the global
> > scope?
> > - How to create complex structures like classes and objects?
> >
> > is basically "how would you do it in C"?
>
> I think you're missing the import of his questions. This isn't "how
> you would do it in C". He's asking effectively how to use the
> IRBuilder APIs to do these things, not what in principle you want the
> machine to do or what you might write into a file of text form IR if
> you were bypassing the IRBuilder.
>
> The "how do I construct a string constant in the API" one cost me
> hours to figure out until someone on IRC took pity on me, which was
> unusual, and steered me to the right call (CreateGlobalString).
>
> Now I know what you're thinking. "Why didn't you just look for
> something like CreateGlobalString to begin with?" And the answer is,
> because I had no idea what I was looking for, what form it would take,
> or what it was called, and googling at random wasn't helping. I knew
> what sort of IR output I wanted but I didn't know how to generate it
> in the API.
>
> And yes, a naive newcomer can literally spend hours trying to figure
> out that just one call they wanted was CreateGlobalString. Maybe I
> missed the part of the extant documentation or the tutorial that
> pointed there, but that's kind of the point, isn't it? If it was
> there, I didn't find it, and I needed to ask for help.
>

If you remember, can you explain what process you were using to try to find
out how to do it? Also, if you remember / have IRC logs, it would be useful
to see how you posed the question. I'm guessing that you didn't actually
phrase it like "does IRBuilder have a method that creates a global string
constant?" since if I go to
http://llvm.org/docs/doxygen/html/classllvm_1_1IRBuilder.html and search
for "string" that's basically the first result, and you would have found it
yourself in that case.

Was it unclear whether IRBuilder had a helper method like that and you
would instead have to manually create a global and an array constant etc.?
One thing to keep in mind is that IRBuilder is just a helper around
lower-level IR construction calls, so you can always use the lower-level
calls (e.g. look at the the implementation of
IRBuilder::CreateGlobalString:
http://llvm.org/docs/doxygen/html/IRBuilder_8cpp_source.html#l00027 it
isn't unreasonable to directly use those API's).

I'm trying to hone in on the details of when you say "I had no idea what I
was looking for, what form it would take,or what it was called". Surely you
had some starting point, and I'd like to figure out what was missing
between that and formulating the right question, because I think getting to
the question "does IRBuilder have a method that creates a global string
constant?" is really more important than its answer, since the answer is
easy to find.


>
> It would have been nice to have a place where I could have
> dropped the question and no one would have thought twice about just
> answering.
>
> And thus, my desire for a place for mutual aid for people who are
> figuring out front end stuff or who have already figured it out and
> are willing to help others.
>
> > For example, to assign text to a "string variable", you first have
> > to have a struct that represents you string type, then you do some
> > C-level code to manipulate that data structure.
>
> That's not what he's asking, or at least I don't think he is. He's
> asking about how to generate the right IR. He probably was looking for
> CreateGlobalString and how to put it together with other stuff.  His
> other messages made that seem clear to someone else who had been
> struggling recently with the same issues,.
>
> > For some of the specifics of LLVM IR's SSA form (like phi nodes)
> > working through the Kaleidoscope tutorial should give a feel for
> > how to do it.
>
> A feel isn't enough to write code that compiles and does the right
> stuff. It gives you a tantalizing hint of what you want, but you need
> more than that. The tutorial makes it all look really easy, but it
> skips even a lot of details about what it is doing, and then you start
> to ask questions that go slightly past the edge of the tutorial and
> the answers _really_ aren't obvious.
>
> Here's just one a of a million examples:


This example is really interesting. I'd love to hear more. I still don't
quite have a feel for exactly how to help. From this email, my general
feeling is that what is missing is not documentation about "how to do X"
(multiplied by a large number of specific things) but more general advice
about how to navigate LLVM and discover things for yourself (such as the
trick for feeding C code to Clang, then working backward to find what you
want). I.e. what's missing is more "teach a person to fish" than "give a
person a fish". Could you give a couple more examples?


> in a real compiler you don't
> want to just parse a number into a machine int or double or what have
> you, you want to use arbitrary precision stuff that's machine
> independent, and of course, LLVM has that because it needs it.
>
> Unluckily, even once you know it is there, it takes a while to figure
> out what you need to use it, and there's very little documentation at
> all. There was a point where I wanted to just punt and let the APFloat
> API parse a string from my front end's scanner (after all, why
> duplicate that code and less well?), only there's no documentation on
> what sorts of float formats it parses. If someone could have answered
> that, or if there had been a paragraph or two in the doxygen output,
> it would have saved me lots of time reading code. As it was, the best
> someone had to suggest to me was read the tests, so I went off and
> slogged through. (And yah, it turns out that it handles a variety of
> standard C-like formats including hex floats, which was something I
> wanted to figure out but couldn't just read in the doxygen docs
> because it isn't there.)
>

To be fair, I don't think that any LLVM developer would have very much more
of a clue (possibly not even the person who wrote it) without doing
essentially what you did (stare at the current code / tinker with it).

That being said, I'm obligated to ask: did you send a patch updating the
doxygen docs to have the information you were originally looking for?
For issues in our doxygen documentation, sending patches to llvm-commits
fixing them is 100% the right way to go.

-- Sean Silva



>
> So yah, you get a feel for what you want from the tutorial, but a feel
> isn't enough to write a compiler for something non-trivial. The
> programmer's guide helps a bit more, but you're quickly off the road
> and feeling stuff out.
>
> Note again that I'm not in any way suggesting that anyone should feel
> obligated to help others. No one is! I have no claim on anyone else's
> time, and again, thank all of you for creating LLVM in the first
> place. All I was asking for was just a place for those who are willing
> to help each other over the hump to contact each other. Puzzling
> things out by reading the test cases for a piece of compiler
> middle-end machinery isn't the most fun way to learn an API, and it is
> slow, too.
>
> > However, from your other posts, you say something like:
> > "People working on front ends typically don't really know everything
> > about the innards of LLVM, and thanks to LLVM's very nicely designed
> > architecture, we don't need to, we mostly need to know how to
> > generate IR and hook up to the rest of the system."
> >
> > So it sounds like you're more interested in API usage like
> > IRBuilder, setting up codegen, setting up MC, configuring pass
> > pipelines, etc.? I.e. you have a clear idea about the "big picture"
> > and what you are trying to do, but want to know how that is done in
> > LLVM (e.g. "how do I create a phi node?"). This is a very different
> > kind of question from the ones above.
>
> The questions that guy was asking seem to be part and parcel, though I
> could be wrong.
>
> There's a big IR Builder API, the C bindings for all this stuff are
> poorly documented, the C++ bindings are only slightly better
> documented, and dealing with the important bits is not trivial. The
> language reference superficially looks quite nice, but then you
> realize the only way to actually figure out what IR you need for many
> constructs is to write C code and get Clang to dump out the IR that
> results. (It took me a while to learn that trick, even though, yes, it
> should have been obvious, just like finding CreateGlobalString should
> have been obvious only I didn't know I was looking for it.)
>
> However, even once having seen what Clang generates, you still need to
> figure out what to put into IRBuilder to get the same results. Sure,
> that's probably somewhere in the innards of Clang, but try finding
> it. It's not trivial, and without much documentation it takes a while
> to figure out. I really would have like to have just asked someone
> about a bunch of stuff instead of deciphering things.
>
> Also take his second question about accessing parts of
> arrays. getelementptr is sufficiently weird that there actually _is_ a
> decent bit of documentation on it, but it isn't obvious that you're
> looking for that unless someone tells you. (Luckily I stumbled on it.)
>
> I get that this all seems really easy if you know it already. To
> someone who is coming at this completely naively, without much
> understanding of where to dig in, there's a very, very steep learning
> curve and little documentation once you're done with the tutorial. A
> lot has to be found in blog postings, bits of sample code you happen
> to find, reading the implementation, etc.
>
> And I note that even though I'm still at it, I don't yet have my front
> end doing very much. There's still a lot to learn. And yah, maybe I'm
> not the best at this. Maybe if I was smarter I'd just pick it up from
> reading Clang code or something, but I'm slow and middle aged and yet
> I have a job to get done anyway. So I keep beating my head against
> stuff until I figure it out. But I still wish there were people to ask
> questions.
>
> > I'd really like to hear your feedback. I think that historically,
> > Clang has been the dominant frontend and so there hasn't been much
> > impetus for providing certain kinds of documention, but as you
> > mention upthread, there is now a quite large tail of other
> > frontends that are developed outside of the LLVM community, so that
> > might have to change.
>
> If I had a magic wand, I'd have everything well documented, but that
> seems like it isn't going to happen. As a more practical second best,
> all I really want is to be in contact with other people who work on
> front ends who are sympathetic and remember learning the same thing.
>
> Perry
> --
> Perry E. Metzger                perry at piermont.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170512/cb7d362d/attachment.html>