[llvm-commits] Unbork Sphinx docs build

[Sean Silva]

I can't tell you how awesome this is. Iteration time is now at least
an order of magnitude faster!

In the immortal words of Fred Brooks, an order of magnitude
quantitative difference entails a qualitative difference.

--Sean Silva

On Wed, Sep 5, 2012 at 3:46 PM, Michael Spencer <bigcheesegs at gmail.com> wrote:
> On Tue, Sep 4, 2012 at 5:59 PM, Sean Silva <silvas at purdue.edu> wrote:
>> Michael, could you take a look at this and commit if everything looks OK?
>>
>> --Sean Silva
>
> Committed as r163235
>
> - Michael Spencer
>
>>
>> On Tue, Sep 4, 2012 at 4:42 PM, Bill Wendling <isanbard at gmail.com> wrote:
>>> Um. sure go for it.
>>>
>>> -bw
>>>
>>> On Sep 4, 2012, at 10:07 AM, Sean Silva <silvas at purdue.edu> wrote:
>>>
>>>> Ping^2
>>>>
>>>> On Tue, Aug 28, 2012 at 12:21 PM, Sean Silva <silvas at purdue.edu> wrote:
>>>>> Ping.
>>>>>
>>>>> On Tue, Aug 21, 2012 at 12:04 AM, Sean Silva <silvas at purdue.edu> wrote:
>>>>>> Good catch, it looks like the sphinx option I used to copy them is
>>>>>> putting the HTML files under _static/ which breaks the links. I must
>>>>>> have had stale HTML files sitting in _build/html (I thought I did a
>>>>>> `make clean` beforehand...I guess not).
>>>>>>
>>>>>> I've attached a new patch that remedies the situation by using a hack
>>>>>> in the Makefile and just `cp`'ing the HTML files into the build
>>>>>> directory.
>>>>>>
>>>>>> I couldn't make the corresponding change to `make.bat` due to being
>>>>>> unfamiliar with .bat files. Michael, do you know enough .bat-fu to
>>>>>> fixup `make.bat`? I don't think that `make.bat` is a patch-blocker
>>>>>> anyway, since typically a local Sphinx build is done while writing
>>>>>> documentation, which should be all in Sphinx now anyway (and plus, you
>>>>>> can directly open the .html files in the browser if you really need to
>>>>>> (like, say, the as-yet unconverted LangRef, which gets updates from
>>>>>> time-to-time)). Well, unless the docs for llvm.org are generated on
>>>>>> Windows, which I doubt.
>>>>>>
>>>>>> So basically copy `llvm-theme/` into a new directory _themes/ (as I
>>>>>> described in the OP) and apply the new patch `unbork-sphinx.patch`.
>>>>>> Does that work for you Michael? I double checked and everything is
>>>>>> working here. Locally, tutorial/ is not showing nicely (it just shows
>>>>>> the "file browser" of the local directory) since there isn't a
>>>>>> webserver running to automatically serve index.html; I verified that
>>>>>> this also happens on master, so it is not a regression.
>>>>>>
>>>>>> --Sean Silva
>>>>>>
>>>>>> On Mon, Aug 20, 2012 at 7:01 PM, Michael Spencer <bigcheesegs at gmail.com> wrote:
>>>>>>> On Sun, Aug 19, 2012 at 3:12 PM, Sean Silva <silvas at purdue.edu> wrote:
>>>>>>>> Working on LLVM's Sphinx documentation, I've been noticing that it is
>>>>>>>> abysmally slow to build. In particular, despite Sphinx having
>>>>>>>> capabilities of doing incremental builds, it is rebuilding all of the
>>>>>>>> files every time. I was able to track down the culprit and fix the
>>>>>>>> problem.
>>>>>>>>
>>>>>>>> The problem is that the LLVM Sphinx theme (llvm-theme/) is in the root
>>>>>>>> Sphinx directory. This causes Sphinx add the root docs/ directory to
>>>>>>>> an internal path list which is used to determine whether any of the
>>>>>>>> templates are out of date. When building one of the .rst files,
>>>>>>>> Sphinx's logic considers it out of date if *any* of the templates are
>>>>>>>> newer than it. In order to check if any of the templates are newer
>>>>>>>> than it, it takes the most recent modification time of not just the
>>>>>>>> template files, but any files with a .html extension recursively
>>>>>>>> contained in the above-mentioned path. So basically, since _build is
>>>>>>>> under the root sphinx directory, the generated HTML files in
>>>>>>>> _build/html were being considered.
>>>>>>>>
>>>>>>>> The fix is simple enough, just move llvm-theme into a new _themes/
>>>>>>>> directory, and update `html_theme_path` to be "_themes" instead of
>>>>>>>> ".". Unfortunately, this has a bit of fallout, since apparently the
>>>>>>>> way we are currently handling the legacy HTML-formatted docs with
>>>>>>>> `html_additional_pages`, which ends up treating them like templates,
>>>>>>>> which causes Sphinx to error when handling _themes/llvm-theme since
>>>>>>>> the paths it is being fed are relative to the root Sphinx directory.
>>>>>>>> The fix for this is to add the root directory to the
>>>>>>>> `html_static_path` variable, which causes Sphinx to copy the legacy
>>>>>>>> HTML-formatted documentation pages into the build directory through a
>>>>>>>> different and better mechanism that just copies the files, rather than
>>>>>>>> running them through the slow Python templating mechanism that Sphinx
>>>>>>>> uses (which is a noop anyway since those HTML files don't contain
>>>>>>>> jinja tags).
>>>>>>>>
>>>>>>>> tl;dr
>>>>>>>> How to unbork it:
>>>>>>>>
>>>>>>>> Because the llvm-theme directory contains some binary files, it is not
>>>>>>>> very patch friendly, so please just do
>>>>>>>>
>>>>>>>> cd docs/
>>>>>>>> mkdir _themes
>>>>>>>> git mv llvm-theme _themes
>>>>>>>>
>>>>>>>> and then apply the patch 01-conf.py.patch
>>>>>>>>
>>>>>>>> --Sean Silva
>>>>>>>
>>>>>>> This breaks the link to the tutorial.
>>>>>>>
>>>>>>> - Michael Spencer
>>>