[llvm-commits] Unbork Sphinx docs build

[Michael Spencer]

On Tue, Sep 4, 2012 at 5:59 PM, Sean Silva <silvas at purdue.edu> wrote:
> Michael, could you take a look at this and commit if everything looks OK?
>
> --Sean Silva

Committed as r163235

- Michael Spencer

>
> On Tue, Sep 4, 2012 at 4:42 PM, Bill Wendling <isanbard at gmail.com> wrote:
>> Um. sure go for it.
>>
>> -bw
>>
>> On Sep 4, 2012, at 10:07 AM, Sean Silva <silvas at purdue.edu> wrote:
>>
>>> Ping^2
>>>
>>> On Tue, Aug 28, 2012 at 12:21 PM, Sean Silva <silvas at purdue.edu> wrote:
>>>> Ping.
>>>>
>>>> On Tue, Aug 21, 2012 at 12:04 AM, Sean Silva <silvas at purdue.edu> wrote:
>>>>> Good catch, it looks like the sphinx option I used to copy them is
>>>>> putting the HTML files under _static/ which breaks the links. I must
>>>>> have had stale HTML files sitting in _build/html (I thought I did a
>>>>> `make clean` beforehand...I guess not).
>>>>>
>>>>> I've attached a new patch that remedies the situation by using a hack
>>>>> in the Makefile and just `cp`'ing the HTML files into the build
>>>>> directory.
>>>>>
>>>>> I couldn't make the corresponding change to `make.bat` due to being
>>>>> unfamiliar with .bat files. Michael, do you know enough .bat-fu to
>>>>> fixup `make.bat`? I don't think that `make.bat` is a patch-blocker
>>>>> anyway, since typically a local Sphinx build is done while writing
>>>>> documentation, which should be all in Sphinx now anyway (and plus, you
>>>>> can directly open the .html files in the browser if you really need to
>>>>> (like, say, the as-yet unconverted LangRef, which gets updates from
>>>>> time-to-time)). Well, unless the docs for llvm.org are generated on
>>>>> Windows, which I doubt.
>>>>>
>>>>> So basically copy `llvm-theme/` into a new directory _themes/ (as I
>>>>> described in the OP) and apply the new patch `unbork-sphinx.patch`.
>>>>> Does that work for you Michael? I double checked and everything is
>>>>> working here. Locally, tutorial/ is not showing nicely (it just shows
>>>>> the "file browser" of the local directory) since there isn't a
>>>>> webserver running to automatically serve index.html; I verified that
>>>>> this also happens on master, so it is not a regression.
>>>>>
>>>>> --Sean Silva
>>>>>
>>>>> On Mon, Aug 20, 2012 at 7:01 PM, Michael Spencer <bigcheesegs at gmail.com> wrote:
>>>>>> On Sun, Aug 19, 2012 at 3:12 PM, Sean Silva <silvas at purdue.edu> wrote:
>>>>>>> Working on LLVM's Sphinx documentation, I've been noticing that it is
>>>>>>> abysmally slow to build. In particular, despite Sphinx having
>>>>>>> capabilities of doing incremental builds, it is rebuilding all of the
>>>>>>> files every time. I was able to track down the culprit and fix the
>>>>>>> problem.
>>>>>>>
>>>>>>> The problem is that the LLVM Sphinx theme (llvm-theme/) is in the root
>>>>>>> Sphinx directory. This causes Sphinx add the root docs/ directory to
>>>>>>> an internal path list which is used to determine whether any of the
>>>>>>> templates are out of date. When building one of the .rst files,
>>>>>>> Sphinx's logic considers it out of date if *any* of the templates are
>>>>>>> newer than it. In order to check if any of the templates are newer
>>>>>>> than it, it takes the most recent modification time of not just the
>>>>>>> template files, but any files with a .html extension recursively
>>>>>>> contained in the above-mentioned path. So basically, since _build is
>>>>>>> under the root sphinx directory, the generated HTML files in
>>>>>>> _build/html were being considered.
>>>>>>>
>>>>>>> The fix is simple enough, just move llvm-theme into a new _themes/
>>>>>>> directory, and update `html_theme_path` to be "_themes" instead of
>>>>>>> ".". Unfortunately, this has a bit of fallout, since apparently the
>>>>>>> way we are currently handling the legacy HTML-formatted docs with
>>>>>>> `html_additional_pages`, which ends up treating them like templates,
>>>>>>> which causes Sphinx to error when handling _themes/llvm-theme since
>>>>>>> the paths it is being fed are relative to the root Sphinx directory.
>>>>>>> The fix for this is to add the root directory to the
>>>>>>> `html_static_path` variable, which causes Sphinx to copy the legacy
>>>>>>> HTML-formatted documentation pages into the build directory through a
>>>>>>> different and better mechanism that just copies the files, rather than
>>>>>>> running them through the slow Python templating mechanism that Sphinx
>>>>>>> uses (which is a noop anyway since those HTML files don't contain
>>>>>>> jinja tags).
>>>>>>>
>>>>>>> tl;dr
>>>>>>> How to unbork it:
>>>>>>>
>>>>>>> Because the llvm-theme directory contains some binary files, it is not
>>>>>>> very patch friendly, so please just do
>>>>>>>
>>>>>>> cd docs/
>>>>>>> mkdir _themes
>>>>>>> git mv llvm-theme _themes
>>>>>>>
>>>>>>> and then apply the patch 01-conf.py.patch
>>>>>>>
>>>>>>> --Sean Silva
>>>>>>
>>>>>> This breaks the link to the tutorial.
>>>>>>
>>>>>> - Michael Spencer
>>