[LLVMdev] Proposal: release MDNodes for source modules (LTO+debug info)

[Rafael Espíndola]

Taking a really quick at the gold code it looks like it tries to keep
8176 files open. I would suggest putting a breakpoint in
Descriptors::close_some_descriptor and checking why it is failing to
close the files.

On 15 November 2013 11:03, Stephen Checkoway <s at pahtak.org> wrote:
>
> On Nov 15, 2013, at 10:19 AM, Rafael Espíndola <rafael.espindola at gmail.com> wrote:
>
>> On 14 November 2013 18:01, Stephen Checkoway <s at pahtak.org> wrote:
>>>
>>> On Nov 14, 2013, at 8:19 AM, Rafael Espíndola <rafael.espindola at gmail.com> wrote:
>>>
>>>> But gold has at most 2 objects loaded at any time.
>>>
>>> Are you sure about that? I haven't looked into it but while building Chromium with LTO, I get:
>>>
>>> ../../third_party/gold/gold64: fatal error: out of file descriptors and couldn't close any
>>> clang: error: linker command failed with exit code 1 (use -v to see invocation)
>>> ninja: build stopped: subcommand failed.
>>> [secdev:~/chromium/src/out/Release] (master) s$ ulimit -Sn
>>> 10000
>>>
>>> Looking in /proc, it has 10013 open file descriptors. With only 2 objects loaded at a time, I'd expect many fewer file descriptors to be open. Maybe it only has 2 objects in memory at once but keeps all the file descriptors open?
>>
>> That is odd. I will debug it in a sec, but we have in the claim_file_hook:
>>
>>  if (code_gen) {
>>    if (lto_codegen_add_module(code_gen, M)) {
>>      (*message)(LDPL_ERROR, "Error linking module: %s",
>>                 lto_get_error_message());
>>      return LDPS_ERR;
>>    }
>>  }
>>
>>  lto_module_dispose(M);
>>
>> In fact, with current gold we call get_view, so the plugin uses the
>> same fd as gold. It might actually be a bug with gold trying to cache
>> too many open files.
>>
>> How are you trying to build it?
>
> The standard Chromium build system was modified to add -flto -Os to cflags (which I'm assuming in this case also gets passed to clang++) and -flto to ldflags in chromium/src/build/common.gypi and I think some of the build files for libraries that are built along with Chromium were modified to not use -flto because they didn't work. I'm not the one who did that work, unfortunately, so I can't say for sure exactly what all was modified.
>
> --
> Stephen Checkoway
>
>
>