[llvm-bugs] [Bug 49489] New: lld with thinlto cache hits vm.max_map_count limit when linking many inputs

[via llvm-bugs]

https://bugs.llvm.org/show_bug.cgi?id=49489

            Bug ID: 49489
           Summary: lld with thinlto cache hits vm.max_map_count limit
                    when linking many inputs
           Product: lld
           Version: unspecified
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: enhancement
          Priority: P
         Component: All Bugs
          Assignee: unassignedbugs at nondot.org
          Reporter: hans at chromium.org
                CC: joker.eph at gmail.com, llvm-bugs at lists.llvm.org,
                    peter at pcc.me.uk, rnk at google.com,
                    smithp352 at googlemail.com, tejohnson at google.com

We hit this in Chromium when linking a binary with ca 33,000 inputs using
ThinLTO with the ThinLTO cache enabled.

Here is a stand-alone reproducer:

(Create 35k input files of non-trivial size)
$ for i in `seq 1 35000` ; do perl -e "print qq(void *f$i(void) { return \") .
'x' x (20*1024) . '\";}'" > $i.c ; done

(Create the main file)
$ perl -e 'print "int main() {"; print "extern void *f$_(); f$_();" for
1..35000; print "}"' > main.c

(Compile to bitcode)
$ parallel clang -flto=thin -c -- *.c


Linking without the thinlto cache works:
$ clang -fuse-ld=lld *.o

But when using the cache it fails:
$ clang -fuse-ld=lld -Wl,--thinlto-cache-dir=cache *.o
terminate called after throwing an instance of 'std::system_error'
  what():  Resource temporarily unavailable

Or when trying again, a different error message:
$ clang -fuse-ld=lld -Wl,--thinlto-cache-dir=cache *.o
LLVM ERROR: out of memory
Allocation failed



What's happening is that LLD is hitting the maximum number of virtual memory
mappings limit, which on Linux is fairly low by default, causing mmap,
mprotect, etc. to fail with ENOMEM.

$ sysctl vm.max_map_count
vm.max_map_count = 65530

When using the thinlto cache, in addition to mapping each bitcode input file
lld will also map each generated native object file (both for cache hits and
misses), essentially doubling the number of mapped files.

One way to alleviate this problem would be to increase the threshold at which
llvm uses mmap to read files. Currently that's 16KB (see shouldUseMmap() in
MemoryBuffer.cpp). But that could have downsides too, and would only buy a bit
more time before hitting the limit again.


A more interesting idea would be if LLD could unmap each input bitcode file
once it's done processing it. ThinLTO experts, do you think that would be
possible?

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20210309/e8272b8e/attachment.html>