[cfe-dev] examples of using clang's VFS layer

[Maurizio Vitale via cfe-dev]

Thanks,

  I did consider all these options (except for the first, because my
reasoning was that once you've transferred a file you can as well persist
it in a cache and I was thinking a disk backed cache; this should help for
include files that are probably reused. But a memory FS, or a combination
using unionFS, would also work).

I was more interested in what people have in mind when designing this
feature. I'm quite far from actually be ready to implement anything in this
area. For my application, this is very much tied to move my organization's
build system to Bazel and that's a hugely difficult move given the size and
convolution of our non-standard makefiles (we have our own "make" :-( ).
Bazel is also making strides in remote execution and I'm mainly trying to
get a good idea of where things were going.

The advantages of a FUSE file system is that you can use compilers
different from clang. In our case we'll have gcc and close source compilers
such as icpc (Intel) and nvcc (nVidia).
I know Google has build many FUSE file system and try to stay away as much
as possible. The FUSE file system can very well implemented using the same
service that is used by a modified Clang VFS, so code can be shared.
But I don't see how to do without at least a FUSE implementation. Of course
this is a bit outside clang discussion and have more to do w/ a build
system.

If you can say, which direction has google taken/working on? still using
srcFS/objFS or planning to replace them?


On Mon, Jul 18, 2016 at 6:06 AM, Benjamin Kramer <benny.kra at gmail.com>
wrote:

> There are currently very few implementations of the VFS around and
> virtually no docs, only some examples in the code base. For your
> problem I see a couple of options:
>
> - If you know what files will be used you can load them eagerly into
> an InMemoryFileSystem. There are some examples of doing this in Clang.
> - If you have a fuse implementation you can use RedirectingFileSystem
> (the YAML thing). Eventually I want to decouple this from YAML but
> it's not a high priority.
> - You can still implement your own VFS. This doesn't tie you to fuse
> and allows lazy loading. However it will require code changes when
> Clang changes.
>
> Combinations of the approaches are possible by using file system overlays.
>
> On Thu, Jun 30, 2016 at 9:49 PM, Maurizio Vitale via cfe-dev
> <cfe-dev at lists.llvm.org> wrote:
> > Is there any published paper/tech report/talk about how virtual file
> systems
> > are intended to be used in CLANG?
> >
> > As far as I understand it, presently there're a number of
> implementations in
> > the code base including one that gets a mapping virtual->physical from a
> > json file. Assuming one wanted to take unmodified files from some server
> > that caches the version control system, is the intention to point some of
> > the mappings to a fuse file system that handle caching/communication w/
> the
> > VCS?
> >
> > Or are people writing their own implementation of the abstract file
> system
> > that does whatever they need and merge it with any new clang release
> they're
> > interested in?
> >
> > Thanks,
> >
> >   Maurizio
> >
> > _______________________________________________
> > cfe-dev mailing list
> > cfe-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160718/7fc2d9b0/attachment.html>