[PATCH] D82612: [clangd] Config: compile Fragment -> CompiledFragment -> Config

Tue Jun 30 07:34:10 PDT 2020

sammccall marked 8 inline comments as done.
sammccall added inline comments.

================
Comment at: clang-tools-extra/clangd/CMakeLists.txt:41
+  ConfigCompile.cpp
   ConfigYAML.cpp
   Diagnostics.cpp
----------------
hokein wrote:
> sammccall wrote:
> > hokein wrote:
> > > we have a few config-related files now, I wonder would it make sense to create a config/ subdir?
> > It seems marginal to me - could go either way. We have a small handful (2 headers and 3 impl files). There may be 1-2 more, but I'm not sure yet. (config-files-on-disk, and json->fragment).
> > So 6-7 files... smaller than the other subdirs, but not tiny.
> > 
> > There's also the fact that config.h/cpp is not like the others: all features (not just infra) depend on this, it's not in the `config::` namespace etc. So should it really be in the subdir?
> > 
> > Mind if we hold off on this until at least the scope of D82335 has landed? I think it would be awkward to do this move halfway through this patchset, but fairly easy to do at any point that things are stable.
> > it seems marginal to me - could go either way. We have a small handful (2 headers and 3 impl files). There may be 1-2 more, but I'm not sure yet. (config-files-on-disk, and json->fragment).
> > So 6-7 files... smaller than the other subdirs, but not tiny.
> 
> if we end up with 6-7 files, I'd prefer to create a subdirectory, which make our source structure a bit tidy, our clangd/ dir already has a lot of source files.
> 
> 
> 
> > Mind if we hold off on this until at least the scope of D82335 has landed? I think it would be awkward to do this move halfway through this patchset, but fairly easy to do at any point that things are stable.
> 
> sure, I didn't mean we should do this in this patch.
> if we end up with 6-7 files, I'd prefer to create a subdirectory

SGTM. I'll carry on with this structure for now, and move files after - please remind me if I forget!

================
Comment at: clang-tools-extra/clangd/ConfigCompile.cpp:65
+    if (F.HasUnrecognizedCondition)
+      Conditions.push_back([&](const Params &) { return false; });
+
----------------
hokein wrote:
> I think if this case happened, we might just bail out directly -- adding the regex conditions seems unnecessary, we never execute them on Line 127 (but we might still need computing the regex for diagnostics).
> 
> and I'm not sure the motivation of using `vector` for `Conditions` and `Apply`, it looks like Apply is just 1 element (if we do the bailout for conditions, `Conditions` will be at most 1 element).
> 
> I feel like using a single unique_function for `Conditions` and `Apply` might be a better fit with `Fragment`. There is a comment in `ConfigFragment.h`:
> 
> >  Each separate condition must match (combined with AND).
> >  When one condition has multiple values, any may match (combined with OR).
>  
> so my reading is that:
> 
> - `Fragment` and `CompiledFragment` is a 1:1 mapping
> - A `Fragment::Condition` represents a single condition, so the AND match applies for different `Fragment`s, which is in ConfigProvider?
> - A single condition can have multiple values (`PathMatch`), we use OR match
> 
> Is that correct?
> I think if this case happened, we might just bail out directly -- adding the regex conditions seems unnecessary, we never execute them on Line 127 (but we might still need computing the regex for diagnostics).
>
> and I'm not sure the motivation of using vector for Conditions and Apply, it looks like Apply is just 1 element (if we do the bailout for conditions, Conditions will be at most 1 element).

So the explanation for both of these things is that this file is going to grow a lot, and in particular these sections handling PathMatch etc are going to occur again and again, so we need a scalable pattern. I've deliberately limited the scope of the first patchset to only support one condition, one setting to apply etc, but you have to imagine there are likely to be 5 conditions and maybe 25 settings.

So with that in mind:
 - yes, actually saving the conditions isn't necessary, but we do need to evaluate them for side-effects (diagnostics) and it's simpler not to keep track of them. Optimizing performance of broken configs is not a big win :-)
 - it's marginally simpler to have only one Apply function, but much simpler if separate parts of compile() can push settings independently. Same goes for condition in fact...

> Fragment and CompiledFragment is a 1:1 mapping

Yes. I think this is somewhere where the naming is helpful at least :-)

> A Fragment::Condition represents a single condition, so the AND match applies for different Fragments, which is in ConfigProvider?

The AND match applies to the multiple Conditions in a single CompiledFragment. Each fragment only considers its own conditions. 
So given:
```
If:
  Platform: win
  PathMatch: [foo1/.*, foo2/.*]
CompileFlags:
  Add: -foo
---
CompileFlags:
  Add: -bar
```

We end up with... one Provider, suppling two CompiledFragments (each compiled from a separate Fragment).

The first CompiledFragment has two Conditions, conceptually:
 - `Params.Platform == "win"`
 - `Regex("foo1/.*").matches(Params.File) || Regex("foo2/.*").matches(Params.File)`

The second CompiledFragment has no Conditions. Each fragment has one Apply function.

The CompiledFragments are applied independently. They each consider their own Conditions, ANDing them together.

So CompiledFragment #1 checks both conditions. If any fails, it bails out. Otherwise it runs its Apply function, adding `-foo`.

Then CompiledFragment #2 runs. It has no conditions, so never bails out. It adds `-bar`.

================
Comment at: clang-tools-extra/clangd/ConfigCompile.cpp:84
+  void compile(Fragment::CompileFlagsBlock &&F) {
+    if (!F.Add.empty()) {
+      std::vector<std::string> Add;
----------------
hokein wrote:
> nit: use early return.
Early return isn't a pattern that's compatible with lots of optional pieces - it doesn't scale as this file grows.

================
Comment at: clang-tools-extra/clangd/ConfigProvider.h:47
+  /// This always produces a usable compiled fragment (errors are recovered).
+  explicit CompiledFragment(Fragment, DiagnosticCallback);
+
----------------
hokein wrote:
> sammccall wrote:
> > hokein wrote:
> > > Would it be more nature to have a function `compile`(or so) to do the actual compile (Fragment -> CompiledFragment) rather than doing it in a constructor?
> > I'm not sure. The idea that the input/result is Fragment/CompiledFragment, and that the compilation cannot fail, suggests to me that a constructor is OK here.
> > On the other hand, the DiangosticCallback param (which we don't keep a reference to) is a bit weird.
> > 
> > So I don't really feel strongly about it one way or the other, happy to change it if you do think it would be significantly better.
> You point is fair. 
> I'm not sure using `compile` is *significantly* better, but it is better IMO -- conceptually, the class name `CompiledFragment` implies that there should be a `compile` API (this was my impression when reading the code at the first time)
> 
> it might be ok to keep as-is.
Argh, you convinced me this would be better, but I can't come up with a form I like.

- `static CompiledFragment::compile` repeats itself in a weird way. But any other option requires messing around with friend functions, which feels off
- `Fragment::compile` is maybe the most natural, but the dependencies are backwards (needs the CompiledFragment type, it gets implemented in the wrong file...).
- standalone `compile()` is maybe the most palatable, but seems a little undiscoverable
- having a function return CompiledFragment means that putting it into a shared_ptr is annoying work, and breaks our pointer-logging trick, but having the factory function return a pointer also seems odd.

I'm starting to think maybe the CompiledFragment class shouldn't be public at all, and we should just `using CompiledFragment = unique_function<bool(const Params&, Config&) const>`. WDYT?

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D82612/new/

https://reviews.llvm.org/D82612