[lldb-dev] ASTImporter Tutorial/TechTalk and RoundTable

[Gábor Márton via lldb-dev]

Hi,

Thank you guys for coming to the round table. I am trying to summarize what
we were talking about.

- Error handling: rollback mechanism
We agreed that this is something that would be incredibly problematic to
implement mostly because the AST is not designed to be trimmed. In normal
cases, the AST just grows, but nodes and interconnections between them are
never deleted.

- Replace `StructuralEquivalency` with ODRHash
  - Step 0: StructuralEquivalency should not diagnose when called from the
    Importer, diags should be coming from the importer.
We agreed that the class StructuralEquivalenceContext is used for multiple
things in the ASTImporter. These are 1) ODR violation discovery 2)
identifying the existing part of a redecl chain for a Decl. Actually, ODR
checking is based on tokens if we compare definitions, so structural eq is
not the best thing to use for that.
We have concerns whether we could use the ODRHash, but we agreed that it
would be worth to prototype it.

- ODRViolation handling
  - Class(Var)TemplateSpecializationDecl: Problem! We can't have more than
1 spec
    (https://reviews.llvm.org/D66999)
We agreed that this is fine to be conservative in case of specializations,
in these cases we will always use the existing definition and will report
back an error. (In case of simple classes and other non-specialization
decls ,however, the liberal strategy can be used.)
  - Renaming strategy in case of ODR violation
Raphael is worried about name mangling issues that could cause.

- Strategies for AccumulateChildErrors
  Clients of the ASTImporter should be able to choose an
  appropriate error handling strategy for their needs.  For instance,
  they may not want to mark an entire namespace as erroneous merely
  because there is an ODR error with two typedefs.  As another example,
  the client may allow EnumConstantDecls with same names but with
  different values in two distinct translation units.
We agreed that this is fine to use this approach, probably there is no need
to make this behavior configurable.

- VarTemplateSpecializationDecl: ODR violation is not even detected
We didn't have time to discuss this.

Smaller issues/tasks/FIXMEs (techn debt)
- VisitFunctionDecl:
  - Member function templates are not handled similarly to simple fun
    specializations
We agreed that this is a big technical debt
  - Merge function definitions of members
We discussed that out-of class default definitions can cause this
situation, and at some time we should finish this.
  - Merge exception specifications

- Handling of inheritable attributes (https://reviews.llvm.org/D68634)
  - Use PrevDecl in `GetImportedOrCreateDecl` ?
We agreed that handling of attributes cannot be done with one patch because
there are many different inheritable attributes and each must be handled
differently.

- ObjC/ObjC++ support and stabilization
  - No test cases (not interesting for E///, we don't have objc/c++ code)
We agreed that there is a technical debt here, but we did not show much
interest in fixing that.

- ClassTemplateSpecializationDecl: merge instantiated default arguments,
  exceptions specifications
We understood the problem (that took like 20 minutes and AST dumping) and
recognized that the way we merge initializers of fields is good, and we
should continue doing that with default arg and exceptions spec
instantiations.

We did not have time to discuss these other issues:
- Structural Eq:
  - Polluted cache of nonequivalent declarations
  - Some diagnostics are completely missing, this is misleading
- Several minor issues/fixmes with VarTemplateDecl
- Check visibility/linkage for ClassTemplateDecl, VarTemplateDecl
- Fix import of equivalent but repeated FriendDecls
- Handle redecl chain of TypeDefNameDecl
- Add Decls to their context in a unified way, and only if the "From"DC
  contains it (`AddDeclToContexts`)
- VisitVarDecl:
  - Check for ODR error if the two definitions have different initializers?
  - Diagnose ODR error if the two initializers are different
- Remove obsolate FIXMEs and TODOs
- Import default arguments of templates

Cheers,
Gabor

On Thu, Oct 17, 2019 at 4:11 PM Gábor Márton <martongabesz at gmail.com> wrote:

> Hi,
>
> At the upcoming LLVM Dev Conf, we will have a round table discussion for
> ASTImporter, right after the ASTImporter Tutorial.
> The time slot for the round table is Wednesday, Oct 23 2:55-4:00.
> I have gathered things about possible future work and improvements, bring
> your own topic to discuss!
>
> Thanks and see you at the conference,
> Gabor
>
> Big stuff
> - Error handling: rollback mechanism
> - Replace `StructuralEquivalency` with ODRHash
>   - Step 0: StructuralEquivalency should not diagnose when called from the
>     Importer, diags should be coming from the importer.
> - ODRViolation handling
>   - Class(Var)TemplateSpecializationDecl: Problem! We can't have more than
> 1 spec
>     (https://reviews.llvm.org/D66999)
>   - VarTemplateSpecializationDecl: ODR violation is not even detected
>   - Renaming strategy
> - Strategies for AccumulateChildErrors
>   Clients of the ASTImporter should be able to choose an
>   appropriate error handling strategy for their needs.  For instance,
>   they may not want to mark an entire namespace as erroneous merely
>   because there is an ODR error with two typedefs.  As another example,
>   the client may allow EnumConstantDecls with same names but with
>   different values in two distinct translation units.
>
> Smaller issues/tasks/FIXMEs (techn debt)
> - VisitFunctionDecl:
>   - Member function templates are not handled similarly to simple fun
>     specializations
>   - Merge function definitions of members
>   - Merge exception specifications
> - Handling of inheritable attributes (https://reviews.llvm.org/D68634)
>   - Use PrevDecl in `GetImportedOrCreateDecl` ?
> - ObjC/ObjC++ support and stabilization
>   - No test cases (not interesting for E///, we don't have objc/c++ code)
> - ClassTemplateSpecializationDecl: merge instantiated default arguments,
>   exceptions specifications
> - Structural Eq:
>   - Polluted cache of nonequivalent declarations
>   - Some diagnostics are completely missing, this is misleading
> - Several minor issues/fixmes with VarTemplateDecl
> - Check visibility/linkage for ClassTemplateDecl, VarTemplateDecl
> - Fix import of equivalent but repeated FriendDecls
> - Handle redecl chain of TypeDefNameDecl
> - Add Decls to their context in a unified way, and only if the "From"DC
>   contains it (`AddDeclToContexts`)
> - VisitVarDecl:
>   - Check for ODR error if the two definitions have different initializers?
>   - Diagnose ODR error if the two initializers are different
> - Remove obsolate FIXMEs and TODOs
> - Import default arguments of templates
>
> On Mon, Jul 22, 2019 at 6:28 PM Gábor Márton <martongabesz at gmail.com>
> wrote:
>
>> Hi,
>>
>> I am planning to submit a talk about ASTImporter to the upcoming Dev
>> Meeting at October 22-23, 2019.
>> I'd like to talk about
>> - the API,
>> - how it is used in CTU analysis and in LLDB,
>> - internal subtleties and difficulties (Error handling,
>> ExternalASTSource, ...)
>> The goal would be to attract more developers to use and improve
>> ASTImporter related code, so perhaps this will be a tutorial.
>>
>> Independently from the talk, I'd like to have a round table discussion if
>> there is enough interest.
>> Some topics could cover future development ideas and existing problems we
>> have.
>> Please get back to me if you are interested and think about the topics
>> you have in mind, also don't forget to buy your ticket to the DevMeeting.
>>
>> Thanks,
>> Gabor
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20191024/9eb1a804/attachment-0001.html>