[all-commits] [llvm/llvm-project] 030b8c: [BPF] Attribute preserve_static_offset for structs

eddyz87 via All-commits all-commits at lists.llvm.org
Tue Dec 5 09:25:16 PST 2023 

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 030b8cb1561db4161b108b59044717d89026cf70
      https://github.com/llvm/llvm-project/commit/030b8cb1561db4161b108b59044717d89026cf70
  Author: Eduard Zingerman <eddyz87 at gmail.com>
  Date:   2023-12-05 (Tue, 05 Dec 2023)

  Changed paths:
    M clang/include/clang/Basic/Attr.td
    M clang/include/clang/Basic/AttrDocs.td
    M clang/lib/CodeGen/CGExpr.cpp
    M clang/lib/Sema/SemaDeclAttr.cpp
    A clang/test/CodeGen/bpf-preserve-static-offset-arr.c
    A clang/test/CodeGen/bpf-preserve-static-offset-bitfield.c
    A clang/test/CodeGen/bpf-preserve-static-offset-lvalue.c
    A clang/test/CodeGen/bpf-preserve-static-offset-non-bpf.c
    A clang/test/CodeGen/bpf-preserve-static-offset-pai.c
    M clang/test/Misc/pragma-attribute-supported-attributes-list.test
    A clang/test/Sema/bpf-attr-preserve-static-offset-warns-nonbpf.c
    A clang/test/Sema/bpf-attr-preserve-static-offset-warns.c
    A clang/test/Sema/bpf-attr-preserve-static-offset.c
    M llvm/include/llvm/IR/Intrinsics.td
    M llvm/include/llvm/IR/IntrinsicsBPF.td
    M llvm/lib/Target/BPF/BPF.h
    M llvm/lib/Target/BPF/BPFAbstractMemberAccess.cpp
    M llvm/lib/Target/BPF/BPFCORE.h
    M llvm/lib/Target/BPF/BPFCheckAndAdjustIR.cpp
    A llvm/lib/Target/BPF/BPFPreserveStaticOffset.cpp
    M llvm/lib/Target/BPF/BPFTargetMachine.cpp
    M llvm/lib/Target/BPF/CMakeLists.txt
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-align.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-arr-pai.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-atomic.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-chain-2.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-chain-oob.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-chain-u8-oob.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-chain-u8-type-mismatch.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-chain-u8.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-chain.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-inline.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-non-const.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-ptr-pai.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-simple.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-struct-pai.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-undo-align.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-undo-chain-oob.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-undo-chain-u8.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-undo-chain.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-undo-simple.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-undo-volatile.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-union-pai.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-unroll-inline.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-unroll.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-volatile.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/load-zero.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-align.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-atomic.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-chain-2.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-chain-oob.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-chain-u8-oob.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-chain-u8.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-chain.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-pai.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-simple.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-undo-align.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-undo-chain-oob.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-undo-chain-u8.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-undo-chain.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-undo-simple.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-undo-volatile.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-unroll-inline.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-volatile.ll
    A llvm/test/CodeGen/BPF/preserve-static-offset/store-zero.ll

  Log Message:
  -----------
  [BPF] Attribute preserve_static_offset for structs

This commit adds a new BPF specific structure attribte
`__attribute__((preserve_static_offset))` and a pass to deal with it.

This attribute may be attached to a struct or union declaration, where
it notifies the compiler that this structure is a "context" structure.
The following limitations apply to context structures:
- runtime environment might patch access to the fields of this type by
  updating the field offset;

  BPF verifier limits access patterns allowed for certain data
  types. E.g. `struct __sk_buff` and `struct bpf_sock_ops`. For these
  types only `LD/ST <reg> <static-offset>` memory loads and stores are
  allowed.

  This is so because offsets of the fields of these structures do not
  match real offsets in the running kernel. During BPF program
  load/verification loads and stores to the fields of these types are
  rewritten so that offsets match real offsets. For this rewrite to
  happen static offsets have to be encoded in the instructions.

  See `kernel/bpf/verifier.c:convert_ctx_access` function in the Linux
  kernel source tree for details.

- runtime environment might disallow access to the field of the type
  through modified pointers.

  During BPF program verification a tag `PTR_TO_CTX` is tracked for
  register values. In case if register with such tag is modified BPF
  programs are not allowed to read or write memory using register. See
  kernel/bpf/verifier.c:check_mem_access function in the Linux kernel
  source tree for details.

Access to the structure fields is translated to IR as a sequence:
- `(load (getelementptr %ptr %offset))` or
- `(store (getelementptr %ptr %offset))`

During instruction selection phase such sequences are translated as a
single load instruction with embedded offset, e.g. `LDW %ptr, %offset`,
which matches access pattern necessary for the restricted
set of types described above (when `%offset` is static).

Multiple optimizer passes might separate these instructions, this
includes:
- SimplifyCFGPass (sinking)
- InstCombine (sinking)
- GVN (hoisting)

The `preserve_static_offset` attribute marks structures for which the
following transformations happen:
- at the early IR processing stage:
  - `(load (getelementptr ...))` replaced by call to intrinsic
    `llvm.bpf.getelementptr.and.load`;
  - `(store (getelementptr ...))` replaced by call to intrinsic
    `llvm.bpf.getelementptr.and.store`;
- at the late IR processing stage this modification is undone.

Such handling prevents various optimizer passes from generating
sequences of instructions that would be rejected by BPF verifier.

The __attribute__((preserve_static_offset)) has a priority over
__attribute__((preserve_access_index)). When preserve_access_index
attribute is present preserve access index transformations are not
applied.

This addresses the issue reported by the following thread:

https://lore.kernel.org/bpf/CAA-VZPmxh8o8EBcJ=m-DH4ytcxDFmo0JKsm1p1gf40kS0CE3NQ@mail.gmail.com/T/#m4b9ce2ce73b34f34172328f975235fc6f19841b6

This is a second attempt to commit this change, previous reverted
commit is: cb13e9286b6d4e384b5d4203e853d44e2eff0f0f.
The following items had been fixed:
- test case bpf-preserve-static-offset-bitfield.c now uses
  `-triple bpfel` to avoid different codegen for little/big endian
  targets.
- BPFPreserveStaticOffset.cpp:removePAICalls() modified to avoid
  use after free for `WorkList` elements `V`.

Differential Revision: https://reviews.llvm.org/D133361

More information about the All-commits mailing list