[all-commits] [llvm/llvm-project] 34f7e0: Move RDF from Hexagon to Codegen

Wed Jun 24 09:35:56 PDT 2020

  Branch: refs/heads/release/10.x
  Home:   https://github.com/llvm/llvm-project
  Commit: 34f7e00333ac26c0b2b7f34b5988f2fec7d99f03
      https://github.com/llvm/llvm-project/commit/34f7e00333ac26c0b2b7f34b5988f2fec7d99f03
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    A llvm/include/llvm/CodeGen/RDFGraph.h
    A llvm/include/llvm/CodeGen/RDFLiveness.h
    A llvm/include/llvm/CodeGen/RDFRegisters.h
    M llvm/lib/CodeGen/CMakeLists.txt
    A llvm/lib/CodeGen/RDFGraph.cpp
    A llvm/lib/CodeGen/RDFLiveness.cpp
    A llvm/lib/CodeGen/RDFRegisters.cpp
    M llvm/lib/Target/Hexagon/CMakeLists.txt
    M llvm/lib/Target/Hexagon/HexagonOptAddrMode.cpp
    M llvm/lib/Target/Hexagon/HexagonRDFOpt.cpp
    M llvm/lib/Target/Hexagon/RDFCopy.cpp
    M llvm/lib/Target/Hexagon/RDFCopy.h
    M llvm/lib/Target/Hexagon/RDFDeadCode.cpp
    M llvm/lib/Target/Hexagon/RDFDeadCode.h
    R llvm/lib/Target/Hexagon/RDFGraph.cpp
    R llvm/lib/Target/Hexagon/RDFGraph.h
    R llvm/lib/Target/Hexagon/RDFLiveness.cpp
    R llvm/lib/Target/Hexagon/RDFLiveness.h
    R llvm/lib/Target/Hexagon/RDFRegisters.cpp
    R llvm/lib/Target/Hexagon/RDFRegisters.h

  Log Message:
  -----------
  Move RDF from Hexagon to Codegen

RDF is designed to be target agnostic. Therefore it would be useful to have it available for other targets, such as X86.

Based on a previous patch by Krzysztof Parzyszek

Differential Revision: https://reviews.llvm.org/D75932

  Commit: fe73b9416dea5b309a6846a2dfb2d81bcb514c64
      https://github.com/llvm/llvm-project/commit/fe73b9416dea5b309a6846a2dfb2d81bcb514c64
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M llvm/lib/Target/X86/CMakeLists.txt
    M llvm/lib/Target/X86/X86.h
    M llvm/lib/Target/X86/X86FastISel.cpp
    M llvm/lib/Target/X86/X86FrameLowering.cpp
    M llvm/lib/Target/X86/X86ISelDAGToDAG.cpp
    M llvm/lib/Target/X86/X86ISelLowering.cpp
    M llvm/lib/Target/X86/X86ISelLowering.h
    A llvm/lib/Target/X86/X86IndirectThunks.cpp
    M llvm/lib/Target/X86/X86InstrCompiler.td
    M llvm/lib/Target/X86/X86InstrControl.td
    M llvm/lib/Target/X86/X86InstrInfo.td
    M llvm/lib/Target/X86/X86MCInstLower.cpp
    R llvm/lib/Target/X86/X86RetpolineThunks.cpp
    M llvm/lib/Target/X86/X86Subtarget.h
    M llvm/lib/Target/X86/X86TargetMachine.cpp
    M llvm/test/CodeGen/X86/O0-pipeline.ll
    M llvm/test/CodeGen/X86/O3-pipeline.ll

  Log Message:
  -----------
  [X86][NFC] Generalize the naming of "Retpoline Thunks" and related code to "Indirect Thunks"

There are applications for indirect call/branch thunks other than retpoline for Spectre v2, e.g.,

https://software.intel.com/security-software-guidance/software-guidance/load-value-injection

Therefore it makes sense to refactor X86RetpolineThunks as a more general capability.

Differential Revision: https://reviews.llvm.org/D76810

  Commit: a09ebfdafb9dbfc8aa4ca493389ff9da956eabc1
      https://github.com/llvm/llvm-project/commit/a09ebfdafb9dbfc8aa4ca493389ff9da956eabc1
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M llvm/lib/Target/X86/X86IndirectThunks.cpp

  Log Message:
  -----------
  [X86] Refactor X86IndirectThunks.cpp to Accommodate Mitigations other than Retpoline

Introduce a ThunkInserter CRTP base class from which new thunk types can inherit, e.g., thunks to mitigate https://software.intel.com/security-software-guidance/software-guidance/load-value-injection.

Differential Revision: https://reviews.llvm.org/D76811

  Commit: 071acfdd4694bb2b94efe6122128c5e7f840ce46
      https://github.com/llvm/llvm-project/commit/071acfdd4694bb2b94efe6122128c5e7f840ce46
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M clang/docs/ClangCommandLineReference.rst
    M clang/include/clang/Driver/Options.td
    M clang/lib/Driver/ToolChains/Arch/X86.cpp
    M clang/test/Driver/x86-target-features.c
    M llvm/lib/Target/X86/X86.td
    M llvm/lib/Target/X86/X86ISelLowering.cpp
    M llvm/lib/Target/X86/X86IndirectThunks.cpp
    M llvm/lib/Target/X86/X86Subtarget.h
    A llvm/test/CodeGen/X86/lvi-hardening-indirectbr.ll

  Log Message:
  -----------
  [X86] Add Indirect Thunk Support to X86 to mitigate Load Value Injection (LVI)

This pass replaces each indirect call/jump with a direct call to a thunk that looks like:

lfence
jmpq *%r11

This ensures that if the value in register %r11 was loaded from memory, then
the value in %r11 is (architecturally) correct prior to the jump.
Also adds a new target feature to X86: +lvi-cfi
("cfi" meaning control-flow integrity)
The feature can be added via clang CLI using -mlvi-cfi.

This is an alternate implementation to https://reviews.llvm.org/D75934 That merges the thunk insertion functionality with the existing X86 retpoline code.

Differential Revision: https://reviews.llvm.org/D76812

  Commit: 6a4589599d74cae8c4ac7b0ff7ae14aeeb2f988b
      https://github.com/llvm/llvm-project/commit/6a4589599d74cae8c4ac7b0ff7ae14aeeb2f988b
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M llvm/lib/Target/X86/CMakeLists.txt
    M llvm/lib/Target/X86/X86.h
    A llvm/lib/Target/X86/X86LoadValueInjectionRetHardening.cpp
    M llvm/lib/Target/X86/X86TargetMachine.cpp
    M llvm/test/CodeGen/X86/O0-pipeline.ll
    M llvm/test/CodeGen/X86/O3-pipeline.ll
    A llvm/test/CodeGen/X86/lvi-hardening-ret.ll

  Log Message:
  -----------
  [X86] Add RET-hardening Support to mitigate Load Value Injection (LVI)

Adding a pass that replaces every ret instruction with the sequence:

pop <scratch-reg>
lfence
jmp *<scratch-reg>

where <scratch-reg> is some available scratch register, according to the
calling convention of the function being mitigated.

Differential Revision: https://reviews.llvm.org/D75935

  Commit: e3ba468fc3c123880cfd03dfbc9d1ed61d5904c6
      https://github.com/llvm/llvm-project/commit/e3ba468fc3c123880cfd03dfbc9d1ed61d5904c6
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M clang/include/clang/Driver/Options.td
    M clang/lib/Driver/ToolChains/Arch/X86.cpp
    M clang/test/Driver/x86-target-features.c
    M llvm/lib/Target/X86/CMakeLists.txt
    A llvm/lib/Target/X86/ImmutableGraph.h
    M llvm/lib/Target/X86/X86.h
    M llvm/lib/Target/X86/X86.td
    A llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp
    M llvm/lib/Target/X86/X86Subtarget.h
    M llvm/lib/Target/X86/X86TargetMachine.cpp
    M llvm/test/CodeGen/X86/O0-pipeline.ll
    M llvm/test/CodeGen/X86/O3-pipeline.ll
    A llvm/test/CodeGen/X86/lvi-hardening-gadget-graph.ll

  Log Message:
  -----------
  [X86] Add a Pass that builds a Condensed CFG for Load Value Injection (LVI) Gadgets

Adds a new data structure, ImmutableGraph, and uses RDF to find LVI gadgets and add them to a MachineGadgetGraph.

More specifically, a new X86 machine pass finds Load Value Injection (LVI) gadgets consisting of a load from memory (i.e., SOURCE), and any operation that may transmit the value loaded from memory over a covert channel, or use the value loaded from memory to determine a branch/call target (i.e., SINK).

Also adds a new target feature to X86: +lvi-load-hardening

The feature can be added via the clang CLI using -mlvi-hardening.

Differential Revision: https://reviews.llvm.org/D75936

  Commit: 2530f4e0ce44b5641ad00019dc946dba59adcf77
      https://github.com/llvm/llvm-project/commit/2530f4e0ce44b5641ad00019dc946dba59adcf77
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp
    A llvm/test/CodeGen/X86/lvi-hardening-loads.ll

  Log Message:
  -----------
  [X86] Add Support for Load Hardening to Mitigate Load Value Injection (LVI)

After finding all such gadgets in a given function, the pass minimally inserts
LFENCE instructions in such a manner that the following property is satisfied:
for all SOURCE+SINK pairs, all paths in the CFG from SOURCE to SINK contain at
least one LFENCE instruction. The algorithm that implements this minimal
insertion is influenced by an academic paper that minimally inserts memory
fences for high-performance concurrent programs:

http://www.cs.ucr.edu/~lesani/companion/oopsla15/OOPSLA15.pdf

The algorithm implemented in this pass is as follows:

1. Build a condensed CFG (i.e., a GadgetGraph) consisting only of the following components:
  -SOURCE instructions (also includes function arguments)
  -SINK instructions
  -Basic block entry points
  -Basic block terminators
  -LFENCE instructions
2. Analyze the GadgetGraph to determine which SOURCE+SINK pairs (i.e., gadgets) are already mitigated by existing LFENCEs. If all gadgets have been mitigated, go to step 6.
3. Use a heuristic or plugin to approximate minimal LFENCE insertion.
4. Insert one LFENCE along each CFG edge that was cut in step 3.
5. Go to step 2.
6. If any LFENCEs were inserted, return true from runOnFunction() to tell LLVM that the function was modified.

By default, the heuristic used in Step 3 is a greedy heuristic that avoids
inserting LFENCEs into loops unless absolutely necessary. There is also a
CLI option to load a plugin that can provide even better optimization,
inserting fewer fences, while still mitigating all of the LVI gadgets.
The plugin can be found here: https://github.com/intel/lvi-llvm-optimization-plugin,
and a description of the pass's behavior with the plugin can be found here:
https://software.intel.com/security-software-guidance/insights/optimized-mitigation-approach-load-value-injection.

Differential Revision: https://reviews.llvm.org/D75937

  Commit: 5bc4d476f344e39d80df1c6df01279523adb83ef
      https://github.com/llvm/llvm-project/commit/5bc4d476f344e39d80df1c6df01279523adb83ef
  Author: Craig Topper <craig.topper at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp
    R llvm/test/CodeGen/X86/lvi-hardening-loads.ll

  Log Message:
  -----------
  Revert "[X86] Add Support for Load Hardening to Mitigate Load Value Injection (LVI)"

This reverts commit 62c42e29ba43c9d79cd5bd2084b641fbff6a96d5

Reverting to address coding standard issues raised in post-commit
review.

  Commit: d758416a9152a9c7e41671493ae4f77a0dfc1993
      https://github.com/llvm/llvm-project/commit/d758416a9152a9c7e41671493ae4f77a0dfc1993
  Author: Craig Topper <craig.topper at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M clang/include/clang/Driver/Options.td
    M clang/lib/Driver/ToolChains/Arch/X86.cpp
    M clang/test/Driver/x86-target-features.c
    M llvm/lib/Target/X86/CMakeLists.txt
    R llvm/lib/Target/X86/ImmutableGraph.h
    M llvm/lib/Target/X86/X86.h
    M llvm/lib/Target/X86/X86.td
    R llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp
    M llvm/lib/Target/X86/X86Subtarget.h
    M llvm/lib/Target/X86/X86TargetMachine.cpp
    M llvm/test/CodeGen/X86/O0-pipeline.ll
    M llvm/test/CodeGen/X86/O3-pipeline.ll
    R llvm/test/CodeGen/X86/lvi-hardening-gadget-graph.ll

  Log Message:
  -----------
  Revert "[X86] Add a Pass that builds a Condensed CFG for Load Value Injection (LVI) Gadgets"

This reverts commit c74dd640fd740c6928f66a39c7c15a014af3f66f.

Reverting to address coding standard issues raised in post-commit
review.

  Commit: ac40e7a02efbadd5c91d2fec85aac29ab9a6af18
      https://github.com/llvm/llvm-project/commit/ac40e7a02efbadd5c91d2fec85aac29ab9a6af18
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M llvm/lib/Target/X86/X86LoadValueInjectionRetHardening.cpp

  Log Message:
  -----------
  [X86] Fix to X86LoadValueInjectionRetHardeningPass for possible segfault

`MBB.back()` could segfault if `MBB.empty()`. Fixed by checking for `MBB.empty()` in the loop.

Differential Revision: https://reviews.llvm.org/D77584

  Commit: 1d8639336d961f6d04142e935be5da16f24e4503
      https://github.com/llvm/llvm-project/commit/1d8639336d961f6d04142e935be5da16f24e4503
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M clang/test/Driver/x86-target-features.c

  Log Message:
  -----------
  [X86] Add tests to clang Driver to ensure that SLH/Retpoline features are not enabled with LVI-CFI

Differential Revision: https://reviews.llvm.org/D77427

  Commit: 363720c2b0f60b512c5f5b8bfbf010f86d078152
      https://github.com/llvm/llvm-project/commit/363720c2b0f60b512c5f5b8bfbf010f86d078152
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M clang/include/clang/Driver/Options.td
    M clang/lib/Driver/ToolChains/Arch/X86.cpp
    M clang/test/Driver/x86-target-features.c
    M llvm/lib/Target/X86/CMakeLists.txt
    A llvm/lib/Target/X86/ImmutableGraph.h
    M llvm/lib/Target/X86/X86.h
    M llvm/lib/Target/X86/X86.td
    A llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp
    M llvm/lib/Target/X86/X86Subtarget.h
    M llvm/lib/Target/X86/X86TargetMachine.cpp
    M llvm/test/CodeGen/X86/O0-pipeline.ll
    M llvm/test/CodeGen/X86/O3-pipeline.ll
    A llvm/test/CodeGen/X86/lvi-hardening-gadget-graph.ll

  Log Message:
  -----------
  [X86] Add a Pass that builds a Condensed CFG for Load Value Injection (LVI) Gadgets

Adds a new data structure, ImmutableGraph, and uses RDF to find LVI gadgets and add them to a MachineGadgetGraph.

More specifically, a new X86 machine pass finds Load Value Injection (LVI) gadgets consisting of a load from memory (i.e., SOURCE), and any operation that may transmit the value loaded from memory over a covert channel, or use the value loaded from memory to determine a branch/call target (i.e., SINK).

Also adds a new target feature to X86: +lvi-load-hardening

The feature can be added via the clang CLI using -mlvi-hardening.

Differential Revision: https://reviews.llvm.org/D75936

  Commit: 8aa8abae349dc1607884c24ca3b685d4c7d597d1
      https://github.com/llvm/llvm-project/commit/8aa8abae349dc1607884c24ca3b685d4c7d597d1
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp
    A llvm/test/CodeGen/X86/lvi-hardening-loads.ll

  Log Message:
  -----------
  [X86] Add Support for Load Hardening to Mitigate Load Value Injection (LVI)

After finding all such gadgets in a given function, the pass minimally inserts
LFENCE instructions in such a manner that the following property is satisfied:
for all SOURCE+SINK pairs, all paths in the CFG from SOURCE to SINK contain at
least one LFENCE instruction. The algorithm that implements this minimal
insertion is influenced by an academic paper that minimally inserts memory
fences for high-performance concurrent programs:

http://www.cs.ucr.edu/~lesani/companion/oopsla15/OOPSLA15.pdf

The algorithm implemented in this pass is as follows:

1. Build a condensed CFG (i.e., a GadgetGraph) consisting only of the following components:
  -SOURCE instructions (also includes function arguments)
  -SINK instructions
  -Basic block entry points
  -Basic block terminators
  -LFENCE instructions
2. Analyze the GadgetGraph to determine which SOURCE+SINK pairs (i.e., gadgets) are already mitigated by existing LFENCEs. If all gadgets have been mitigated, go to step 6.
3. Use a heuristic or plugin to approximate minimal LFENCE insertion.
4. Insert one LFENCE along each CFG edge that was cut in step 3.
5. Go to step 2.
6. If any LFENCEs were inserted, return true from runOnFunction() to tell LLVM that the function was modified.

By default, the heuristic used in Step 3 is a greedy heuristic that avoids
inserting LFENCEs into loops unless absolutely necessary. There is also a
CLI option to load a plugin that can provide even better optimization,
inserting fewer fences, while still mitigating all of the LVI gadgets.
The plugin can be found here: https://github.com/intel/lvi-llvm-optimization-plugin,
and a description of the pass's behavior with the plugin can be found here:
https://software.intel.com/security-software-guidance/insights/optimized-mitigation-approach-load-value-injection.

Differential Revision: https://reviews.llvm.org/D75937

  Commit: 72bff7855d8ce42b831922a51763f9a0732bd473
      https://github.com/llvm/llvm-project/commit/72bff7855d8ce42b831922a51763f9a0732bd473
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M llvm/lib/Target/X86/X86.h
    M llvm/lib/Target/X86/X86LoadValueInjectionLoadHardening.cpp
    M llvm/lib/Target/X86/X86TargetMachine.cpp
    M llvm/test/CodeGen/X86/O0-pipeline.ll
    M llvm/test/CodeGen/X86/lvi-hardening-loads.ll

  Log Message:
  -----------
  [X86] Add an Unoptimized Load Value Injection (LVI) Load Hardening Pass

@nikic raised an issue on D75936 that the added complexity to the O0 pipeline was causing noticeable slowdowns for `-O0` builds. This patch addresses the issue by adding a pass with equal security properties, but without any optimizations (and more importantly, without the need for expensive analysis dependencies).

Reviewers: nikic, craig.topper, mattdr

Reviewed By: craig.topper, mattdr

Differential Revision: https://reviews.llvm.org/D80964

  Commit: d24d5c8e308e689dcd83cbafd2e8bd32aa845a15
      https://github.com/llvm/llvm-project/commit/d24d5c8e308e689dcd83cbafd2e8bd32aa845a15
  Author: Scott Constable <scott.d.constable at intel.com>
  Date:   2020-06-24 (Wed, 24 Jun 2020)

  Changed paths:
    M llvm/test/CodeGen/X86/lvi-hardening-loads.ll

  Log Message:
  -----------
  fixed broken test after cherry pick

Compare: https://github.com/llvm/llvm-project/compare/f9da0a7f36f7...d24d5c8e308e