A new compiler for managing data layouts in hardware accelerators

Arknife uses a constraint-based layout algebra to handle instruction requirements across different hardware targets.

Researchers developed Arknife, a retargetable compiler. It treats instruction-visible layout as first-class instruction semantics.

Modern hardware accelerators use specialized instructions and managed memory hierarchies. These instructions have strict layout requirements for operands.

Instruction-visible layout determines if instructions execute correctly and how much data movement they incur. A tensor layout maps logical axes to hardware axes.

Arknife uses a constraint-based layout algebra. Instruction operands declare layout requirements, and target-provided repairs resolve conflicts. Layouts are solved globally across the program.

Target-side instruction descriptions supply ISA-specific layout facts. This allows the compiler to reuse the layout algebra and solver when retargeting.

Arknife matches cuBLAS on NVIDIA A100. On x86 CPU and Hexagon NPU, it outperforms vendor-optimized libraries by 2.17×2.17\times and 3.78×3.78\times.

Key facts

  • Arknife matches cuBLAS performance on the NVIDIA A100.
  • Arknife outperforms vendor-optimized libraries by 2.17×2.17\times on x86 CPU.
  • Arknife outperforms vendor-optimized libraries by 3.78×3.78\times on Hexagon NPU.

All sections

Edition