Study identifies gaps in quantum software testing fault representation

Researchers found that existing mutation tools and benchmarks fail to represent 86.3% of real-world quantum software bugs.

Quantum software uses compilers, intermediate representations, and execution frameworks to transform programs. Faults in these components can change the behavior of correct quantum programs.

An investigation audited four quantum mutation-testing tools and three curated bug benchmarks. Researchers also collected 410 verified bugs from ten quantum software repositories over two years.

The study classified faults into a taxonomy of 56 bug families. Only 56 of the 410 real-world bugs belong to families represented by mutation tools or curated benchmarks.

The remaining 354 bugs belong to families absent from both sources. The largest unrepresented families involve compiler transformations with 66 bugs, type handling with 53, and input validation with 47.

Existing mutation operators primarily modify circuits and program constructs. Many real-world faults arise in compilers, representations, or framework interfaces that depend on intermediate representations or dependency versions.

Addressing this gap requires testing semantic preservation across compiler transformations and correctness at representation boundaries. This includes looking at interactions between software components.

Key facts

  • 86.3% of real-world bugs belong to families absent from mutation tools and curated benchmarks.
  • The largest unrepresented bug families concern compiler transformations, type handling, and input validation.
  • Mutation tools and curated benchmarks together represent only 13.7% of the 410 verified real-world bugs.

All sections

Edition