Calibrating AlphaFold confidence scores for interface quality

Researchers calibrated confidence scores to estimate the probability that predicted complexes exceed a DockQ threshold.

AlphaFold confidence scores rank protein complex predictions and select model subsets. These scores distinguish prediction quality, but their numerical values do not represent the probability that a prediction satisfies a structural quality criterion.

This work calibrates commonly used confidence scores to estimate the probability that a predicted complex exceeds a chosen DockQ threshold. Post-hoc isotonic calibration improves agreement between predicted probabilities and observed DockQ success rates on antibody-antigen and general multimer datasets.

The calibration preserves the underlying score ordering. These calibrated probabilities also provide more accurate estimates of the fraction of selected models expected to pass a chosen structural quality threshold.

The mapping is fitted within a given prediction distribution and transfers imperfectly across prediction domains. However, reliable estimates are recovered from a small number of labeled target-domain samples.

The calibration is entirely post hoc and requires no AlphaFold model fine-tuning. It provides a straightforward and customizable way to convert existing model quality rankings into calibrated probabilities.

Key facts

  • Post-hoc isotonic calibration improves agreement between predicted probabilities and DockQ success rates.
  • Calibrated probabilities estimate the fraction of selected models expected to pass a structural quality threshold.
  • Reliable estimates can be recovered from a small number of labeled target-domain samples.

All sections

Edition