Calibrating AlphaFold confidence scores for interface quality
Researchers calibrated confidence scores to estimate the probability that predicted complexes exceed a DockQ threshold.
AlphaFold confidence scores rank protein complex predictions and select model subsets. These scores distinguish prediction quality, but their numerical values do not represent the probability that a prediction satisfies a structural quality criterion.
This work calibrates commonly used confidence scores to estimate the probability that a predicted complex exceeds a chosen DockQ threshold. Post-hoc isotonic calibration improves agreement between predicted probabilities and observed DockQ success rates on antibody-antigen and general multimer datasets.
The calibration preserves the underlying score ordering. These calibrated probabilities also provide more accurate estimates of the fraction of selected models expected to pass a chosen structural quality threshold.
The mapping is fitted within a given prediction distribution and transfers imperfectly across prediction domains. However, reliable estimates are recovered from a small number of labeled target-domain samples.
The calibration is entirely post hoc and requires no AlphaFold model fine-tuning. It provides a straightforward and customizable way to convert existing model quality rankings into calibrated probabilities.
Key facts
- Post-hoc isotonic calibration improves agreement between predicted probabilities and DockQ success rates.
- Calibrated probabilities estimate the fraction of selected models expected to pass a structural quality threshold.
- Reliable estimates can be recovered from a small number of labeled target-domain samples.