A new benchmark tests how well models design DNA
BEND 2.0 measures if models can predict DNA changes and guide sequence design.
Researchers introduced BEND 2.0, which stands for BENchmark for DNA language models. This tool tests how well models predict the effects of DNA changes. It also tests if these predictions can guide sequence design.
The benchmark uses eight tasks built on five experimental datasets. It looks at how model choice and how representations are accessed affect performance. Functional post-training substantially improves model capabilities.
Linear readouts are fitted to frozen representations. These can match or exceed native outputs. Linear readouts transfer well across experimental assays and genes. This suggests models capture functional patterns beyond the data used to fit individual predictors.
Strong prediction of average effects can conceal weaker prediction of context-specific effects. Accurate prediction of combined mutation effects does not imply accurate prediction of their interactions. Model predictions can reduce the number of candidates needed to find a desired response.
There is still substantial room for better design choices. This is true for context-specific and multi-cell objectives. Genomic models display signs of general utility but have yet to saturate on performance in even simple tasks.
Key facts
- BEND 2.0 uses eight tasks and five experimental datasets.
- Functional post-training improves model capabilities.
- Linear readouts can match or exceed native outputs.
- Models have yet to saturate on performance in even simple tasks.