Structural-confidence metrics for the Adaptyv Bio EGFR round 2 designs

About this validation

In September 2026 we ranked 380 third-party protein designs from the Adaptyv Bio EGFR round 2 competition. Their binding had already been measured in a wet laboratory: these are designs we did not make, and results we did not produce. We used the measured outcomes only to check our own ranking, after the fact.

Two things came out of it. Our top twenty held about three times as many true binders as chance would give. And on this target, our ranking agreed strongly with the field-standard AlphaFold2 pipeline on how the same designs should be ordered, at a rank correlation of 0.73. That agreement is one target and one run, and it should not be read as a constant.

No score predicts binding outright, ours included, which is why we never report an affinity. This exercise tested how we rank designs, not what we design. Our own design work is a separate benchmark, still to come.

A metric that did not work

We also tested whether a purely geometric measure — how much of the intended binding site a design actually touches — ranks designs better than the structural-confidence scores above. On one summary statistic it looked better. It did not survive resampling, and at the top of the ranked list, which is the only part anyone acts on, it performed no better than the scores we already use.

We report it because we would have reported the opposite. That measure turned out to be useful for a different question — whether a design goes where it was aimed — and is published as a separate benchmark ↗.

Licence

These values are our additions to the Adaptyv Bio EGFR round 2 database and are released here to satisfy clause 4.6(b) of the Open Database License.

Contains information from the Adaptyv Bio EGFR round 2 dataset, which is made available under the Open Database License (ODbL) v1.0: opendatacommons.org/licenses/odbl. This file is released under the same licence.

Data (380 designs)

Download CSVREADME