John Canny

One protein language model trained across many families on whole multiple sequence alignments, with interleaved row and column attention, reportedly outperforms unsupervised structure-learning methods by a wide margin.

Across five protein tasks with evolution-aware splits, self-supervised pretraining helped nearly every model on every task, yet alignment-based features still beat learned embeddings on secondary structure and contact prediction.