A. X. Lu

Feature Reuse and Scaling: Understanding Transfer Learning with Protein Language Models

This study analyzes transfer learning in protein language models, finding that downstream task performance often relies on early pretraining features rather than scaling with pretraining duration.

Protein generation with evolutionary diffusion: sequence is all you need

This paper introduces EvoDiff, a diffusion model that generates diverse, high-fidelity proteins in sequence space, overcoming limitations of structure-based models.

Convolutions are competitive with transformers for protein sequence pretraining

This study shows that convolutional neural networks can be as effective as transformers for protein sequence pretraining, offering better scalability and performance on long sequences.