Comparison of AI protein structure ensemble prediction tools

Multiple AI prediction tools for protein structural ensembles have recently been released, building on the much heralded advances from AlphaFold, large language models, and other machine-learning approaches. Here we report on a comparison of several tools (BioEmu, AFSample2, ESMFlow) using a small test set of proteins, including three which exhibit well-studied structural transitions. Overall, while the AI platforms generate structurally diverse ensembles with overlapping regions, each tool produces clearly distinct conformational distributions. Thus, it is impossible that all the tools generate ensembles of high biophysical quality, analogous to a Boltzmann distribution. Experimental structures are often, but not always, covered by the ensembles in dimensionally reduced spaces. In cases where point mutations are known experimentally to cause large structural shifts, the AI tools exhibit either small or negligible shifts. Although our current analysis cannot evaluate the absolute quality of an ensemble, and hence cannot identify a best-performing AI tool, the results suggest users pursuing downstream applications such as protein engineering or drug design should interpret these ensembles with caution.