How Well Can AI and Physics-Based Simulations Predict the Probability a Cryptic Pocket Is Open?

Identifying and understanding cryptic pockets remains a compelling goal in drug discovery, as they can offer new avenues for targeting proteins that are otherwise challenging to modulate. While artificial intelligence (AI) methods for structure prediction have been revolutionary, they have not learned enough physics to characterize the rest of a proteins conformational ensemble, such as structures with cryptic pockets. This limitation has motivated the development of new AI-based methods for characterizing ensembles, or properties thereof, such as AlphaFlow, BioEmu, PocketMiner, and CryptoBank. Here, we benchmark how well these models and physics-based molecular dynamics (MD) simulations can recapitulate the thermodynamics of known cryptic pockets in Ebola VP35 and TEM {beta}-lactamase. These two pockets were chosen because the probability that they are open and the impact of point mutations have been well characterized experimentally. Multiple methods are remarkably successful at predicting whether a mutation will increase or decrease the probability of cryptic pocket opening. However, none can reliably predict the absolute probability of pocket opening. MD is very close for the two wild-type proteins but all the methods struggle for pockets with small probabilities of opening experimentally (e.g. less than 1%). BioEmu and PocketMiner capture some trends between variants with experimental populations over 1% but have systematic errors and poorer performance for rare pockets (e.g. <1% open experimentally). These results highlight the promise of AI- and simulation-based strategies for cryptic pocket characterization while pointing to the need for further improvements to achieve robust predictors.