Real Science Is Harder Than Benchmarks: Evaluating Advanced AI Frameworks on Published Studies. I. Uncertainty Quantification, ML on Therapeutic Data Commons, and Agent-Based Modeling
Mohammed O. Ahmed, Sahil Anil Amale, Rhythm Dipak Bhavsar, Pratham Chopra, Amos Jaimes, Arpita Kachhwah, Casual Dineshbhai Kalotra, Peizhao Li, Xingbei Li, Yuantian Liao, Rahul Roy, Nivethini Senthilselvan, Yukun Shao, Alok Dinesh Sharma, Arjun Shrivatsan, , Anton V. Sinitskiy
This paper evaluates five AI frameworks on real scientific tasks, finding they excel in hypothesis generation and reporting but struggle with depth and reliability in real-world research.
bioRxiv · Jun 27, 2026 · PreprintBioinformatics