Multi-Model LLM Architectures for Personalized Summarization and Relevance Ranking in Biomedical Literature

Abstract

ObjectiveTo develop and evaluate a personalized literature review system that efficiently processes and summarizes biomedical literature to provide timely, relevant insights for researchers. MethodsThe system integrates ontology-aware keyword extraction (MeSH/ACM constrained TF-IDF from CV/Research Statement), citation-informed retrieval (PubMed and NIH iCite API), and dual-model large language model (LLM) summarization (Google Gemini 2.0 flash, OpenAI GPT-4o-mini). These LLMs leverage advanced Transformer architectures, building on foundations such as BERT, BART, and BioBERT. A two-stage ranking algorithm combines Relative Citation Ratio (RCR) with cosine similarity. Summary quality was evaluated using ROGUE-1/2/L and BERTScore. The system is deployed as a Streamlit web application. ResultsAcross 20 biomedical queries, the system demonstrated strong average performance (BERT-F1{approx} 0.86), with cosine similarity strongly correlating with summary quality. Human evaluation involving 10 users yielded average scores above 4.5/5 across summary fidelity and keyword relevance. ConclusionHybrid ranking and ensemble LLM summarization significantly accelerate scientific sense-making. These findings suggest broad applicability to various domains beyond biomedicine.