transformer / foundation model

Probing EEG Foundation Model Attention Maps for Lightweight Interpretable Emotion Recognition

This paper proposes a lightweight, interpretable method for emotion recognition using EEG foundation models by analyzing attention maps, achieving competitive performance with fewer computations.

Hierarchical Temporal Transformer for Cancer Grade Prediction and Cross-Cancer Transfer Learning from Pathology Reports

This paper introduces a hierarchical temporal transformer for cancer grade prediction from pathology reports, showing improved performance across cancer types and using visit timing when identifiable.

Learning Strand-Equivariant Tokenizations of DNA with MirrorDNA

This paper introduces MirrorDNA, a DNA language model with a strand-equivariant tokenizer that improves prediction consistency across DNA strands.

Generative Models for Novel Enzymatic Reactions: How Far Do They Go?

This paper evaluates generative models for predicting novel enzymatic reactions, finding they can generate correct reactions for 25% of test cases and improve over existing methods.

Generative 3D Spatial Reconstruction of Cellular Organization from Single Cell Expression

ORBIS reconstructs 3D tissue geometry from single-cell gene expression without spatial references, achieving lower shape error and faster sampling than existing methods.

RPFace: From biased structural benchmarks to generalizable RNA-protein interface prediction

RPFace predicts RNA-protein interfaces directly from sequence, outperforming existing methods on RNA interfaces and generalizing to in vivo data without pretraining.

Stoichiometric Encoding of Amplification Kinetics Enables High-Level Multiplexing in Single-Channel TaqMan Real-Time PCR

This paper introduces a method to increase multiplexing in TaqMan qPCR by encoding target-specific information into amplification curves and decoding them with a transformer model, achieving 8-plex detection with high accuracy.

Towards a foundation model for raw nanopore signal through discrete tokenization

This paper introduces Brieno, a self-supervised model for raw nanopore signals that improves read classification and methylation detection without reference genomes.

RadMultiBench: A Reproducible Dual-Task Benchmark Exposing the Limits of Pre-Trained Vision-Language Models Across Thoracic and Breast Imaging

RadMultiBench evaluates pre-trained vision-language models on medical imaging tasks, showing performance gaps in report generation and classification across thoracic and breast imaging.

Joint protein-DNA language modeling enables prediction and engineering of DNA recognition specificity

This paper introduces PDLMs, protein-DNA language models that jointly represent protein and DNA sequences to predict and engineer DNA recognition specificity.

A comparative transcriptomic framework across kidney organoid, assembloid, and transplantation models

The paper compares transcriptomic profiles across kidney organoid, assembloid, and transplantation models to evaluate their maturity and suitability for research.

PartiDesign: A structure-sequence co-design paradigm for practical all-to-all biomolecules design

PartiDesign is a structure-sequence co-design model that generates molecular structures and sequences for proteins, nucleic acids, and small molecules, achieving high design throughput across benchmarks.

FoldAbS: Repurposing the Protein Folding Model as a Foundation Encoder for Antibody Screening

FoldAbS repurposes protein folding models as foundation encoders to improve antibody screening by evaluating specificity, affinity, and developability in a unified framework.

Decoding Cell-type Specific Gene Regulation with CellReg3D by Integrating DNA Large Language Models and 3D Genome Structure

CellReg3D integrates DNA language models and 3D genome structure to model cell-type-specific gene regulation, achieving state-of-the-art performance in predicting epigenetic marks and gene expression.

MicrobeRT: A Framework and Suite of Fine-tuned Language Models for Genomic Sequences

MicrobeRT is a framework of fine-tuned language models for metagenomic analysis, achieving strong performance in taxonomic classification, antibiotic resistance detection, and pathogenicity assessment.

AlphaFold 3 Prediction Accuracy and Reliability across RNA Structures and Protein-RNA Interfaces

This study evaluates AlphaFold 3's accuracy and reliability in predicting RNA structures and protein-RNA interfaces, finding that confidence scores do not always correlate with correct predictions.

Cross-cancer knowledge transfer for lymph node metastasis prediction: Applicability of an axillary lymph node-based deep learning model

This paper evaluates a deep learning model trained on axillary lymph nodes for predicting metastasis in lymph nodes from different anatomical regions across cancers.

Neural dynamics become significantly faster and more nonlinear at the peak of information processing

The paper shows that neural populations adjust their optimal temporal resolution during stimulus processing, becoming faster and more nonlinear when decoding information is most accurate.

All-Atom Structure Tokenizer: AtomLM-9b for Biomolecular Structure Prediction

The paper introduces AtomLM-9b, a language model using an all-atom structure tokenizer to predict protein structures with performance comparable to AlphaFold 3.

XAMPGen: An Interpretable SI Pipeline For Novel Antimicrobial Peptide Discovery

This paper introduces XAMPGen, an interpretable generative model for discovering novel antimicrobial peptides using fine-tuned protein language models and explainable AI techniques.

2D U-NET Deep Learning Architecture for Atherosclerotic Lesion Segmentation and Quantification

This paper develops a 2D U-NET model to segment and quantify atherosclerotic lesions in mouse aortic sections, achieving high accuracy and significantly reducing measurement time.

Omics-scale decoding of protein N-glycosylation using protein language model embeddings

The paper decodes protein N-glycosylation sites from protein language model embeddings using a zero-shot framework, revealing biological insights and disease associations.

VIGAT: VLM-guided gene activity quantification from single-cell ATAC-seq

VIGAT uses a vision-language model to quantify gene activity from single-cell ATAC-seq by analyzing accessibility patterns and genomic annotations across marker genes.

The Reasoning Paradox in Scientific Sandboxes: Why Extended Reasoning Fails in Autonomous Bioinformatics and How to Fix It.

This paper identifies two failure modes in extended reasoning models within bioinformatics tasks and introduces DTDR to improve accuracy and reduce errors.

Parameter-Efficient Adaptation of Protein Language Models for Dual-Target Antimicrobial Peptide Discovery Under Strict Homology Constraints

This paper develops a parameter-efficient transfer learning framework to discover antimicrobial peptides with high potency and low cytotoxicity using protein language models.

PhenoMe: An interactive AI framework for interpretable, microscopy-driven phenotyping

PhenoMe provides an open-source framework for analyzing microscopy images using foundation model-derived embeddings and experimental metadata to enable phenotyping and data exploration.

MaxCAT-Net: Hybrid Transformer-Based Skin Lesion Classification and Lightweight Segmentation with Calibration and Cross-Dataset Validation

MaxCAT-Net improves skin lesion classification and segmentation with hybrid architecture and calibration, showing performance on benchmark datasets but with preliminary results.

The transformer as a candidate canonical cortical microcircuit

The paper compares transformer architectures to cortical microcircuits, finding that transformers uniquely capture circuit details, functional maps, and wiring efficiency.

Fox: Predicting rates of host-symbiont co-evolution events with association-aware attention

Fox predicts host-symbiont co-evolution event rates directly from sequence alignments without tree reconstruction, achieving high accuracy and speed compared to existing methods.

Transformer Language Models Reveal Distinct Patterns in Aphasia Subtypes and Recovery Trajectories

This paper uses GPT-2 to analyze aphasia subtypes and recovery by examining activation patterns in narrative speech over time, finding distinct differences across subtypes and changes during recovery.