Improved Metagenomic Binning with Transformers

AO_SCPLOWBSTRACTC_SCPLOWTraditional metagenome binning methods cluster contiguous DNA sequences (contigs) based on uncontextualized features of the sequences which ignores both the semantic relationship between genes and the positional embedding of k-mers. This paper presents a novel binning method that addresses these concerns. Firstly, taken from natural language processing literature, a sequence representation model - Bidirectional Encoder Representations from Transformers (BERT) - is utilized to generate semantic and positional contig embeddings. Secondly, two workflows are presented; one which applies a hierarchical density-based clustering algorithm to find metagenomic bins and the other which incorporates contig embedding into a state-of-the-art binner. Experimental results on a publicly available metagenomic dataset show superior clustering for shorter contigs compared to traditionally used tetranucleotide frequency (TNF), reconstruction of up to 17% more high-precision genomes, and improved semantic understanding of contigs.