henhaobio
Sign in

bioRxivNeurosciencePreprint

Petascale neural circuit reconstruction: automated methods

Thomas MacrinaiD, Kisuk LeeiD, Ran LuiD, Nicholas L. TurneriD, Jingpeng WuiD, Sergiy PopovychiD, William SilversmithiD, Nico KemnitziD, J. Alexander BaeiD, Manuel CastroiD, Sven DorkenwaldiD, Akhilesh HalageriiD, Zhen JiaiD, Chris A. JordaniD, Kai LiiD, Eric MitchelliD, Shanka Subhra MondaliD, Shang MuiD, Barak NehoraniD, William Wong, Szi-chieh YuiD, Ágnes L. BodoriD, Derrick BrittainiD, JoAnn BuchananiD, Daniel J. BumbargeriD, , H. Sebastian SeungiD

PDFDOICodeData

In-depth summary

Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 1 (5 Aug 2021) · CC BY 4.0

Cloud-run convolutional nets aligned 19,939 serial EM sections of mouse visual cortex despite cracks and folds, segmented neurons and detected 523 million synapses; code and the reconstruction are public.

3D EM had mapped invertebrate nervous systems but only small pieces of mammalian brain, and the authors' earlier terascale pipeline for a MICrONS pilot handled cracks and folds by blacking out misaligned tissue with human-in-the-loop fixes, which could not scale to a 400 times larger, more defect-ridden volume. Here self-supervised convolutional nets compute nonsmooth alignments, and the whole pipeline runs on the cloud over roughly 1.5 petapixels of mouse cortex.

Key results

  1. ShownSelf-supervised convolutional-net alignment, with a defect detector gating the smoothness penalty, aligned a contiguous series of 19,939 serial sections riddled with cracks and folds without a global elastic relaxation step.

    19,939 contiguous sections were aligned out of over 27,000 cut; the authors cite a prior record of 10,000 aligned sections. 34 sections with cracks displacing more than ~30 μm were first corrected by manual point correspondences, and a stretch of 38 partial sections after a sectioning hiatus was completed by compositing. No quantitative alignment error is reported. · Image acquisition and alignment; Large crack correction; Alignment at 1024 nm & 64 nm; Alignment in two parts for handling partial…

  2. SuggestedAutomated synapse detection across the volume yielded 523 million synapses, with cleft detection and partner assignment highly accurate on small held-out sets after iteratively correcting clefts that human annotators had missed.

    523 million synapses (186 million in subvolume-35, 337 million in subvolume-65). Cleft detection: 98.2 precision and 98.2 recall on a test set with 56 clefts. Partner assignment: perfect on 81 test clefts, 96% accuracy on 110 validation clefts. Label review, padding annotation, 10 extra small volumes and one blood-vessel volume quadrupled the cleft training set to 914.0 μm^3 and 561 clefts. · Synapse detection and partner assignment; Synapse detection and synaptic partner…

  3. ShownInference, watershed and agglomeration were split into independent or octree-merged chunks on commercial cloud, with chunked agglomeration designed to reproduce the result of a global mean-affinity agglomeration.

    Net inference covered hundreds of thousands of chunks on thousands of cloud GPUs for several days; agglomeration took several weeks, at times on over 50,000 cloud CPUs, using 256 ✕ 256 ✕ 512 voxel chunks merged through an 11-layer octree; synapse partner assignment took around 2.5 days on 800 nodes. · Distributed computation; Watershed; Synapse detection and synaptic partner…

  4. SuggestedBoundary-detector training relied on purely manual ground truth targeted at observed failure modes and on simulated defects; correcting inconsistent labels of invaginating structures visibly reduced merge errors at those sites.

    943 μm^3 of MICrONS pilot ground truth for pretraining plus 425 μm^3 from the new dataset for finetuning (compared with 432 μm^3 manual and 31,063 μm^3 semi-automated for the Drosophila hemibrain). Simulated defects included misalignments with missing data and missing, blurry, thick and lost sections. The invagination fix is shown only in example images. · Leveraging ground truth of other datasets; Handling diverse cellular structures; Simulating image defects; Ground truth consistency

  5. SuggestedQualitatively, pyramidal-cell dendrites and the thicker axons of non-pyramidal cells such as basket cells are often reconstructed well, while thin pyramidal axon collaterals are frequently split.

    Example L5 pyramidal and basket cell reconstructions, with arrows marking true terminations, split errors and merge errors; apical and basal dendrites are recognisable. No error rates are given, and proofreading is left to a companion paper. · Examples of neurons; Figure legends

Methods, data and reproducibility

Serial-section EM images were aligned by self-supervised convolutional nets predicting nonsmooth displacement fields, refined by gradient descent with defect-gated smoothness; other nets predicted boundary affinities, five semantic classes, synaptic clefts and partners, and affinities were segmented by distributed watershed and mean-affinity agglomeration with semantic and size constraints, run on Google Cloud.

  • Serial-section transmission EM volume spanning V1 and higher visual areas LM, RL and AL, roughly 1.3 mm by 0.88 mm by 0.8 mm, about 1.5 petapixels at 4 ✕ 4 ✕ 40 nm^3 · mouse (P87 male) · n = 19,939 sections aligned (over 27,000 cut)
  • Boundary-detection ground truth: MICrONS pilot datasets (pretraining) and the new dataset (finetuning) · n = 943 μm^3 pilot + 425 μm^3 new
  • Semantic-labeling ground truth in 9 categories collapsed to 5 classes, plus two sparse blood-vessel volumes · n = 18 volumes, 333.7 μm^3 (253.5 train / 80.2 validation); sparse 5.6 and 8.3 μm^3
  • Synaptic cleft ground truth from aligned subregions of this dataset, iteratively corrected · mouse · n = 914.0 μm^3, 561 clefts (initially 227.6 μm^3, 135 clefts); test set 56 clefts
  • Synaptic partner assignment training data, including other EM volumes · n = 1660 μm^3, 867 clefts; test 81, validation 110 clefts
  • Fold and crack detector training samples (256 ✕ 256 px, labeled in VAST) and resin detector training samples (manually labeled) · n = 3,830 fold, 3,894 crack, 50 resin samples
  • Training subset of the rough aligned stack for the 64 nm alignment net · mouse · n = 10 hand-picked + 1200 random samples of 5 consecutive sections
Methods & tools
SEAMLeSS, DeepEM, Chunkflow, abiss, seuron (Apache Airflow), Synaptor, Igneous, CloudVolume, Kimimaro, Neuroglancer, Render / render-modules, VAST
Software & environment
Google Cloud with Docker containers, cloud task queues and Kubernetes; Apache Airflow DAGs for segmentation workflows; PyTorch for the semantic net. No software versions or requirement files are stated.
Experimental conditions
Inference: thousands of GPUs for several days; agglomeration: several weeks, at times >50,000 CPUs; partner assignment: ~2.5 days on 800 nodes. Training: 4 or 8 Titan X Pascal GPUs (boundary); GTX 1080Ti, ~4 days for the first 300k iterations, then a further 300k (semantic).
Reproducibility gaps
The alignment method and its distributed implementation are left to later publications; no software versions or requirement files are given; inference code for defect detection is on a SEAMLeSS branch (alex-emdetector).
Limitations (6)
  • Residual misalignments are mostly small but still account for a significant fraction of segmentation errors; the extremes of L1 and L6 were excluded because alignment could not cope with their higher artifact rates. · stated by the authors
  • Boundary detection still erases plasma membranes where mitochondria press against them, at endoplasmic reticulum to plasma membrane contacts and near other unidentified membranous structures; low contrast and missing or thick sections also cause errors. · stated by the authors
  • Thin pyramidal axon collaterals have frequent split errors and need substantial human proofreading; automated error detection and correction has not been deployed in the pipeline. · stated by the authors
  • The volume was segmented as two separate subvolumes (about 65% and 35% of sections) that are not yet united, and only subvolume-65 has been proofread. · stated by the authors
  • Scaling to a whole mouse brain (about 1000✕ larger) is limited by the top octree agglomeration step, whose shared-memory machine would need to be about 100✕ larger than the one used here. · stated by the authors
  • Accuracy is quantified only for synapses, on small test sets (56 clefts for detection, 81 for partner assignment); neuron segmentation and alignment quality are judged only qualitatively from example cells and images, with no error rates. · noted in this summary

Read this if you need to align, segment or detect synapses in a large serial-section EM volume, or want reusable cloud tooling (Chunkflow, CloudVolume, Igneous, Synaptor) for chunked processing of very large 3D images.

Citations
55

Abstract

3D electron microscopy (EM) has been successful at mapping invertebrate nervous systems, but the approach has been limited to small chunks of mammalian brains. To scale up to larger volumes, we have built a computational pipeline for processing petascale image datasets acquired by serial section EM, a popular form of 3D EM. The pipeline employs convolutional nets to compute the nonsmooth transformations required to align images of serial sections containing numerous cracks and folds, detect neuronal boundaries, label voxels as axon, dendrite, soma, and other semantic categories, and detect synapses and assign them to presynaptic and postsynaptic segments. The output of neuronal boundary detection is segmented by mean affinity agglomeration with semantic and size constraints. Pipeline operations are implemented by leveraging distributed and cloud computing. Intermediate results of the pipeline are held in cloud storage, and can be effortlessly viewed as images, which aids debugging. We applied the pipeline to create an automated reconstruction of an EM image volume spanning four visual cortical areas of a mouse brain. Code for the pipeline is publicly available, as is the reconstructed volume.

Reproducibility ledger

Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.

Sign in to file a reproducibility report.

Related papers

Similarity Map
SelectedSame fieldCross-field
THISPopovych '24Hubbard '20Yin '20Marrett '25Mahalingam '22Marrett '21
79% semantic matchNeuroscience
Sergiy Popovych, Thomas Macrina, Nico Kemnitz+10 more·Nature Communications·Jan 4, 2024· 39 citations