Data Synthesis Improves 3D Myotube Instance Segmentation

arXiv:2604.14720 · cs.CV · Submitted 2026-04-16 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom, and with me are Jane, Lu, senior AI researcher at Tsinghua, Meng, lead engineer at a mysterious AI startup and Lalam, the in-house Large Language Model.

Jane: Today's paper: "Data Synthesis Improves 3D Myotube Instance Segmentation".

Tom: Myotubes are crucial model systems for studying muscle physiology and disease, but existing 3D segmentation models fail to generalize due to a lack of large annotated datasets.

Jane: First, who's behind it and why it matters.

Title and authors: Tom: Moving on to the authors and title, it seems David Exler and his team have really put together a solid framework here for generating training data where it was previously missing <ref:2604.14720#pg0>.

Jane: That's right. The authors are showing how they moved beyond just using existing models and instead built a system that generates the necessary training material from scratch, which is a big step forward in self-supervised learning approaches <ref:2604.14720#pg1>.

Lu: Their approach to modeling the geometry—using polynomial centerlines and specific modifications for thickness—shows a deep understanding of how muscle fibers actually branch and taper in nature <ref:2604.14720#pg0>.

Meng: I’m thinking about the practical side here; generating realistic noise and optical artifacts is one thing, but ensuring that synthetic data genuinely captures the subtle nuances we need for high-accuracy segmentation is another challenge <ref:2604.14720#pg1>.

Lalam: The detail in their synthesis pipeline suggests a level of control over data creation that could lead to incredibly robust and generalizable AI systems down the road, which is inspiring <ref:2604.14720#pg0>.

The paper's summary: Tom: So what they are summarizing is that they developed a geometry-driven synthesis pipeline to create synthetic three dee myotube data, and then used this synthetic data to train two compact residual three dee U-Nets with self-supervised pretraining for instance segmentation <ref:2604.14720#pg0>.

Jane: Essentially, they showed that by synthesizing data based on real geometry, they could train a model—specifically the SSL (DA) variant—that achieves a mean Injective Panoptic Quality of zero point two two on real data <ref:2604.14720#pg1>.

Lu: The core idea is that this compact U-Net, especially when paired with the self-supervised encoder pretraining using FCMAE on real myotube data, performs better than models that are just trained from scratch <ref:2604.14720#pg1>.

Meng: That zero point two two IPQ score is interesting; what does that mean in terms of segmentation quality when we compare it to the established zero-shot models they tested? <ref:2604.14720#pg3>.

Lalam: It means this method provides a strong baseline performance, proving that controlled synthesis can substitute for costly manual annotations in domains where annotated data is scarce <ref:2604.14720#pg0>.

The paper's improvements: Tom: Now let’s talk about the specific improvements they highlight, and it seems the main thing is this combination of SSL pretraining with domain adaptation using CycleGAN, which they call the SSL (DA) model <ref:2604.14720#pg1>.

Jane: They’ve shown that training on unadapted synthetic data alone isn't enough; you need that domain adaptation step to bridge the gap between the synthetic training environment and real imaging conditions <ref:2604.14720#pg1>.

Lu: The fact that they trained the decoder twice—once on raw synthetic data and once after adapting it—and then compared all four variants against established models like CSAM and psG is a really thorough experimental design <ref:2604.14720#pg3>.

Meng: I see the complexity in that setup; they have four distinct model variants, including the SSL-pretrained UNet + DA version, which suggests that combining those two techniques yields the best result <ref:2604.14720#pg3>.

Lalam: This layered approach is powerful because it shows how combining structure learning from real data features with domain adaptation can significantly enhance the final segmentation output <ref:2604.14720#pg3>.

Conclusion: Tom: Wrapping things up, the conclusion is that this geometry-driven synthesis pipeline allows for instance segmentation in domains without large annotated datasets, proving that controlled synthesis can replace expensive manual annotations <ref:2604.14720#pg0>.

Jane: So the main implication is that we can create a method where synthetic data generation based on real anatomy helps train compact U-Nets to perform well, reaching an IPQ of zero point two two <ref:2604.14720#pg3>.

Lu: The paper points out the limitation clearly: the remaining gap to a perfect score is attributed to the domain shift between synthetic training data and real imaging conditions <ref:2604.14720#pg3>.

Meng: That domain shift is something we have to address next, especially when moving toward more complex tasks where disease-specific morphological variations exist <ref:2604.14720#pg0>.

Lalam: Ultimately, the findings on "Data Synthesis Improves three dee Myotube Instance Segmentation" suggest a viable path for creating high-quality segmentation models for many biological systems by substituting manual annotation costs with controlled data generation <ref:2604.14720#pg0>.

David Exler, Nils Friederich, Martin Krüger, John Jbeily, Mario Vitacolonna, Rüdiger Rudolf, Ralf Mikut, Markus Reischl

Institute for Automation and Applied Informatics at Karlsruhe Institute of Technology (KIT) · Institute of Biological and Chemical Systems at Karlsruhe Institute of Technology (KIT) · CeMOS Research and Transfer Center, Technische Hochschule Mannheim

cs.CV

Submitted: 2026-04-16

Updated: 2026-04-16

Comments: 4 pages, 4 figures, submitted to BMT (VDE) 2026 Conference

Journal ref: Current Directions in Biomedical Engineering (2026)

DOI: 10.1515/cdbme-2026-0134

Code: https://github.com/DavidExler/syn_myo

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: Myotubes are crucial model systems for studying muscle physiology and disease, but existing 3D segmentation models fail to generalize due to a lack of large annotated datasets.

Key concepts

Geometry-driven Synthesis Pipeline
This is a multi-stage process that extracts geometric features from real myotube data to generate synthetic training volumes. It involves sampling centerlines using Chebyshev basis functions, modulating local thickness with polynomials and sinusoids, inserting branching segments, and placing ellipsoidal structures along the centerline.
Self-Supervised Learning (SSL)
The encoder part of the U-Net is pre-trained on real myotube data using Fully Convolutional Masked Autoencoding (FCMAE). This teaches the model to reconstruct missing 3D patches by masking parts of an input volume, providing a strong initial understanding of 3D shapes before task-specific training.
Domain Adaptation (DA)
CycleGAN is used to adapt the synthetic data generated by the synthesis pipeline to resemble real imaging conditions. This process helps bridge the gap between the artificially generated data and actual experimental images, improving the model's generalization.

Terminology

Summary

Myotubes are crucial model systems for studying muscle physiology and disease, but existing 3D segmentation models fail to generalize due to a lack of large annotated datasets. This paper introduces a geometry-driven synthesis pipeline that generates synthetic 3D myotube data, enabling the training of a compact U-Net to achieve superior instance segmentation performance compared to established zero-shot models.

The gist

A compact 3D U-Net with self-supervised encoder pretraining, trained exclusively on synthetic data, achieves a mean IPQ of 0.22 on real data, significantly outperforming three established zero-shot segmentation models.

How it works: Data Synthesis Pipeline

The researchers derive geometrical characteristics from real myotube data to generate synthetic annotation volumes in three distinct stages. First, polynomial centerlines are sampled in XY and Z using damped Chebyshev basis functions Tk of degree K with coefficients c˜k ∼ U(−1, 1) scaled by k −α to suppress boundary oscillations. Local thickness is then modulated by a smooth polynomial and a sinusoidal term controlled by γ and δ. Second, straight branching segments are inserted while preserving slope continuity. Finally, ellipsoidal structures are placed along the centerline, with hollow ellipsoids along the shaft and solid caps at endpoints. These synthetic volumes are rendered with realistic artifacts, including additive Poisson and Gaussian noise, salt-and-pepper textures, debris artifacts, anisotropic halos, and Gaussian PSF blurring, further enhanced by a CycleGAN for Domain Adaptation (DA).

How it works: Segmentation Model Architecture

The study trains two compact residual 3D U-Nets with four encoder stages. The encoder of one model is pretrained on real myotube data using Fully Convolutional Masked Autoencoding (FCMAE), which involves randomly masking 3D input patches and learning to reconstruct the missing regions. This SSL variant is extended with additional layers that are trained from scratch, while the SSL-pretrained encoder layers are frozen. The segmentation decoder is trained twice: once on unadapted synthetic data and once on the domain-adapted data generated using CycleGAN. This results in four model variants: (1) UNet, (2) UNet + DA, (3) SSL-pretrained UNet, and (4) SSL-pretrained UNet + DA. The model predicts foreground and centerline probabilities, which are then used to predict a binary mask via thresholding the foreground probability and a seeded Watershed algorithm using the thresholded centerline probability to separate instances.

How it works: Validation and Results

The models were benchmarked against three established zero-shot models: CellposeSAM, PlantSeg, and StarDist. Visual inspection showed that established models like CSAM merges multiple instances into large irregular blobs and psG segments numerous small round regions. In contrast, the SSL (DA) model produced the closest match to real instances, with the domain-adapted version achieving a mean IPQ of 0.22. The ablation study revealed that combining SSL pretraining with DA leads to a substantial performance gain (0.22), explaining why combining SSL with DA leads to a substantial performance gain (0.22), outperforming all other variants by a large margin.

How it works: Quantitative Comparison

Performance was evaluated using the Injective Panoptic Quality (IPQ) over n = 40 annotated instances. The results demonstrated that our models consistently outperformed all three zero-shot baselines, with the SSL (DA) model achieving a mean IPQ of 0.22. Specifically, Our domain-adapted SSL model achieves a mean IPQ of 0.22, demonstrating the substantial impact of our data synthesis pipeline. The remaining gap to a perfect score is attributed to the domain shift between synthetic training data and real imaging conditions.

Conclusion

The paper presents the first geometry-driven synthesis pipeline for 3D myotube data, enabling instance segmentation in a domain where no annotated training data exists. This approach demonstrates that controlled geometry-driven synthesis can substitute for costly manual annotations and generalizes to similar domains such as 3D nerve fiber or mycelium networks. Future work will focus on closing the remaining domain gap, incorporating disease-specific morphological variations, and scaling toward a large, well-performing instance segmentation model. The synthesis pipeline is publicly available at github.com/DavidExler/syn myo.

The gist

A compact 3D U-Net with self-supervised encoder pretraining, trained exclusively on synthetic data, achieves a mean IPQ of 0.22 on real data, significantly outperforming three established zero-shot segmentation models.

How it works: Data Synthesis Pipeline

The researchers derive geometrical characteristics from real myotube data to generate synthetic annotation volumes in three distinct stages.

Improvements for AI systems

Here are the specific improvements for AI systems derived from this research, focusing on what these improved systems can achieve:

  1. Improved Generalization of 3D Biomedical Segmentation Models:

  2. Enhanced Performance on Scarce/Novel Datasets:

  3. Creation of Self-Supervised Learning (SSL) Pretraining Strategies for Complex Structures:

  4. Development of Robust Domain Adaptation Techniques for Synthetic-to-Real Transfer:

Here is what the improved AI systems can do, based on the paper's findings:

This improved AI system, specifically the SSL (DA) model, can perform the following specific tasks and achieve these capabilities:

  1. Can accurately segment individual myotube instances in 3D microscopy images when no large, manually annotated datasets exist (i.e., zero-shot performance).

  2. Can detect elongated structures and complex geometries (like branching and overlapping fibers) that fail to be captured by models trained on compact or spherical cell morphologies (e.g., CSAM, psG, StarDist).

  3. Can produce clean, high-fidelity segmentation masks that accurately delineate the full length of highly elongated muscle fibers, including features like nuclear bulges and tapering distal ends.

  4. Can maintain high instance quality (as measured by a mean IPQ of 0.22) even when trained exclusively on synthetic data generated via a geometry-driven synthesis pipeline, effectively substituting for expensive manual annotations.

  5. Can achieve superior performance compared to established state-of-the-art models (e.g., outperforming CellposeSAM, PlantSeg, and StarDist) by leveraging both geometry-driven synthesis and SSL pretraining on real data features to bridge the domain gap between synthetic training data and real imaging conditions.

Abstract

Myotubes are multinucleated muscle fibers serving as key model systems for studying muscle physiology, disease mechanisms, and drug responses. Mechanistic studies and drug screening thereby rely on quantitative morphological readouts such as diameter, length, and branching degree, which in turn require precise three-dimensional instance segmentation. Yet established pretrained biomedical segmentation models fail to generalize to this domain due to the absence of large annotated myotube datasets. We introduce a geometry-driven synthesis pipeline that models individual myotubes via polynomial centerlines, locally varying radii, branching structures, and ellipsoidal end caps derived from real microscopy observations. Synthetic volumes are rendered with realistic noise, optical artifacts, and CycleGAN-based Domain Adaptation (DA). A compact 3D U-Net with self-supervised encoder pretraining, trained exclusively on synthetic data, achieves a mean IPQ of 0.22 on real data, significantly outperforming three established zero-shot segmentation models, demonstrating that biophysics-driven synthesis enables effective instance segmentation in annotation-scarce biomedical domains.

Sources

Related papers