TACoS: Weakly Supervised Learning of Two-Dimensional Materials from Scribble Annotations to Precise Segmentation

arXiv:2607.07169 · cs.CV · Submitted 2026-07-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "TACoS: Weakly Supervised Learning of Two-Dimensional Materials from Scribble Annotations to Precise Segmentation".

Jane: The precise pixel-level localization of two-dimensional material flakes is crucial for high-throughput screening, and this paper proposes Tree-based Asymmetric Contrast Segmentation (TACoS),

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Now we're getting into the actual summary of TACoS, and it seems the core idea is a unified framework that combines semi-supervised consistency learning with structured tree energy constraints to recover complete segmentation masks from those sparse scribble annotations. It sounds like a very integrated approach.

Jane: That integration is key; they use three main components—Unlabeled Weak–Strong Distribution Alignment, Tree Energy Regularization, and Asymmetric Regional Contrastive Learning—to make sure the model learns effectively even in the unlabeled parts of the data without needing dense labels everywhere.

Lu: What's really compelling is how they handle those unlabeled regions; they use UWSD to enforce consistency between weak and strong augmentation views only on those unlabeled pixels, while TER generates structure-aware soft references based on minimum spanning trees built from backbone features.

Meng: The idea of building two MSTs—one for low-level textures and one for high-level semantics—to create a soft reference seems quite sophisticated; I wonder how computationally expensive that online computation is during training.

Lalam: If the TER module can dynamically update based on the current feature structure, it means the model is constantly refining its understanding of the object's structure in real time, which could lead to much more robust segmentation across different samples.

The paper's summary: Tom: So, to summarize what we’ve covered so far about TACoS, this framework aims to solve the need for dense annotations by using a three-pronged approach: UWSD for consistency on unlabeled data, TER for structure-aware soft references, and ARCL for refining pseudo labels. This is all put together into one single training objective.

Jane: It sounds like they’ve managed to get this entire system into a single-stage design where everything is optimized together at once, which is much cleaner than older methods that required generating pseudo-labels offline first.

Lu: The paper highlights that this strategy enhances intra-class cohesion and inter-class separation at the representation level, which effectively reduces category confusion in low-contrast edges and complex backgrounds when dealing with those scribble annotations.

Meng: I see how that addresses the weak contrast issue; if the representation learning part is strong enough to handle ambiguous boundaries, then relying on sparse input shouldn't cause the segmentation to drift too much.

Lalam: It’s really about improving the underlying representation quality so that even with minimal supervision, we can extract high-quality structural information from the data.

The paper's improvements: Tom: Moving on to what they actually improved, TACoS suggests specific technical enhancements. They introduce UWSD to provide dense training signals for unlabeled regions and ARCL to fuse those weak predictions with the sparse scribbles into augmented labels.

Jane: And the authors emphasize that ARCL simultaneously enforces region-level asymmetric contrast constraints in both the representation space and the decision space, which is a neat trick for boundary discrimination.

Lu: Furthermore, they use an exclusion penalty based on asymmetric distance sampling to focus constraints on challenging pixels in unlabeled boundary regions when applying those strong augmentation branch logits. This specifically targets the hard parts of segmentation that are often missed by simpler methods.

Meng: Focusing the constraint specifically on those challenging boundary pixels using that penalty term sounds like a practical way to handle the ambiguity of weakly contrasted edges without needing massive amounts of data just for those areas.

Lalam: I think this asymmetry in how they sample and constrain boundaries is what really sets it apart; it shows a deep understanding of where the model struggles and how to apply targeted learning there efficiently.

Conclusion: Tom: Alright, we've covered the title, the summary of TACoS, and those specific improvements focusing on UWSD, TER, and ARCL. Overall, this paper shows a very efficient way to leverage sparse scribble annotations for precise segmentation in material science.

Jane: It sounds like TACoS achieves over ninety-six percent of fully supervised performance using less than zero point six percent annotated data while maintaining better structural coherence and boundary stability than previous methods, which is a significant result <ref:2607.07169#pg1,TACoS achieves over 96% of fully supervised performance using less than 0>.

Lu: I think the main implication here is that we can finally move towards automated high-throughput screening of two-dimensional material flakes without needing expensive, dense labeling for every single sample.

Meng: From my side, the practical impact is huge; if this framework scales well, it could drastically speed up the experimental workflow in research labs and reduce the time researchers spend on manual annotation tasks.

Lalam: This work really pushes the culture of AI development in materials science by showing how we can build tools that are highly effective even under severe data scarcity constraints.

Tom: So, to wrap things up, TACoS is a specialized scribble segmentation framework that uses a unified approach to overcome sparse annotation challenges for 2D materials <ref:2607.07169#pg1>. It’s a powerful tool for making high-throughput screening feasible without massive labeling efforts.

Jane: It’s an exciting direction because it proves that sophisticated consistency learning and structural regularization can work together effectively in complex visual tasks like this.

Lu: The future work will likely involve testing how well these constraints hold up when applied to even more diverse material types beyond graphene and MoS2, exploring the generalizability of the tree energy constraints.

Meng: I'm curious if we can adapt those tree structures for other complex three dee data representations later on; it’s a versatile concept <ref:2607.07169#pg1>.

Lalam: This paper sets a high bar for what sparse annotation can achieve in scientific imaging, and it opens up exciting avenues for applying this type of learning to other fields that deal with rare but important visual data.

Jiabei Chena, Liping Zhanga, Jiang-Bin Wub, Zhongming Weib, Enhao Ninga, Su Yana, Weijun Lia, Ping-Heng Tanb, Xin Ninga

AnnLab · Institute of Semiconductors, Chinese Academy of Sciences · State Key Laboratory of Semiconductor Physics and Chip Technologies, Institute of Semiconductors, Chinese Academy of Sciences · Center of Materials Science and Optoelectronics Engineering & School of Integrated Circuits, University of Chinese Academy of Sciences

cs.CV

Submitted: 2026-07-08

Updated: 2026-10-04

Comments: 35 pages, 7 figures

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: The precise pixel-level localization of two-dimensional material flakes is crucial for high-throughput screening, and this paper proposes Tree-based Asymmetric Contrast Segmentation (TACoS), a

Key concepts

Unlabeled Weak–Strong Distribution Alignment (UWSD)
This module enforces consistency on unlabeled pixels by comparing predictions from weak and strong augmented views. It uses the prediction distribution of the weak view as a soft target to constrain the strong view's output, providing dense training signals without updating gradients from the weak branch.
Tree Energy Regularization (TER)
TER builds minimum spanning trees on backbone features to understand pixel relationships and create structure-aware soft references. It uses dual trees (shallow and deep features) to map feature distances into an affinity matrix, which is then filtered to generate a reference signal for the segmentation.
Asymmetric Regional Contrastive Learning (ARCL)
ARCL refines unlabeled pseudo-labels by learning class prototypes through region-level contrastive learning. It also applies an exclusion penalty based on asymmetric distance sampling to boundary neighborhoods, specifically focusing constraints on challenging pixels near edges.

Terminology

Summary

The precise pixel-level localization of two-dimensional material flakes is crucial for high-throughput screening, and this paper proposes Tree-based Asymmetric Contrast Segmentation (TACoS), a specialized scribble segmentation framework designed to achieve this precise segmentation using extremely sparse annotations. The gist: TACoS achieves over 96% of fully supervised performance using less than 0.6% annotated data while exhibiting superior structural coherence and boundary stability in scenarios with weakly contrasting edges and complex backgrounds.

The Framework Overview

TACoS is a unified framework that integrates semi-supervised consistency learning with structured tree energy constraints to recover complete segmentation masks from extremely sparse scribble annotations. The framework comprises three tightly coupled key designs: Unlabeled Weak–Strong Distribution Alignment (UWSD), Tree Energy Regularization (TER), and Asymmetric Regional Contrastive Learning (ARCL). This approach enables efficient learning in unlabeled regions at low annotation cost while significantly enhancing boundary and structural stability, unlike multistage approaches which require offline pseudo-label generation.

Unlabeled Weak–Strong Distribution Alignment (UWSD)

This module enforces consistency constraints on prediction distributions between weak/strong augmentation views only on unlabeled pixels, providing dense training signals for the dominant unlabeled regions. The process involves constructing weak augmentation view and strong augmentation view inputs, where the geometric transformation is applied to generate the weak branch input, and appearance perturbations are applied to this result for the strong branch. For unlabeled regions in set omegaU, UWSD employs the prediction distribution of the weak augmentation view as a soft objective, constraining predictions of the strong augmentation view to align with it using a cosine consistency constraint defined by equation (4). Crucially, gradients from the weak-augmentation branch are stopped because it only provides a relatively stable soft target and is not updated by this loss, preventing distribution drift.

Tree Energy Regularization (TER)

TER builds a minimum spanning tree (MST) on backbone features to capture pixel affinity relationships and generates structure-aware soft references. It utilizes a dual-tree architecture: shallow features capture microscopic textures and color gradients, while deep features encode high-level semantics. By constructing two MSTs, Glow and Ghigh in the low-level and high-level feature spaces respectively, the method maps tree distances to an affinity matrix (equation 8). A structure-aware soft reference is then generated by applying a tree filter operator T to this affinity matrix: a = Thigh(Tlow (omega, ps, A)) (equation 9). This process is computed online from the current feature structure and prediction, dynamically updated at each iteration. The objective for TER on the unlabeled pixel set omegaU is to minimize the discrepancy between the strong augmentation-branch prediction and this soft reference: L TER = 1/omegaU X i∈omegaU ps(i) − sg(a(i)) 1 (equation 10).

Asymmetric Regional Contrast Learning (ARCL)

ARCL completes unlabeled pseudo labels using high-confidence predictions from the weak augmentation branch, which are then fused with scribble annotations to form augmented labels. It simultaneously enforces region-level asymmetric contrast constraints in the representation space and decision space. The module performs two key functions: first, it constructs class prototypes by performing region-level contrastive learning on deep decoder features, defined by the loss LRegion, which promotes intra-class cohesion and expands inter-class separation at the representation level. Second, it applies an exclusion penalty based on asymmetric distance sampling to boundary neighborhoods on strong augmentation branch logits. This is achieved by selecting challenging pixels S and applying a penalty term LBoundary: LBoundary = 1/E X i∈E 1 + w(i) · ⟨zs(i+), zs(i-)⟩ zs(i+) squared zs(i-) 2 (equation 15). This mechanism enhances boundary discrimination accuracy by focusing constraints on challenging pixels in unlabeled boundary regions.

Overall Optimization and Results

The final optimization objective is a combination of sparse supervised loss, UWSD, TER, and ARCL: L = Lsup + λc LUWS D + λt LT ER + LARCL (equation 18). The framework employs a single-stage design where all objectives are jointly minimized in one continuous training process. Experimental results on the graphene and MoS2 datasets demonstrate that TACoS achieves over 96% of fully supervised performance using less than 0.6% annotated data, with superior structural coherence and boundary stability compared to existing methods like URSS and CC4S, achieving the best Boundary IoU performance in both material domains. The ablation studies confirm that the full combination of all three modules (UWSD, TER, ARCL) achieves optimal scores of 84.09% mIoU on graphene and 85.20% mIoU on MoS2.

Improvements for AI systems

Here are specific improvements to existing AI systems based on the TACoS framework, along with what those improved systems can achieve:

)

  1. Improve segmentation accuracy for 2D materials from sparse annotations (scribbles).

  2. Enhance boundary stability and discrimination in low-contrast/complex background scenarios.

  3. Enable high-throughput, cost-effective automated screening of 2D material flakes in experimental workflows.

Sources

Related papers