Partial AUC Maximization from Positive-unlabeled Data

arXiv:2610.00284 · cs.LG, cs.AI, stat.ML · Submitted 2026-09-25 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Partial AUC Maximization from Positive-unlabeled Data".

Jane: The proposed method addresses the challenge of maximizing partial area under the ROC curve (pAUC) from positive-unlabeled (PU) data,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: Jane, we've got this paper on Partial AUC Maximization from Positive-unlabeled Data. The main thing they're tackling is how to get that pAUC metric without needing labeled negative data, which is a huge problem in areas like cybersecurity and medical care where negatives are just out of reach.

Jane: That sounds really relevant because we often run into situations where getting those true negative examples is practically impossible for privacy reasons or just because it takes too much specialized knowledge to annotate them correctly. The paper's core thesis seems to be that they can reformulate pAUC using only the positive and marginal densities, which lets us train a classifier without needing any actual negative labels in the training set.

Lu: I'm really interested in that reformulation part; if you can express the objective purely through density functions, it opens up so much creative space for how we define what 'positive' means in these unlabeled scenarios. It suggests a way to build robust estimators from sparse information rather than relying on assumptions about ranking unlabelled instances.

Meng: From an engineering standpoint, that’s interesting because it moves us away from those heuristic assumptions that existing semi-supervised methods make—the ones where you just rank the unlabeled data and assume anything not ranked highly is negative—which the paper points out doesn't always match the true pAUC. We need something more mathematically grounded for practical deployment.

Lalam: If we consider how this affects our culture, Lalam finds that if we can build models that are inherently less reliant on perfectly labeled data, it might lead to a more distributed and less centralized AI development approach where smaller entities can contribute meaningfully to model refinement.

Tom: Exactly, Meng; the shift from heuristic ranking to density-based representation sounds like the key mechanism they're proposing here in "Partial AUC Maximization from Positive-unlabeled Data." They show how pAUC, including its FPR-dependent thresholds, can be represented using only positive and marginal densities.

Jane: And that representation allows them to derive an empirical estimator for pAUC directly from the positive and marginal densities of the data, which is a big step because it bypasses the need for labeled negative examples entirely when training.

Lu: That derivation linking the FPR-dependent thresholds to just positive and marginal densities seems like a really sophisticated mathematical trick; it shows how you can capture that complex performance measure using only what you actually have available in the PU data.

Paper summary: Meng: But I gotta ask, how robust is this empirical estimator when we apply it to real-world scenarios where the underlying density assumptions might be slightly off? We need to know if these bounds hold up under messy, real-world conditions.

Lalam: It’s exciting because if this technique works consistently across different data types, imagine how much faster we could deploy high-accuracy classifiers in fields that are currently bottlenecked by data labeling requirements.

Tom: So, we've seen the setup for the "Partial AUC Maximization from Positive-unlabeled Data" paper: they established that pAUC can be expressed through only positive and marginal densities, leading to a new empirical estimator. Now we move into what this actually means for us in the real world.

Jane: Precisely; the authors show a formal way to derive an objective function that depends only on those densities, which then feeds into a loss function for training our AI systems by maximizing pAUC. This is about training without needing those pesky negative labels we usually struggle to get.

Lu: The theoretical analysis they do later is where things get really deep; they use concentration inequalities like the Dvoretzky–Kiefer–Wolfowitz inequality to bound the estimation errors for both marginal and positive score survival probabilities. That gives us a solid mathematical guarantee about how close their estimate is to the true pAUC.

Meng: A mathematical guarantee is great, but I’m still thinking about scalability; if we're dealing with massive datasets, how does this estimator perform computationally compared to methods that might require more complex sampling or instance selection processes?

Lalam: If the consistency bounds they prove hold true as N and Np grow very large, it suggests that this approach could support the development of much larger, more reliable foundational models where data scarcity is a major issue.

Tom: So, to wrap up this part of the discussion about "Partial AUC Maximization from Positive-unlabeled Data," we've heard that they provide a method to estimate pAUC using only positive and marginal densities, and their theoretical analysis gives us high-probability convergence bounds for that estimate.

Jane: Essentially, the paper shows a path to training classifiers effectively when you're stuck with positive-unlabeled data by reformulating the objective around those densities. This moves beyond just using simple ranking heuristics to achieve a more accurate pAUC maximization.

Lu: The potential for this is huge because it addresses a fundamental bottleneck in many domains where obtaining complete labeled datasets is simply not feasible, suggesting we can unlock performance from the data we already possess.

Paper summary: Meng: For practical implementation, the paper's focus on deriving an empirical estimator suggests a pipeline that could be integrated into existing machine learning workflows without requiring a massive overhaul of how we collect our training sets.

Lalam: I think this work opens up avenues for building more resilient and data-efficient AI systems across various industries, because it tackles the data scarcity problem head-on in a way that respects the constraints of real-world data collection.

Tom: We're getting to that big picture now; if we can deploy this kind of training method reliably, it means AI development won't be limited only to scenarios with perfectly balanced datasets.

Jane: That’s right; the title "Partial AUC Maximization from Positive-unlabeled Data" points directly at how we can extract valuable performance metrics even when the negative class information is missing.

Lu: The implication for research is that future work could explore how these density representations generalize across different complex data distributions, which is where I see a lot of exciting possibilities brewing.

Meng: From an engineering perspective, the paper’s validation across ten distinct real-world datasets like MNIST and CIFAR10 shows that this isn't just theoretical; it has some empirical backing on diverse data types.

Lalam: That diversity across datasets is what makes it so compelling; if it holds up across such varied domains, the potential for improving AI performance in cybersecurity or medical diagnostics becomes much more tangible.

Tom: So, we've looked at how the "Partial AUC Maximization from Positive-unlabeled Data" paper achieves its goal: formulating pAUC using only positive and marginal densities and providing a theoretically sound way to estimate it from PU data.

Jane: And the conclusion is that this method offers a viable path for training classifiers in settings where labeled negative data are unavailable, shifting the focus to leveraging what we have.

Lu: It really pushes us to rethink how we define 'performance' when we don't have access to the full label distribution, which is a deep conceptual shift for AI research.

Meng: I think the immediate impact will be in helping us build more practical, deployable models for applications where data collection is inherently difficult or restricted.

Lalam: Indeed; this kind of work helps foster an environment where we can push the boundaries of what's possible with existing data constraints, making AI tools more accessible everywhere.

Tom: That’s a solid overview of the "Partial AUC Maximization from Positive-unlabeled Data" paper; it clearly outlines the problem, presents a new mathematical formulation, and gives us some good theoretical backing for its estimator.

Conclusion: Tom: So, we've seen how this paper tackles maximizing partial AUC using only positive and marginal densities from positive-unlabeled data, and now we're wrapping up with some big thoughts on what this actually means for us.

Jane: It really boils down to taking a performance metric that usually needs perfect labels—pAUC—and finding a way to calculate it reliably when the negative examples are just missing, which is exactly what these authors set out to do.

Lu: The title itself, "Partial AUC Maximization from Positive-unlabeled Data," perfectly captures the essence of their contribution because they're focusing on extracting value from a dataset that is inherently incomplete.

Meng: From an engineering standpoint, the implication here is that we can start building more useful AI models for applications like medical imaging or cybersecurity where getting those rare negative cases is just not feasible in practice.

Lalam: I think the real cultural impact lies in making AI development accessible to a much wider range of people and industries because it lowers the barrier for creating high-quality models without needing massive, perfectly balanced datasets from scratch.

Tom: Exactly, Lalam; it’s about democratizing model training by working around data limitations instead of waiting for perfect data collection.

Jane: And the authors really drive this home by showing how to build an empirical estimator that works directly with the densities we already have, which is a lot simpler than relying on complex negative sampling techniques.

Lu: That mathematical reformulation they did using only positive and marginal densities is what makes it so intriguing; it's a clever way to capture the essence of performance without needing the full label distribution upfront.

Meng: It’s interesting that the paper validated this across ten different real-world datasets, suggesting that these density-based methods aren't just theoretical exercises but have some traction in diverse data environments.

Lalam: That diversity is what makes me feel really optimistic about its future; if it holds up across such varied domains, we can start seeing more reliable AI tools deployed in specialized fields where data scarcity is a major bottleneck.

Tom: So, we’ve seen the core method for pAUC estimation and now we see how this work aims to make that estimation practical and robust for real-world use.

Jane: It’s a lot to take in, but the simplest way to put it is that they've given us a tool to build better AI when we don't have the full picture of what the negative instances look like.

Lu: The authors’ conclusion points toward this being a powerful direction for future research, suggesting that exploring how these density representations generalize across different complex data distributions is the next logical step.

Meng: I'm curious to see how this estimator scales when we move from smaller datasets to those massive ones we deal with in production environments; that practical scalability is something I’ll be watching closely.

Lalam: For me, the biggest implication is fostering a culture where researchers are encouraged to tackle real-world data constraints head-on, leading to more resilient and genuinely useful AI systems for everyone.

NTT, Inc.

cs.LG, cs.AI, stat.ML

Submitted: 2026-09-25

Updated: 2026-09-25

Importance score: 89/100

The gist: The proposed method addresses the challenge of maximizing partial area under the ROC curve (pAUC) from positive-unlabeled (PU) data, which is crucial in real-world applications like cybersecurity and

Key concepts

Partial AUC Maximization (pAUC): pAUC measures true positive rates within a specific range of false positive rates. It is used instead of standard AUC because it is more accurate when dealing with highly imbalanced datasets, such as those found in cybersecurity or medical applications where negative examples are scarce.
Positive and Marginal Densities: The core idea is to define the missing negative density using only the known positive class prior and the marginal density. This mathematical reformulation allows pAUC to be calculated directly from the available positive data without needing explicit negative labels or assumptions about them.
Empirical Estimator: The paper derives a specific formula (Eq. 14) for pAUC that depends only on positive and marginal densities. This estimator is derived from the original theoretical definition of pAUC, making it directly applicable to training a classifier using only positive-unlabeled data.

Terminology

AUC — 60× in this paper · explained in 2 episodes
Score Function — 20× in this paper · explained in 4 episodes
Adam — 5× in this paper · explained in 2 episodes
Core — 4× in this paper · explained in 2 episodes
Objective Function — 4× in this paper · explained in 2 episodes
Accuracy — 2× in this paper · explained in 1 episode
Consistency — 2× in this paper · explained in 3 episodes
Covariate Shift — 2× in this paper · explained in 2 episodes
Loss Function — 2× in this paper · explained in 2 episodes
Semi-supervised Learning — 2× in this paper · explained in 4 episodes
VC Dimension — 2× in this paper · explained in 2 episodes
Adaptation — 1× in this paper · explained in 1 episode
Alignment — 1× in this paper · explained in 1 episode
Benchmarking — 1× in this paper · explained in 3 episodes
Distribution Shift — 1× in this paper · explained in 6 episodes
Embeddings — 1× in this paper · explained in 2 episodes

Summary

The proposed method addresses the challenge of maximizing partial area under the ROC curve (pAUC) from positive-unlabeled (PU) data, which is crucial in real-world applications like cybersecurity and medical care where labeled negative data are often unavailable. The core contribution is a novel approach that reformulates pAUC using only positive and marginal densities, enabling classifier training without requiring negative labels.

Core Problem and Motivation

The paper focuses on maximizing the pAUC, which summarizes true positive rates within a specific range of false positive rates (FPRs), as this metric is more appropriate than the standard AUC in class-imbalanced scenarios. Existing methods for pAUC maximization typically require both labeled positive and negative data, but obtaining labeled negative data is often impractical due to privacy or annotation expertise requirements. The authors show that existing semi-supervised methods, which rely on heuristic assumptions (like ranking unlabeled instances), do not always coincide with the true pAUC. The paper aims to overcome this limitation by directly reformulating the pAUC objective using only positive and marginal densities, thereby deriving an empirical estimator from PU data.

Derivation of the Empirical Estimator

The method begins by representing the negative density, as defined in Eq. (8), based on the marginal density and positive class-prior:

p n(x) = 1 / (1 - π) [p(x) - πpp(x)]

By substituting this into the definition of pAUC (Eq. 5), the authors derive an expression for pAUC that depends only on positive and marginal densities, even including its FPR-dependent thresholds:

pAUC(α, β) = 1 / (β − α) [Exp∼pp Ex∼p [I(s(x p n) > s(x n)) I(s(x n) ∈ [tβ, tα])] - π Exp∼pp Ex¯p∼pp [I(s(x p n) > s(x¯p m)) I(s(x¯p m) ∈ [tβ, tα])]]

Crucially, the FPR-dependent thresholds, such as tα and tβ, are also represented using only positive and marginal densities (Eq. 10), which allows for a direct reformulation of the pAUC. This leads to an empirical estimator of the pAUC from PU data (Eq. 14).

Classifier Training via Smoothed Estimator

To make the derived empirical pAUC differentiable for training, the indicator function is replaced with a sigmoid function:

**pAUC(α, β) = 1 / (β − α)(1 − π) [X Np X N m=1 σ(s(x p n) − s(x m)) I(s(x m) ∈ [tˆβ, tˆα]) - π X Np X n!=m σ(s(x p n) − s(x p m)) I(s(x p m) ∈ [tˆβ, tˆα])] **

This objective function is then used as the loss function for training a classifier by maximizing pAUC, which is equivalent to minimizing 1 − pAUC. The training procedure involves sampling unlabeled data (size U) and positive data (size P), calculating thresholds tˆα and tˆβ using the sampled data (Eqs. 12 and 13), calculating the loss, and updating the score function parameters via gradient descent.

Theoretical Analysis of Consistency

The theoretical analysis establishes the finite-sample properties of the estimator in Eq. (14). By defining uniform estimation errors for marginal and positive score survival probabilities (∆N and ∆Np), they apply concentration inequalities, such as the Dvoretzky–Kiefer–Wolfowitz inequality, to bound these errors with high probability. This leads to bounds showing that the empirical pAUC estimator converges uniformly as N and Np approach infinity:

sup s pAUC(α, β) - pAUC s(α, β) = O [1 / (β − α)(1 − π) r log(M/δ) N + r log(M/δ) Np + 1 / Np!] with probability at least 1 − δ.

Experimental Validation and Performance

The method was experimentally validated across ten real-world datasets, including MNIST, FMNIST, SVHN, CIFAR10, SUSY, CODRNA, IJCNN, and EPSILON. The results demonstrate that the proposed method (Ours) consistently performed well or better than existing methods like NNPU (non-negative PU learning), PUAUC (AUC maximization with PU data), PAUC (pAUC maximization treating all unlabeled data as negative), and PUpAUCh (heuristic scoring assumption).

Improvements for AI systems

Here are the specific improvements that can be made to AI systems based on this scientific paper, along with what those improved systems can achieve:

)Improved System Capabilities:

Enhanced Performance in Class-Imbalanced Scenarios (pAUC Maximization):

The core improvement is moving from maximizing standard Area Under the ROC Curve (AUC) to maximizing the partial AUC (pAUC). This allows the AI system to explicitly optimize performance within a specific, critical range of False Positive Rates (FPRs) relevant to real-world constraints.

Robustness in Data-Scarce Environments:

The proposed method can train effective classifiers using only positive and unlabeled (PU) data, completely bypassing the need for difficult-to-obtain labeled negative data. This is crucial for domains like cybersecurity, rare disease diagnosis, or specialized medical imaging where negative examples are scarce or privacy-sensitive.

Fidelity to Specific Operational Constraints (Threshold Control):

The system can be trained to optimize a desired trade-off between True Positive Rate (TPR) and FPR within a defined window (the range from α to β). This means the AI doesn't just aim for good overall performance; it aims for good performance under strict risk limits (e.g., achieving 95% detection accuracy while ensuring the false alarm rate never exceeds 0.1%).

Adaptive Learning Under Class-Prior Uncertainty:

The method is robust against inaccurate estimates of the class prior (π). It can perform reliably even when the initial assumption about how many positive instances exist in an unlabeled batch is wrong, as long as the error in that assumption is not excessively large. This makes it practical for real-world deployment where prior knowledge might be imperfect.

Efficiency and Scalability:

The proposed training procedure (Algorithm 1) uses empirical estimators derived from PU data, which are computationally tractable. Furthermore, the theoretical analysis shows convergence bounds that depend on the size of the dataset (N and Np), suggesting that performance can be guaranteed as more data is collected.

)Specific Improvements for AI Systems:

System Type: Any binary classifier (Image classification, text classification, anomaly detection).
Training Paradigm Shift: Replace standard supervised learning with a PU-learning framework where the model learns the score function using only positive instances and unlabeled data.
Objective Function Modification: Instead of minimizing standard cross-entropy loss or maximizing global AUC, implement an objective that minimizes the deviation from the empirical pAUC estimator (Equation 15).
Threshold Determination: The system must dynamically estimate optimal classification thresholds, not just rely on a single fixed threshold, by using the derived PU-based threshold estimators (Equations 12 and 13) during training.

)What the Improved AI System Can Do:

The improved AI system can perform highly specialized tasks in real-world applications with high confidence regarding risk management:

In Cybersecurity: It can detect sophisticated malware (positive instances) while maintaining an extremely low False Positive Rate (FPR), ensuring that benign software or communications are not falsely flagged, thus minimizing operational disruption for end-users.
In Medical Diagnostics (e.g., Mammography): It can screen for rare microcalcifications with high sensitivity, but the system is explicitly tuned to ensure that the rate of unnecessary biopsies (false alarms) remains below a clinically acceptable threshold (e.g., FPR < 0.1).
In Advertising/User Experience: It can identify highly relevant advertisements while ensuring that irrelevant or intrusive ads are minimized, thus preserving a high user experience score despite operating in a class-imbalanced environment where positive interactions are rare.
In Privacy-Sensitive Domains: It can build reliable detection models in scenarios (like online session analysis) where obtaining truly negative examples is impossible due to privacy, leveraging the vast amount of unlabeled data available to train a high-quality detector.