2608.07285-How Much AI Is in This Track? Quantifying the Proportion of AI-Generated Stems in Hybrid Music Mixtures

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "How Much AI Is in This Track? Quantifying the Proportion of AI-Generated Stems in Hybrid Music Mixtures".

Jane: The paper was written by Fernando Garcia de la Cruz, David López-Ayala, Pablo Zinemanas, Emilio Molina and Martín Rocamora from Universitat Pompeu Fabra and BMAT Licensing S.L..

Tom: Stay tuned as we take you through the paper and discuss its implications.

Title: Tom: Hiya, Jane.

Jane: Tom — what a paper to kick things off with.

Tom: We've got a really good one today. Fernando Garcia de la Cruz and colleagues from UPF and BMAT in Barcelona.

Jane: They're asking the question that's been buzzing around the music industry for a while now — "How much eye is in this track?" And that's actually a much trickier question than it sounds.

Tom: Right, because most detectors out there give you a simple yes or no — eye or not eye. But the paper's point is that real production workflows are increasingly hybrid.

Jane: Exactly. A producer might get a synthetic drum loop, record live vocals, use a generated bassline — and now you're looking at a mix that's, say, forty percent eye.

Tom: That's the real world they're describing. Independent artists, bedroom producers, even top studios are blending generated stems with human performances.

Jane: And it's happening at scale. Industry reports cited in the paper say more than half of new uploads on major streaming platforms are fully eye-generated. So this isn't a niche problem.

Tom: The authors are essentially saying our detection tools are stuck in 2022 — binary, coarse, and ill-suited for the modern studio.

Jane: And the title captures that gap beautifully. It's not asking "is this eye?" — it's asking "how much?"

Tom: That's the fundamental reframe. From classification to measurement. And that shift has implications beyond detection — think royalty payout, copyright disputes, transparency for listeners.

Jane: Let's be real, that's where the industry is heading. Streaming platforms need to know what share of a track is eye-generated for compensation schemes.

Tom: Especially under regulations like the EU eye Act, which the paper references. There's a legal requirement to disclose eye content, but to do that you need to quantify it, not just label it.

Jane: And the authors are upfront about their approach. They're not claiming to have solved real-world deployment. They've built a controlled, reproducible methodology using codec artifacts.

Tom: Which is the smart, honest way to approach it. Set up a clean experimental framework, learn what's measurable, and iterate from there.

Jane: So we've got a paper that's asking a new question, and building the tools to answer it — I'm curious how.

Tom: Before we dig in, the obvious question — how exactly do you quantifiy "how much eye" is in a mix?

Jane: Good question. Actually the next bit explains it — they propose a formal definition, an "eye energy ratio." Let's get into it.

Summary: Jane: So last segment we sketched out the problem — mostly hybrid music, binary detectors — but I want to get into what this paper actually does mechanically.

Tom: Right. They transform the problem into something measurable. Instead of asking "eye or not," they define a quantity called the eye energy ratio, which they write as alpha.

Jane: Alpha is simply the fraction of a mix's total acoustic energy contributed by eye-generated stems. If alpha is 0, everything is human. If it's 1, everything is eye. Anything in between is hybrid.

Tom: The key innovation is how they create their test data. They take professionally produced multi-track recordings from MoisesDB — 240 tracks — and they reconstruct individual stems using EnCodec.

Jane: EnCodec is a neural audio codec. It's designed for compression, but it leaves a specific spectral fingerprint on the audio — those artifacts come from the transposed convolutions in its decoder.

Tom: So they take a human-performed stem, run it through this codec, and get what they call an "eye stem" that has the artifacts but retains the musical content.

Jane: And because it's the same musical content, they've removed a big confound — the eye and human conditions sound identical except for the artifacts.

Tom: Then they do something clever. For each track, they combinatorially generate every possible mix of real and eye-reconstructed stems. For a track with n stems, that gives 2^n possible configurations.

Jane: And each configuration has a computable alpha. You can sum the energies of the eye stems and divide by the total. So they get dense coverage of the whole range from 0 to 1.

Tom: They ended up with 21,212 unique mixes across the corpus. That's a solid experimental foundation.

Jane: Now the interesting results. They train a binary CNN detector using Afchar's method — the one that hits 99 point 97 percent accuracy on pure tracks. Then they show what happens when it encounters mixed content.

Tom: And this is where it gets fascinating. The binary detector was never trained on mixtures, but its output rises with alpha anyway. It acts like a noisy, miscalibrated estimator.

Jane: Miscalibrated is the word. Its median score stays around 0 point 05 until alpha crosses about 0 point 5, then jumps up to nearly 1 when alpha is above 0 point 9. So it massively underestimates eye content in the low and mid range.

Tom: Meaning, in the region most relevant to real production — say a track with 40 percent eye stems — a binary detector would confidently tell you it's human-made.

Jane: So then they train a regression model on the same architecture, but with the output trained directly on alpha. That model achieves a mean absolute error of 0 point 076 and an R-squared of 0 point 85.

Tom: Those numbers are promising. But I think the most revealing part is the instrument-level analysis.

Jane: Yes. The paper breaks down detection sensitivity by instrument. Drums and guitar — strongly detectable. Vocals — only partially. Bass — essentially invisible.

Jane: Because those stems carry different amounts of codec artifacts. Their fakeprint analysis — basically the spectral residue left by the codec — shows drums have clear separation from real content above about 2 point 5 kHz.

Tom: Guitar actually shows a crossover pattern — the eye-reconstructed guitar shows less energy in the low-mid range and more in the high frequencies. The artifacts don't look the same across all instruments.

Jane: And that's a crucial insight. The same classification of "eye content" carries very different detectability depending on the instrument.

Tom: So the summary is: yes, a regression formulation works much better than binary classification. But the underlying detectability is uneven.

Improvements and future work: Jane: We've established that the paper does solid work. But I want to talk about the improvements and future directions the authors propose — because that's where you see where this field is heading.

Tom: They're careful about their claims. They don't pretend to have a deployment-ready detector. This is a controlled foundation.

Jane: Right. They're transparent about the limitations — one codec, EnCodec, at one bitrate, 3 kbps, and the eye stems are reconstructions rather than actual generative-model output.

Tom: But the methodology itself is designed to be extensible. It's codec-agnostic, corpus-agnostic, and the library is released — they call it ARIA.

Jane: So anyone can take a different multi-track dataset, or a different codec like DAC, and reproduce the pipeline. That's a real contribution.

Tom: One concrete improvement they propose is instrument-aware detection. If you know the bass is hard to detect reliably, instead of detecting on the final mix, you separate it into stems and query each stem individually.

Jane: That's a meaningful next step. Because the paper shows that the same alpha can come from very different mixtures — 50 percent eye drums versus 50 percent eye vocals — with totally different detectability.

Tom: When you're at the stem level, you can tailor the detector to the instrument. Drums get one model, bass gets a different, more sensitive model.

Jane: They also mention the possibility of temporal localization. Since the models produce per-window scores, you could in principle track how eye content varies over time within a track.

Tom: And beyond that, they suggest finer granularity — detecting a single synthetic drum hit within a stem, rather than treating the whole stem as eye or human.

Jane: Which is going to require mixtures built at a finer scale than the stem. The paper notes that explicitly.

Tom: Another important direction — moving from codec reconstructions to actual generator output. The limitation right now is that commercial generators like Suno or Udio don't offer stem-level access.

Jane: That's a data access problem as much as a technical problem. And the authors propose two solutions — partnering with platforms that have stem catalogs, and using stem-conditioned generative models.

Tom: There's a real opportunity here. If you can generate a stem that fits human-performed material, then you have paired data — the human version and the generated version — and you can train on exact differences.

Jane: Which is a much more direct path to a detector that works in production.

Tom: Before we wrap, I want to look at the first page more closely. The abstract and introduction set up the whole framing — the industry stats, the regulatory pressure, the 97 percent figure.

Jane: Yeah, let's dig into that. The opening alone is worth a conversation — the fact that most people can't tell eye music apart from human music.

First page: Tom: Let's pull up the first page. The paper opens with those industry numbers — fully eye-generated tracks accounting for more than 50 percent of daily uploads on major streaming platforms. That's a striking claim.

Jane: It is. And then the survey result — 97 percent of listeners can't reliably distinguish fully eye-generated tracks from human-made recordings.

Tom: Those two numbers together set up a credibility problem for the industry. You have eye music flooding platforms, and listeners can't tell what's what.

Jane: That's why the authors anchor their work in the EU eye Act, specifically Article 50, which is about disclosure obligations. There's a legal need for this kind of technology.

Tom: And the gap they're pointing to is right there in the abstract — current eye music detection systems are binary. They treat tracks as either fully eye or fully human.

Jane: Which is precisely the frame the paper is pushing back on. The reality, as they say, is that producers are integrating synthetic drums, basslines, and vocals alongside human-performed instruments.

Tom: So you're getting a production ecosystem that is hybrid by default, while the detection systems operate on a false dichotomy.

Jane: There's a bigger pattern here — and the paper notes it explicitly. In deepfake detection, the field has already shifted from whole-image binary classification to localizing manipulated regions.

Tom: So it's not just a music problem. It's a general trend in eye detection — moving from "is it fake" to "where and how much is fake."

Jane: The mechanisms are analogous. But music has its own particular properties, especially the issue of artifacts being distributed unevenly across frequency bands.

Tom: I want to emphasize something from the first page that's easy to miss — the authors acknowledge that the eye stems in their study are codec reconstructions, not generator output.

Jane: That's a serious epistemic caveat. They're measuring the proportion of codec-generated energy under controlled conditions, not the proportion of "true" generative eye in a track.

Tom: But that's what makes the contribution defensible — they're building a measurement apparatus, not making inflated claims about real-world detection accuracy.

Jane: And as a research community, you need that clean foundation before you can do the messier, applied work.

Tom: The first page is really the seed of the whole paper. It defines the problem, hones the motivation, and sets up the practical constraints.

Jane: And it ends with a clear statement of contributions — the reformulation, the methodology, the characterization of binary detector behavior, and a regression baseline.

Tom: The regression baseline is the part I'm most impressed by, honestly. Not because it's flashy, but because it's principled. They benchmark the naive reuse of a binary score, and show it's dramatically worse.

Jane: It quantifies the hidden cost of binary thinking — a detector that's 99 point 97 percent accurate on pure tracks becomes worse than a random baseline on mixtures.

Tom: Right. And that's the kicker. People might argue "just use the score of a binary detector, it already encodes some information." And the paper says — no, it doesn't work.

Conclusion: Tom: Alright, let's pull the threads together. This paper asks a deceptively simple question — how much eye is in a track — and turns it into a well-defined regression problem.

Jane: They did this by defining the eye energy ratio, building a combinatorial mixing pipeline using MoisesDB and EnCodec, and then showing that codec artifacts in hybrid mixtures carry proportional information.

Tom: The core empirical finding is the binary detector, despite being near-perfect on pure tracks, becomes a noisy and miscalibrated estimator on mixed content.

Jane: At matched 5-second windows, the binary implicit estimate has an R² of minus 0 point 85. The regression model trained directly on alpha achieves an R² of 0 point 85. That's a stark contrast.

Tom: The instrument analysis is the other key contribution. Drums and guitar are highly detectable, vocals are weak, bass is essentially transparent.

Jane: Which tells us that an "eye content" label conceals real differences — you need to know which stems are eye, not just how much total energy they contribute.

Tom: The authors are careful about scope: one codec, one bitrate, and a controlled setup that doesn't include professional post-processing like EQ or compression.

Jane: But their methodology is a foundation, and they've released the library — ARIA — so future work can build on it. That's the right move.

Tom: And the path forward is clear — instrument-aware detection, fine-grained temporal localization, and eventually pairing with actual generator stems.

Jane: There's also a regulatory angle that's worth holding onto. The EU eye Act's Article 50 demands disclosure, and this kind of proportional measurement is exactly what would enable that.

Tom: So this paper doesn't just solve a research problem — it addresses an emerging legal and economic reality in the music industry.

Jane: And to be honest, it opens up the question for the rest of us — how do we think about musical authorship in a world where you can no longer say something is "AI," but rather "30 percent eye, from these stems, with these artifacts."

Tom: It changes the conversation from detection to measurement.

Jane: Now we're ready to move on to the next paper.

Tom: Let's get to it.