2608.07359-Assessing AI-Generated Music Detection in Real-World Broadcast Monitoring

Introduction to the show: ident: Paper Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Assessing AI-Generated Music Detection in Real-World Broadcast Monitoring".

Jane: The paper was written by David López-Ayala, Fernando Garcia de la Cruz, Pablo Zinemanas, Emilio Molina and Martín Rocamora from Music Technology Group, Universitat Pompeu Fabra and BMAT Licensing S.L..

Tom: Stay tuned as we take you through the paper and discuss its implications.

Paper Summary: Tom: So today's paper comes from the Music Technology Group at Pompeu Fabra, in collaboration with BMAT Licensing, and it asks a very practical question: can we actually detect eye-generated music in real television broadcasts, not just in clean demo tracks?

Jane: And the answer, based on their experiments, is basically no — not reliably. The best detector they test gets an F1 score around 0 point 47 on real TV audio, even though the exact same kind of model scores above 0 point 99 on clean, isolated music.

Lu: That gap is the whole story of the paper. They built a new dataset called BAMM — forty hours of real TV recordings, split evenly between eye-generated and human-made music — and they show that current CNN-based detectors just don't hold up there.

Meng: The clever part is how they built the ground truth. They took clean reference tracks with known labels, matched them inside a global television archive using audio fingerprinting, and then extracted the actual broadcast clips. So the labels rest on tracks that genuinely aired on TV.

Lalam: And they were very strict about those labels. Human-made tracks had to predate Suno v3 point 5, the model that really kicked off high-quality generative music. For eye tracks, five separate detectors had to agree unanimously before the track was accepted.

Tom: That makes the poor results even more striking. A clean-trained model and a broadcast-trained model both degrade badly, and while the broadcast-trained one is clearly better, the score distributions for eye and human music overlap massively on real TV. The artifacts these models rely on just don't survive the broadcast chain.

Jane: This matters beyond academic curiosity. Deezer recently reported that half of its daily uploads are fully eye-generated, BMAT is seeing eye music creep into television, and the EU eye Act is pushing for transparency requirements. Everyone wants detectors, and this paper says the detectors aren't ready for the environment they'd actually work in.

Lu: It's also a warning about evaluation. Previous work used synthetically constructed broadcast mixtures, and this paper shows those simulations actually flatter the models. Real TV is harder.

Meng: So the contribution is double: a public benchmark that exposes the problem, and a demonstration that the field's evaluation practices have been too optimistic. If you want to deploy this in monitoring, you need to test on the real thing.

Lalam: And the authors keep their scope honest. They focus on one generative model family, Suno v3 point 5, because model-agnostic detection is still an open problem. That's a sobering statement about how far the field has to go.

Tom: So with that setup, let's look at how the paper builds the case, starting with the motivations on the first page.

Page 1 of the Paper: Tom: We've set the stage with the big result, so now page one shows us the problem from the industry's point of view. The paper opens with generative models lowering the barrier to music production and the legal fights that followed — the lawsuits against Suno and Udio, and licensing deals between generative platforms and rightsholders.

Jane: And there's a regulatory side too. The EU eye Act introduces transparency requirements so eye-generated content can be identified, and the paper ties that directly to the need for detection tools. Policy is creating demand for this technology.

Lu: Then come the numbers that show the scale. Deezer reported that fifty percent of its daily uploads were detected as fully eye-generated, and BMAT, the company behind this work, was reporting a growing presence of eye music in television recordings. This isn't a lab curiosity anymore.

Meng: The technical motivation is just as clear. Existing detectors score near-perfectly on isolated, high-fidelity tracks, but they're not robust to common audio transformations — random pitch shifting, time stretching, EQ, reverb. And broadcast adds far harsher conditions on top.

Lalam: Right, because television audio is a worst case. Music segments are short, they often sit in the background behind dominant speech or sound effects, and they go through transmission constraints. Earlier benchmarks had already shown degradation even in synthetic mixtures of all that.

Tom: So the gap the authors target is precise: nobody had evaluated eye music detectors on real broadcast content. That's where BAMM comes in, and they're upfront about the audio quality — monophonic, 8 kHz, encoded with AAC-LC at bitrates of at least 40 kbps. That's below consumer standards, but it reflects the low-bitrate proxy streams used in global industrial monitoring.

Jane: And the dataset reflects that industrial scale. The archive monitors more than 4,200 channels worldwide, and the clips run from five to sixty seconds, aired between January 2025 and March 2026. These are the practical constraints of the real deployment.

Lu: I appreciate that they call it a reference evaluation setting. The recordings are public on Zenodo, the code is on GitHub, so other groups can measure their models against the same real-world conditions.

Meng: And the way they built the labels — matching clean references to their broadcast occurrences through fingerprinting — is what makes the dataset trustworthy enough to serve as a benchmark. Without that, you'd just be guessing at what's eye and what isn't.

Lalam: So the page sets up an uncomfortable position: we have legal and policy pressure to identify eye music, we have detectors that shine in the lab, and we have zero evidence they work in the real target environment. The rest of the paper chases that evidence.

Tom: And the next section shows what the field had already learned about detection — and where those lessons stop applying.

Page 2 of the Paper: Tom: Picking up from that uncomfortable position, page two reviews what the field already knew about detecting synthetic audio, and it turns out the foundations are broader than just music.

Jane: The paper situates eye music detection alongside voice spoofing and synthetic speech detection, then zeroes in on the music-specific work. The key lineage starts with Afchar and colleagues, who showed that generative systems leave unintentional artifacts, and those artifacts can be picked up by CNN-based models, even when the audio goes through neural codecs like Encodec or DAC.

Lu: Then there's the SONICS dataset — songs generated with Suno and Udio matched against human-made tracks — and the SpecTTTra models, which achieve near-perfect performance on it. That's where the confidence in clean-condition detection comes from.

Meng: But the paper also cites work showing that confidence is fragile. Cros Vila and colleagues demonstrated that changes in sampling rate and bit rate can push detectors into relying on dataset-specific shortcuts instead of true eye artifacts. Near-perfect in-domain performance just doesn't generalize.

Lalam: The closest predecessor is the earlier work by the same group — López-Ayala and colleagues built eye-OpenBMAT, a synthetic broadcast benchmark that combines human production music with stylistically matched Suno continuations, then mixes in speech, transitions, dynamics changes, and low-quality encoding. That study showed substantial degradation in simulated broadcast conditions.

Tom: And that's exactly the gap this paper exploits. eye-OpenBMAT reproduces the duration patterns and loudness relationships of real TV, but the mixtures are constructed in a lab. The authors want to know whether those synthetic results actually predict what happens on real television.

Jane: The review also brings in useful neighbors: relative music loudness estimation, with the OpenBMAT dataset, and audio fingerprinting, with the BAF benchmark. Those fields describe how music actually appears in broadcast audio, and they provide the toolkit for finding it.

Lu: Right, and the fingerprinting angle is essential for what comes next. Without a reliable way to match a clean reference track to its broadcast occurrence, you can't build a real-world dataset at scale. So the related work is really the construction plan for the paper.

Meng: There's also a framing point worth making: music detection is part of the broader audio deepfake problem, but music is harder than speech. A single voice gives you one consistent signal to analyze, while music is layers of instruments with far subtler artifacts.

Lalam: So the literature gives us strong lab detectors, known fragility under simple transformations, and a synthetic broadcast benchmark that already shows trouble. The natural next question is whether real broadcast is even harder — and the next page answers it by showing how they built the dataset.

Page 3 of the Paper: Tom: So the dataset construction is where this paper really shows its craft, because labeling eye music in the wild is genuinely hard. Page three walks through the first stages of their pipeline.

Jane: They start with a pool of clean MP3 reference tracks. For the human-made class, they only accepted releases from January 2020 through December 2022, which guarantees everything predates Suno v3 point 5 and other public generative systems that could produce comparable quality. That closes the loophole of a "human" track secretly being synthetic.

Lu: For the eye class, they refuse to trust any single detector. They built an ensemble of five — the clean CNN from this paper, SpecTTTra, an in-house CNN, a logistic regression on artifact fingerprints, and an MLP on the same features. A track only becomes a candidate if all five unanimously agree it's eye.

Meng: All five were trained on clean foreground music and each scored above ninety-eight percent F1. Then the authors calibrated every model to achieve zero false positives on Da-TACOS, a verified human-composed set of five thousand tracks, plus a control set of five hundred eye tracks. That's a very conservative labeling bar.

Lalam: And there's a subtle consequence. Because they demand unanimous agreement, the eye-labeled tracks skew toward the clearly detectable ones. So the dataset should be easier than the real population of eye music — which makes the poor detection results later even more damning.

Tom: They also validate the labeling strategy with a temporal analysis. Plotting eye probability scores for tracks released between 2021 and 2026, scores stay low before Suno v3 point 5 appeared in summer 2024, jump up sharply after its release, and then drift downward as newer model versions come out. That pattern matches the real adoption timeline.

Jane: That's a beautiful sanity check. If the labels were noise, you wouldn't see such a clean alignment with actual model release dates. It confirms the pipeline is finding genuine eye-generated music in the wild.

Lu: Then comes retrieval. They use a landmark-based audio fingerprinting system to locate those clean references inside the broadcast archive, searching recordings that aired between January 2025 and March 2026. The matched segments are cut directly from the original TV emissions.

Meng: And a final filtering stage uses a deep music detector that classifies each segment as foreground music, background music, or speech. They keep only the music-labeled clips, which removes bad fingerprint matches and sound effects — and it gives them the foreground and background labels they use later when analyzing performance.

Lalam: So the whole pipeline is: curate trustworthy references, fingerprint them into the archive, filter by music detection. The result is forty hours of authentic broadcast audio with labels you can defend, and now the question is what happens when you train detectors on it — which is what the experimental setup on page four addresses.

Page 4 of the Paper: Tom: With the dataset in place, page four sets up the experiment as a clean comparison. The authors take the same CNN architecture and train two versions of it, changing only the training domain.

Jane: The architecture comes from Afchar and colleagues — six convolutional layers with filter sizes growing from sixteen to five hundred twelve, then average pooling and two fully connected layers. Inputs are five-second audio windows turned into mel-spectrograms from 8 kHz audio.

Lu: CNN Clean is the straightforward one. It trains on clean foreground music, with FMA-medium for the human class — about twenty-five thousand songs — and the Suno v3 point 5 subset of SONICS for the eye class, about nineteen thousand tracks. Each batch samples five random five-second snippets per track.

Meng: CNN Broadcast is where they try to build robustness. Same music sources, but seventy percent of the training samples are mixtures with speech from LibriSpeech, at signal-to-noise ratios anywhere from minus thirty to plus thirty decibels. The other thirty percent stay clean, and everything gets exported as mono 8 kHz MP3 at 40 kbps.

Lalam: So they're forcing the model to learn from degraded, masked audio from the start. That's a sensible response to the broadcast problem, and it will probably buy some robustness. But if the artifacts themselves get harder to see inside a real mix, training data alone won't solve it.

Tom: Then the evaluation forms a ladder with three rungs. The first rung, clean foreground music, uses the source material behind eye-OpenBMAT — four hundred seventy-six human-eye pairs at 22 kHz and high bitrate, with the eye tracks generated from their human partners using Suno's extend function, so style and timbre are closely matched.

Jane: The second rung is the full synthetic eye-OpenBMAT dataset — over three thousand tracks, nearly fifty-five hours of simulated broadcast content. And the top rung is BAMM itself, the real TV recordings with their authentic low-bitrate encoding. That ladder lets them isolate exactly where performance gets lost.

Lu: I like that the clean rung controls for musical content. Because each eye track is an extension of its human counterpart, the detector can't cheat by picking up on genre or production style — it has to use actual artifacts.

Meng: That makes the clean rung a meaningful baseline rather than just a formality. If the models fail there, something is wrong with the training. And if they pass there but fail higher up, the problem is the broadcast chain itself.

Lalam: Either way, the ladder is designed to expose where the domain gap lives, and the next page shows exactly how the models perform at each step.

Page 5 of the Paper: Tom: Now the results, and the numbers are stark. On clean foreground music, both models look excellent — F1 scores above 0 point 99 and ROC values above 0 point 996. So the architecture, the artifact signal, the training: all of that works in ideal conditions.

Jane: The synthetic broadcast rung is where things start falling apart. CNN Clean's F1 crashes from 0 point 992 to 0 point 342, with ROC at 0 point 909. CNN Broadcast holds up much better, with F1 at 0 point 661 and ROC at 0 point 926, which shows the mixed training did teach something about masking.

Lu: But real TV is a different world. On BAMM, CNN Clean drops to an F1 of 0 point 186 with ROC 0 point 707, and CNN Broadcast reaches only 0 point 472 F1 with ROC 0 point 775. The broadcast-trained model is clearly more robust, but neither is close to being reliable for a monitoring service.

Meng: The ROC curves add nuance. When they separate BAMM clips into foreground and background music, both models do better on foreground — CNN Broadcast reaches an AUC of 0 point 858 there — but background music drops it to 0 point 745. Masking is a major source of degradation, though clearly not the only one.

Lalam: And the score distributions show which way the errors go. Human samples mostly land near zero, so the models are good at rejecting human content. But lots of eye samples also land at low scores, especially in background music. The detectors are missing eye music, not crying wolf on human music.

Tom: That asymmetry really matters operationally. For a rights monitoring company, false positives on human music would be a nightmare, but false negatives mean silently letting eye content pass as if it were human. And this paper finds the false negatives dominate.

Jane: The comparison between the synthetic and real rungs is also a result in itself. STB overestimates performance relative to RTB, which means controlled mixtures don't capture the full complexity of actual broadcasts — the sound effects, the transitions, the diverse mixing strategies, the very low bitrate encoding.

Lu: So the paper's verdict is sobering: with current CNN-based detectors, whether trained on clean music or broadcast-oriented data, reliable eye music detection in broadcast monitoring remains out of reach. The domain gap is critical, not cosmetic.

Meng: And because the models retain decent ability on foreground music, it's tempting to think a better training set would fix things. But the authors suggest something deeper — the artifacts themselves become less visible under real broadcast conditions. That points toward different model families or different signal representations.

Lalam: Which naturally sets up the conclusions, where they draw out what this means for the field and for anyone trying to deploy these systems.

Conclusion: Tom: So let's wrap this up. The paper's contribution is BAMM, a forty-hour dataset of real television broadcasts, with clips that preserve authentic degradation — short segments, background music, speech, sound effects, and very low bitrate audio. And the labels are anchored to known tracks through fingerprinting and a five-detector ensemble.

Jane: The finding is that detectors degrade substantially as you move up the ladder from clean music to synthetic broadcasts to real TV. Broadcast-oriented training improves robustness, but not enough — the scores still overlap heavily between eye and human music in practice.

Lu: The deeper implication is that synthetic benchmarks have been flattering the field. eye-OpenBMAT already showed degradation, but BAMM shows more, which means simulated conditions systematically overestimate how ready our detectors are. That's a hard lesson for evaluation practice.

Meng: And the failure mode is specific: detectors miss eye music rather than flagging human music. If you're designing a monitoring pipeline, that tells you where the risk sits, and it suggests you'd need very different operating thresholds than lab evaluations imply.

Lalam: There's also a policy dimension. The EU eye Act is demanding transparency, and companies like Deezer and BMAT are drowning in eye content, so the demand for detection is real and immediate. This paper says the supply side isn't ready — and it gives the community a benchmark to close that gap.

Tom: The authors are also careful about scope. They focus on Suno v3 point 5 because model-agnostic detection remains open, and they don't pretend otherwise. That honesty makes the results more useful, not less.

Jane: So the short version for listeners is simple: eye music detection in clean audio basically works, and eye music detection in real television broadcasts basically doesn't. Now there's a public dataset that proves it and lets others work on the problem.

Lu: And the dataset and code are public — the recordings on Zenodo, the baseline code on GitHub. That means the next paper on this topic can be measured against the same real-world conditions, which is exactly how a field makes progress.

Meng: I'm curious what comes next. The results leave obvious directions open — larger models, different architectures, richer audio representations, maybe some way to model the broadcast context rather than just the musical signal.

Lalam: And if this benchmark gets adopted as a standard evaluation for eye music detection in the wild, then the paper will have changed how the problem is defined, not just measured. That would be a genuine contribution.

Tom: Good place to leave it. Thanks for joining us — we'll be back with the next paper on arXiv soon.

Jane: Until then, keep your ears open. That background music on TV might not be what it sounds like.