Nord-Parl-TTS: Finnish and Swedish TTS Dataset from Parliament Speech

Text-to-speech (TTS) development is limited by scarcity of high-quality, publicly available speech data for most languages outside a few high-resource languages. Nord-ParlTTS presents an open TTS dataset for Finnish and Swedish based on speech found in the wild, addressing this resource gap between high- and lowerresourced languages.

The gist: Nord-ParlTTS is an open TTS dataset for Finnish and Swedish based on speech found in the wild, extracted from Nordic parliament recordings, resulting in 900 hours of Finnish and 5090 hours of Swedish TTS data.

Dataset Curation

The dataset is built using an adapted version of the Emilia data processing pipeline. For Finnish, the process involves several steps: first, extracting the audio track and standardizing it to a 24 kHz mono-channel format; then, separating it from noise using a pretrained UVR-MDX-Net Inst 34 model and performing speaker diarization using speaker-diarization-3.1 from Pyannote [15]. A fine-grained VAD is conducted on the diarized speech using Silero VAD [16]. To validate the transcript, an extra ASR model, specifically a Wav2Vec2-large model pretrained and fine-tuned solely on Finnish [17], is used to check the Whisper model’s transcript. The pipeline concludes with DNSMOS [18] prediction, where samples with a DNSMOS P.835 OVRL score less than 3.0 to be excluded.

For Swedish, the processing utilizes the RixVox-v2 data, involving resampling to 24 kHz and using the same speech separation model for noise cleaning. Utterances are cut based on existing sentence-level timestamps if gaps exceed 2 seconds, and speaker diarization is used to exclude utterances with multiple speakers. Intelligibility is evaluated by transcribing the utterance using Swedish Whisper-large [19] and filtering out those with a Word Error Rate (WER) larger than 10%. Finally, utterances are filtered based on the DNSMOS score, excluding those with a score less than 3.0, yielding the final 5090-hour Swedish dataset.

Evaluation Sets and Benchmarking

To support model development, the paper proposes unified evaluation sets following [1, 10]. For Finnish, they curate 500 prompt-target pairs from Perso Synteesi [11], ensuring a gender balance by sampling 250 prompts from male speakers and 250 prompts from female speakers. All prompt and target speeches are constrained to be between 3 seconds and 20 seconds with content having more than 10 characters.

For Swedish, the evaluation dataset is curated from CommonVoice [20] Swedish 22.0, keeping utterances between 3 to 20 seconds with non-empty gender labels. They sample a maximum of 30 utterances to create the pool of candidates per speaker, balancing the number of utterances per gender and per speaker. A native Swedish speaker then performs a bespoke curation tool to filter out speech with unclear speech, low volume, strong non-native accents, and background noise, before forming 500 prompt-target pairs.

Model Training and Objective Evaluation

The researchers train two non-autoregressive (NAR) TTS models: Matcha-TTS [21] and F5-TTS [1], both diffusion-based. For Matcha-TTS, they use monotonic alignment search (MAS) [22] to find optimal alignment, replacing the speaker embedding table with a pretrained SimAMResNet Speaker Encoder from WeSpeaker [25], and strengthening it with Classifierfree Guidance (CFG) [26]. The Finnish model uses character sequences directly due to its near one-to-one phoneme-to-grapheme mapping.

For F5-TTS, they use a stack of Diffusion Transformers (DiT) for implicit alignment. Both models are trained on 1 Nvidia A100 GPU for 500k updates with a batch size of 64. The Swedish model is trained using Phonemizer [27] for Swedish G2P. Objective evaluation involves synthesizing content and using ASR models to calculate the Character Error Rate (CER) between synthetic speech and ground truth content. They also calculate the Cosine Speaker Similarity (SIM) between synthetic speech and prompt speech, utilizing Wav2Vec2-large [17] for Finnish ASR and Swedish Whisper-large [19] for Swedish ASR, along with a WavLM-large-based speaker verification model for SIM.

Subjective Evaluation

For subjective evaluation, they invite participants recruited via Prolific7.

Here are the specific improvements that can be made to AI systems based on the Nord-Parl-TTS dataset and its associated research, along with what those improved systems could achieve:

The core contribution is a high-quality, in-the-wild TTS dataset for Finnish and Swedish derived from parliamentary speech. Improvements can focus on leveraging this data for cross-lingual generalization, robustness against noise/variability, and fine-grained control over prosody.

Here are the specific improvements:

1. ---

2. Extract highly robust, speaker-invariant representations from the Nord-Parl-TTS data (e.g., using the DNSMOS filtering and diarization techniques) to train more generalized TTS models that are less sensitive to acoustic noise or spontaneous speech variations encountered in real-world parliamentary recordings.

3. ---

4. Implement cross-lingual transfer learning mechanisms, specifically utilizing the unified evaluation sets for Finnish and Swedish TTS (as proposed in Section 1), to enable a single model trained on one language (e.g., Finnish) to achieve strong performance on the other (Swedish) by leveraging shared linguistic structures learned during pre-training.

5. ---

6. Integrate explicit prosodic control mechanisms derived from the analysis of duration and DNSMOS scores in the dataset to allow TTS systems to generate speech with more natural, contextually appropriate rhythm and emphasis, moving beyond simple text-to-speech towards expressive speech generation suitable for narration or synthetic dialogue.

7. ---

8. Leverage the speaker similarity metric (SMOS) from the evaluation phase to develop a speaker embedding system that can accurately clone or synthesize speech in a target language while preserving the unique vocal characteristics of a specific speaker, even when that speaker's voice is spoken in a parliamentary context.

9. ---

10. Develop specialized pre-processing pipelines (like the one detailed in Section 3) for handling low-resource languages by adapting established high-resource processing methods (e.g., adapting the Emilia pipeline modifications to other Nordic languages like Danish or Norwegian), thereby drastically lowering the barrier to entry for creating high-quality datasets for previously underserved linguistic groups.

11. ---

12. Optimize the diffusion model architectures (Matcha-TTS and F5-TTS) by incorporating explicit attention mechanisms that better capture sentence-level prosody, ensuring that the generated speech adheres strictly to the natural timing and phrasing observed in parliamentary discourse, leading to higher intelligibility metrics (lower CER).

The improved AI systems can achieve the following:

1. ---

2. The resulting TTS systems will generate Finnish and Swedish speech with significantly reduced errors (CER improvement) and better speaker fidelity compared to models trained on studio data alone, making them viable for real-time applications in news synthesis or digital assistants.

3. ---

4. A cross-lingual system will allow researchers to rapidly prototype high-quality TTS systems for new Nordic languages by transferring knowledge from established datasets (Finnish/Swedish) without needing to collect massive amounts of new, labeled data for that specific language immediately.

5. ---

6. The system can generate speech with nuanced emotional or narrative delivery, as the prosody modeling ensures the rhythm and emphasis align with natural human speaking patterns observed in complex discourse like parliamentary debate.

7. A high-fidelity voice cloning tool will be able to synthesize a target speaker's voice in either Finnish or Swedish with superior speaker similarity, enabling more realistic synthetic media creation for audiobooks or virtual assistants.

8. ---

9. The AI infrastructure will become significantly more democratized; researchers and small teams can build competitive TTS systems for low-resource languages by simply adapting the proven data processing pipeline, effectively closing the resource gap in speech synthesis accessibility across the Nordic region.

10. ---

11. The final synthesized speech output will exhibit superior temporal accuracy and natural flow, making it suitable for high-stakes applications where precise timing and intelligibility are critical, such as automated transcription systems or advanced voice interfaces.

Text-to-speech (TTS) development is limited by scarcity of high-quality, publicly available speech data for most languages outside a few high-resource languages. We present Nord-Parl-TTS, an open TTS dataset for Finnish and Swedish based on speech found in the wild. Using recordings of Nordic parliamentary proceedings, we extract 900 hours of Finnish and 5090 hours of Swedish speech suitable for TTS training. The dataset is built using an adapted version of the Emilia data processing pipeline and includes unified evaluation sets to support model development and benchmarking. By offering open, large-scale data for Finnish and Swedish, Nord-Parl-TTS narrows the resource gap in TTS between high- and lower-resourced languages.