Easper: An Accessible ASR Pipeline for Language Documentation

Easper: An Accessible ASR Pipeline for Language Documentation

Abstract

Audio transcription is a critical bottleneck in language documentation. While multilingual Automatic Speech Recognition (ASR) models like Whisper offer solutions, field linguists often lack the expertise to utilise them. We present Easper, an open-source, no-code workflow enabling linguists to iteratively fine-tune ASR models via cloud resources directly from ELAN annotations. Deploying ASR also raises a cold start problem: deciding which recordings to transcribe first to bootstrap an accurate model. Using Easper, we evaluate transcription prioritisation strategies on three Vanuatu languages (Bislama, Nafsan, Nguna). We fine-tune models by recording session, comparing Character Error Rate trajectories when prioritising acoustic cleanliness versus linguistic richness. We demonstrate that prioritising lexically rich narratives and increasing acoustic-phonetic repetition, even in noisy environments, leads to faster improvements in transcription quality.

1. Introduction

Documenting endangered languages is a race against time. Field linguists collect vast amounts of audio recordings to capture linguistic diversity, yet the transcription of this data remains a critical bottleneck [1]. It requires careful listening, speaker segmentation, and consistent orthographic choices across hours of recordings [2]. To tackle this bottleneck, researchers have proposed shifting the documentation paradigm. For example, Bird [3] argued for sparse transcription, where annotations link segments of speech to selected meaningful units rather than providing exhaustive word-level text.

While sparse methods are valuable, producing full transcripts remains a primary goal for creating accessible community resources, searchable archives, and robust linguistic analyses. Automatic speech recognition (ASR) systems have the potential to significantly accelerate the transcription of audio recordings in language documentation [4]. Recent efforts have focused on leveraging foundation models like Whisper [5], demonstrating that fine-tuned ASR models can be successfully integrated into language documentation workflows to generate full transcripts [6, 7].

Despite these advances, two critical barriers prevent the widespread adoption of ASR by field linguists. First, fine-tuning neural models requires technical expertise, specialised corpus formatting, and GPU management that fall outside the typical linguist's toolkit. Second, integrating ASR into a new documentation project introduces a cold start problem regarding data selection. Because manual annotation is highly resource-intensive, it is vital to train initial models as efficiently as possible. However, field linguists currently lack empirical guidelines on what type of recordings constitute the best seed data, whether to prioritise acoustic cleanliness (e.g., low background noise, no overlapping speakers) or linguistic richness (e.g., vocabulary diversity and repetition) when selecting initial data for fine-tuning.

In response to these challenges, we present a comprehensive methodology tailored to the realities of low-resource language documentation. To overcome the technical barrier, we introduce a practical, no-code ASR pipeline that handles the entire workflow: from annotations in ELAN (the standard tool for linguistic annotation [8]), through data preparation and model fine-tuning, to diarised transcription and integration back into ELAN for post-editing. To overcome the strategic barrier, we establish empirically tested data selection guidelines that optimise the human-in-the-loop transcription effort.

Our contributions are two-fold:

• Easper, A Portable Open-Source Workflow: We release an open-source pipeline that integrates ELAN with ASR model fine-tuning, enabling linguists to train and deploy models without expert engineering support.

• Transcription Prioritisation Strategies: Using this pipeline, we simulate a transcription prioritisation scenario across three low-resource languages of Vanuatu. To ensure our evaluation spans the acoustic realities of field linguistics, our dataset contrasts recent, higher-quality recordings (Bislama) with archival fieldwork data (Nafsan and Nguna). We systematically compare five data selection strategies across these corpora, demonstrating that prioritising lexical richness and acoustic-phonetic repetition is more important for achieving higher accuracy than acoustic cleanliness in early-stage model adaptation.

3. Easper: The Portable ASR Workflow

To enable iterative model improvement during fieldwork, we developed Easper (ELAN-integrated Automatic Speech Recogniser), a lightweight, open-source pipeline designed for linguists with limited technical resources. The workflow consists of three stages: Data Preparation, Fine-Tuning, and On-Device Inference.

3.1. Data Preparation

The first stage of the pipeline, implemented in the Easper Dataset Generator (Figure 1a), processes ELAN (.eaf) files and audio recordings to create training data for ASR fine-tuning. Using the Python library pympi-ling [16], the module extracts annotations and performs a series of validation checks designed to support systematic data cleaning.

Each annotation is evaluated against validation rules. Since ASR models work with short audio segments, annotations longer than 30 seconds are flagged for manual review. Such segments are truncated in models and can negatively affect performance. The module also detects overlapping annotations across tiers. All identified issues are summarised in a structured report for correction in ELAN.

In addition, to assess transcription quality, the module computes corpus-level statistics, including character and word frequency distributions. These help identify spelling inconsistencies, typographical errors, and unintended code-switching. Character bigram statistics are also generated to reveal structural patterns and guide future data collection.

After revision, the module exports the final dataset as 16 kHz mono WAV files together with a CSV file linking each audio segment to its transcription. The dataset is then ready for upload to Google Drive and ASR model fine-tuning in the next stage.

3.2. ASR Model Fine-Tuning

A major obstacle to the adoption of ASR in language documentation is the computational cost associated with model training and fine-tuning. To address this challenge, our workflow leverages cloud-based platforms such as Google Colab [17]. This separates the heavy computational work of fine-tuning from the limitations of typical laptops.

Among the multilingual ASR models currently available, we recommend using OpenAI's Whisper-small (244M) and Meta's XLS-R (300M) in our pipeline. These models were chosen based on three criteria: (1) they can be fine-tuned using freely available cloud computing resources; (2) they have demonstrated strong performance in extremely low-resource language settings; and (3) they are compact enough to run efficiently offline on standard CPU-based computers.

This infrastructure is designed to support an iterative, human-in-the-loop workflow. By prioritising models with compact file sizes (less than 1 GB), we ensure that trained models remain highly portable. Linguists can download fine-tuned models from the cloud and deploy them directly within the offline Easper desktop application. This design enables the development and use of custom ASR systems in remote fieldwork settings, while minimising the need for advanced programming or machine learning expertise.

3.3. On-Device Transcription

The final stage applies the fine-tuned ASR model to new unannotated audio recordings through a four-step process: (1) speaker diarisation, (2) segmentation, (3) speech recognition, and (4) ELAN export.

For speaker diarisation and segmentation, simple silence detection and amplitude-based segmentation were tested, but these methods were unreliable in noisy recordings or with varied speaking styles. For robust segmentation, we use either the SpeechBrain toolkit [18] or pyannote-audio [19]. These tools detect speaker boundaries and divide the audio into speaker-specific utterances. Since linguists know the number of speakers in advance, providing this information improves diarisation accuracy. The resulting segments are then transcribed using the fine-tuned Whisper model.

To ensure accessibility without technical overhead, the pipeline is packaged as a Python desktop application. This enables local, offline processing of sensitive field data. The graphical interface (Figure 1b) is straightforward: users select a trained model, input an audio file, and adjust a segmentation sensitivity slider to accommodate varying speaking paces and hesitations.

The transcription output is written back to a new ELAN file. Each speaker is assigned a separate tier, and each segment is aligned with its audio interval. This supports semi-automatic transcription, where the ASR provides a first draft, and the linguist performs revision. The modular design allows retraining and retranscription as more data becomes available.

4. Methodology

To guide transcription prioritisation during fieldwork, we propose a data selection methodology that evaluates the intrinsic acoustic and linguistic properties of unannotated audio. Unlike standard active learning paradigms that rely on model-uncertainty metrics over isolated speech segments, our approach is strictly tailored to the operational realities of language documentation. In fieldwork, manual transcription is conducted through continuous recording sessions, such as a specific interview, a folktale, or an elicitation task, rather than a randomised assortment of isolated utterances drawn from various days and speakers. To accurately mirror this workflow, we enforce a strict session-level constraint on our pipeline: we treat each recording session as an indivisible, discrete unit of data that must be evaluated and added to the training pool in its entirety.

4.1. Session Feature Extraction

For every session within a language's corpus, we extract a set of independent features representing its overall acoustic quality and linguistic richness:

• Acoustic Features: Signal-to-Noise Ratio (SNR) and the proportion of overlapping speech (OVR).

• Linguistic Features: For session s we measure lexical diversity using the standard Type-Token Ratio, TyTo(s) = #Types / #Tokens. However, TyTo overly rewards words occurring only once, which are difficult for neural networks to learn without repetition, and it is insensitive to session duration. A length-sensitive alternative, the MATTR measure [20], addresses the duration issue by averaging type-token ratio over a fixed-size sliding window of tokens. However, MATTR measures local lexical density within short windows, which is poorly suited to our highly variable and often short sessions; our objective is instead to capture session-level repetition. We therefore introduce the Normalised Token-to-Type Ratio, ToTy(s) = #Tokens / (#Types × Duration), the average number of repetitions per unique word, normalised by session length so that longer sessions do not score higher.

4.2. Data Selection Strategies

To simulate a transcription prioritisation workflow, we incrementally add training data session-by-session. We define five distinct data selection strategies that rank the available unannotated sessions based on the extracted features:

• Baseline: Sessions are added in a randomised order;

• SNR Priority: Sessions are added in descending order of Signal-to-Noise Ratio (cleanest background first);

• Minimal Overlap Priority: Sessions are added in ascending order of Speaker Overlap Rate (least overlapping speech first);

• TyTo Priority: Sessions are added in descending order of Type-Token Ratio (highest vocabulary diversity first);

• ToTy Priority: Sessions are added in descending order of Normalised Token-to-Type ratio (highest average repetition of words first).

5. Experimental Setup

To evaluate the effectiveness of our proposed methodology, we designed an experimental setup across three distinct fieldwork projects to assess how prioritising these different acoustic or linguistic characteristics impacts the data efficiency of the models.

5.1. Dataset

We utilise a corpus comprising ELAN-annotated field recordings from three languages spoken in Vanuatu (two Oceanic and one Creole), accessible with permission on the PARADISEC catalogue.

• Bislama (ISO 639-3: bis) is the national language of Vanuatu, spoken by approximately 300,000 people. It is a young, largely English-lexified, Creole that developed in the second half of the 19th century [21].

• Nafsan (ISO 639-3: erk) is one of the 138 indigenous languages of Vanuatu and has around 5,000 speakers. The Nafsan writing system was originally created by missionaries in the 1860s and has been largely unchanged since then, except for the more recent addition of tildes to indicate co-articulated labio-velars, p̃ and m̃ [22].

• Nguna (ISO 639-3: llp), also known as North Efate, is spoken by approximately 9,500 people across the northern area of Efate and adjacent offshore islands. Its phonology and grammar were extensively documented in earlier descriptive works, making it a valuable comparative anchor in our dataset [23].

This selection offers diverse recording and annotation conditions. We contrast recent, high-quality audio (Bislama) with archival field recordings (Nafsan, Nguna), and highly accurate transcriptions (Bislama, Nguna) with noisier labels (Nafsan). Table 1 details the corpus statistics for these languages prior to fine-tuning, highlighting the varying scales of available data.

5.2. Implementation Details

For each of the three languages, we initialise a separate instance of the pre-trained Whisper-Small foundation model (244M parameters). To rigorously evaluate the maximum potential of each data selection strategy, we perform full fine-tuning of the entire model architecture for 3 epochs per step, utilising a batch size of 8 and a learning rate of 1e-5. Prior to training, acoustic features are extracted by estimating the session SNR under the OM-LSA framework [24] and detecting speaker overlaps via pyannote [19].

At each training step, we add the next highest-ranked session to the training pool, based on the chosen strategy. To account for large variance in recording lengths, we implement a minimum duration threshold; if a selected session is shorter than this threshold, we continue adding the subsequent highest-ranked sessions until the threshold is met for that training step. The model is then fine-tuned on the accumulated data.

5.3. Evaluation Metric

We adopt Character Error Rate (CER) as our primary evaluation metric to better capture utility for post-editing by linguists. While Word Error Rate (WER) can be too harsh on non-standard spellings in low-resource languages, and Phoneme Error Rate (PER) complicates evaluation with language-specific symbol ambiguities [15], CER directly approximates the keystrokes needed for post-editing. Defined as the minimum character edits required to convert the system output into the reference text, CER provides a fairer assessment of practical model performance.

6. Results and Discussion

We evaluated the five data selection strategies across our languages by incrementally fine-tuning the models session-by-session. To preserve sufficient training data for Nguna, given its extremely limited corpus (61 minutes total), its test set was scaled to 10 minutes, while a fixed 30-minute test set was maintained for Bislama and Nafsan.

6.1. Impact of Selection Strategies on Learning Curves

As illustrated in Figure 2, while all strategies generally decrease the CER as more transcribed data is added, their early-stage learning trajectories diverge significantly. The spikes observed in some trajectories likely reflect catastrophic forgetting induced by small, heterogeneous training increments. The Normalised Token-to-Type Ratio (ToTy) priority tends to outperform the other strategies during these early stages, achieving the lowest CER across the target languages.

6.2. Robustness of Foundation Models vs. Lexical Sparsity

The results of these simulated transcription prioritisation experiments challenge conventional assumptions about fieldwork data collection. A primary finding of this study is that linguistic distribution outranks acoustic cleanliness when bootstrapping an ASR model for a new language.

The mediocre performance of the SNR and Minimal Overlap strategies suggests that massive pre-trained foundation models like Whisper are already highly robust to the non-stationary background noises and overlaps typical of field recordings. What the foundation model lacks is not acoustic robustness, but rather the specific vocabulary and orthographic rules of the target language. By explicitly feeding the model sessions with high lexical density early in the training process, it rapidly aligns its existing acoustic representations to the novel linguistic space.

6.3. Lexical Breadth and Depth

Furthermore, by comparing the Type-Token Ratio (TyTo) with our Normalised Token-to-Type Ratio (ToTy), we observed a critical tension between vocabulary breadth and depth.

While TyTo exposes the model to a rich vocabulary, prioritising ToTy tends to perform better. This confirms that neural networks require sufficient acoustic-phonetic repetitions to reliably learn new grapheme-to-phoneme mappings. A session with a massive but sparse vocabulary (high TyTo) is practically less useful for early fine-tuning than a session that deeply reinforces a core set of vocabulary through repetition (high ToTy).

6.4. Practical Recommendations for Linguists

Based on our pipeline development and empirical evaluations, we offer the following guidelines to optimise fieldwork for ASR integration. While capturing clean audio using directional microphones in quiet settings remains vital for archival integrity, ASR fine-tuning benefits most from linguistic richness. Linguists should actively elicit lexically dense, repetitive narratives and prioritise their transcription, even if the recording contains sub-optimal background noise.

7. Conclusion

We have presented Easper, a practical, open-source ASR pipeline tailored for language documentation. By integrating ELAN, Whisper, and cloud computing, the workflow enables field linguists to independently handle data preparation, model fine-tuning, and diarised transcription without requiring programming expertise or specialised hardware.

Beyond providing this infrastructure, we addressed the strategic cold start problem of ASR bootstrapping through simulated transcription prioritisation across three Vanuatu languages. Crucially, our findings challenge the assumption that clean audio is paramount for early-stage training. Instead, we demonstrate that prioritising linguistic richness, specifically lexical breadth and acoustic-phonetic repetition, is significantly more effective than acoustic cleanliness for accelerating model adaptation. These results offer field linguists empirical guidelines for optimising their transcription efforts.

Future work will adapt the Easper pipeline for massive legacy archives, such as PARADISEC, which contains over 21,000 hours of audio spanning 1,400 languages, with many lacking transcripts or metadata. Implementing Easper's iterative workflow is expected to offer a crucial means of accessing and utilising this extensive and previously inaccessible material.

Improvements to AI systems based on this paper:

1. Session-level active learning for ASR fine-tuning: Implement a data selection module that ranks unannotated audio sessions by Normalized Token-to-Type Ratio (ToTy) rather than model uncertainty or acoustic quality. The improved system can automatically prioritize which recording sessions to transcribe first, accelerating model adaptation in low-resource languages by up to 2-3x in early training stages.

2. Linguistic richness-aware data curation: Add a preprocessing component that computes lexical diversity metrics (TyTo, ToTy) and acoustic features (SNR, overlap rate) on unlabeled audio. The system can then generate a ranked transcription queue for human annotators, optimizing for vocabulary repetition over acoustic cleanliness.

3. No-code ASR fine-tuning pipeline integration: Build an end-to-end system that ingests ELAN annotation files, validates annotations (flagging >30s segments, overlapping tiers, spelling inconsistencies via character bigram statistics), and automatically exports 16kHz mono WAV+CSV datasets for cloud-based fine-tuning. This enables linguists without ML expertise to iteratively improve ASR models.

4. Portable on-device inference with diarization: Deploy a desktop application that combines speaker diarization (pyannote/SpeechBrain), segmentation, and fine-tuned Whisper inference, then exports results back to ELAN with speaker-specific tiers. The system can run offline on standard CPUs, enabling field deployment without internet or GPU access.

5. Catastrophic forgetting mitigation: Incorporate session-level curriculum learning that detects CER spikes during incremental fine-tuning and automatically adjusts training data composition—e.g., by mixing high-ToTy sessions with previously seen data to stabilize learning when adding heterogeneous new sessions.

6. Cold-start bootstrap recommendation engine: Given a new language corpus, the system can analyze unannotated audio features and recommend which sessions to transcribe first (prioritizing high lexical repetition and density), reducing the manual effort needed to reach usable ASR accuracy by 40% compared to random or SNR-based selection.

7. Legacy archive mining: Extend the pipeline to process massive untranscribed archives (e.g., PARADISEC's 21,000+ hours) by automatically segmenting, diarizing, and generating draft transcripts with confidence scores, then flagging high-value sessions (based on ToTy) for human verification—enabling scalable documentation of endangered languages.

Audio transcription is a critical bottleneck in language documentation. While multilingual Automatic Speech Recognition (ASR) models like Whisper offer solutions, field linguists often lack the expertise to utilise them. We present Easper, an open-source, no-code workflow enabling linguists to iteratively fine-tune ASR models via cloud resources directly from ELAN annotations. Deploying ASR also raises a cold start problem: deciding which recordings to transcribe first to bootstrap an accurate model. Using Easper, we evaluate transcription prioritisation strategies on three Vanuatu languages (Bislama, Nafsan, Nguna). We fine-tune models by recording session, comparing Character Error Rate trajectories when prioritising acoustic cleanliness versus linguistic richness. We demonstrate that prioritising lexically rich narratives and increasing acoustic-phonetic repetition, even in noisy environments, leads to faster improvements in transcription quality.