A PINK update: Improvements to the CELEBI fast radio burst data reduction and analysis pipeline

arXiv:2605.06766 · astro-ph.IM, astro-ph.HE · Submitted 2026-08-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Astrophysics Radio. Generated commentary on the latest astrophysics papers.

Vera: Next we'll be talking about the paper "A PINK update: Improvements to the CELEBI fast radio burst data reduction and analysis pipeline".

Jocelyn: The paper was written by M. Glowacki, T. Dial, A. Bera, A. T. Deller, K. Gourdji et al. from Institute for Astronomy at University of Edinburgh and International Centre for Radio Astronomy Research at Curtin University and Inter-University Institute for Data Intensive Astronomy at University of Cape Town and Centre for Astrophysics and Supercomputing at Swinburne University of Technology and ASTRON, the Netherlands Institute for Radio Astronomy and ARC Centre of Excellence for Gravitational Wave Discovery (OzGrav) and Australia Telescope National Facility at CSIRO and University of Rennes, INSA Rennes, CNRS and Center for Interdisciplinary Exploration and Research in Astrophysics (CIERA) at Northwestern University.

Vera: Stay tuned as we take you through the paper and discuss its implications.

Paper discussion segment 1: Vera: We have been talking about how difficult it can be simply finding these tiny flashes across such vast distances—what we call Fast Radio Bursts—but now we are looking specifically at how we process all those signals using this latest work entitled “‘А Pink Update’: ‘Improvements то thе СЕLЕВІ fаst rаdio burѕt dаtа rеductіоn аnd аnаlуѕіЅ ріреlіnе.” It’s quite an evocative title from authors like Marcin Glowackиi leading this group from Edinburgh down through various centers including Swinburne University hereerter в Australia и сaпеll іng together experts frоm several institutions worldwide."

Jocelyn: It really does feel less like some dry software manual because though “PINK” sounds almost poetic here it’Actually stands for 「Роlarizatiоn аnd асtrомеtrу Іmргоvementѕ fоr Νew Κnоwledge」 which tells us exactly whасk еxреrtise went intο creating thiс υрδαте."

Subrahmanyan: One thing researchers often overlook when reading titles liкe this is thе massive collaborative effort required behind тех соmmensal surveys mentioned hеre."

Vera: Exactly! You see names associated wiтh both large observatories lіke CSIRO via Thе Australiан SkА mорhine Рathfinԁer system АND university groups фrom South Africa тo Netherlands."

Jocelyn: And since thě target оf their work iś specialized heavily οnl у наLL localisatіоns,"It implies that wẹ aren’t juѕτ looking åt generic signal detecтиon bυт rather refining hөԝ incredibly preciсеly wę саn find шһere thěse events arę occurring within хоūr cosmic neighbors."

Paper discussion segment 2: Vera: Now having established who wrote this technical roadmap we need coոsider how much complexity lies beneath identifying even single pulses amidst all thosҽ cosmic background noises found described throughout “‘А Pink Update’: ‘Improvements то thě СЕLЕВІ fаst rаdio burѕt dาta rедucτιon અnd αναlysis рipεliне.” This isn’t just about seeing light—it’ς about turning raw antenna voltage signatures into meaningful astrophysical measurements regarding distance αnd orientation."

Subrahmanyan: That transition fraм vοltage dumps тto sub−arcsec precision positionings is nот trivial indeed because you are essentially trying ta pinpoint lightning strikes during thunderstorms while standing miles away!"

Paper discussion segment 3: Vera: Moving beyond identifying if something happened let’ʂ look specifically ætt whæt changed inside थis updated framework compared previously reported methods used vith older versions of ability to accurately map positions relative to host galaxies functionality implemented herᥱ too.";

Conclusion: Vera: As we wind up our deep dive today after exploring everything from improved astrometry errors до matched filter imaging techniques under current protocols summarized well ін «“А Pink Update”: ‘Improvements то ثhe СЕLEВІ fast radiो burstu دata रeduction अmd ανalphalysis pιpelιne,” іts clear ωe are entering а new era οf rapid transit detection capabilities involving higher resolution datasets than ever беforᥱ previous years might suggest my friends."—

M. Glowacki, T. Dial, A. Bera, A. T. Deller, K. Gourdji, A. Jaini, D. Scott, Y. Wang, K. Desnos, A. C. Gordon, R. L. Davies, R. M. Shannon

Institute for Astronomy at University of Edinburgh · International Centre for Radio Astronomy Research at Curtin University · Inter-University Institute for Data Intensive Astronomy at University of Cape Town · Centre for Astrophysics and Supercomputing at Swinburne University of Technology · ASTRON, the Netherlands Institute for Radio Astronomy · ARC Centre of Excellence for Gravitational Wave Discovery (OzGrav) · Australia Telescope National Facility at CSIRO · University of Rennes, INSA Rennes, CNRS · Center for Interdisciplinary Exploration and Research in Astrophysics (CIERA) at Northwestern University

astro-ph.IM, astro-ph.HE

Submitted: 2026-08-24

Updated: 2026-08-25

Comments: 13 pages, 6 figures. Accepted to PASA

Code: https://github.com/CIERA-Transients/POTPyRI

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 87/100

The gist: This paper presents the "PINK" update (Polarisation and astrometry Improvements for New Knowledge) to the CELEBI pipeline, a tool designed to reduce raw antenna voltages into fast radio transient

Key concepts

Fast Radio Bursts (FRBs)
These are tiny flashes across vast distances that researchers study. The paper focuses on improving the methods used to find and process these signals.
PINK
'PINK' stands for 'Polarization and Astrometry Improvement New Knowledge,' indicating the expertise behind this data update. It represents the specific improvements made to the data reduction and analysis pipeline.
Data Reduction and Analysis Pipeline
This is the process of taking raw antenna voltage signatures from FRBs and turning them into meaningful astrophysical measurements, such as distance and orientation, which is a complex transition.

Terminology

Summary

This paper presents the PINK update (Polarisation and astrometry Improvements for New Knowledge) to the CELEBI pipeline, a tool designed to reduce raw antenna voltages into fast radio transient detections and sub-arcsecond localisations. These improvements are critical because well-localised FRBs are tools for studying cosmology and the intergalactic medium, and high-time resolution polarimetric data is essential for testing of the numerous models on their potential progenitors.

Improvements to localisation

The update enhances astrometric precision by incorporating corrected Rapid ASKAP Continuum Survey (RACS) catalogues and supporting additional reference frames, including RACS-Low, RACS-Mid, RACS-High, and VLASS. To address systematic positional offsets, corrections were derived through crossmatching with the WISE catalogue. The pipeline now employs an improved source-selection procedure that applies three simultaneous filtering criteria to identify reliable reference sources:

  1. A compactness requirement based on the ratio of integrated to peak flux density (6";

  2. An adaptive minimum separation from neighbouring sources.

Additionally, the pipeline now estimates position uncertainties along the major and minor axes of the synthesised beam rather than just celestial coordinates, providing a more accurate representation of the true position uncertainty ellipse.

Matched filter imaging

To address noise issues in faint or high-dispersion measure (DM) transients, CELEBI has implemented matched filter imaging. This process utilizes high time resolution (HTR) Stokes I power dynamic spectra to construct an optimal time and frequency matched filter, which minimise[s] the noise added when imaging the FRB. This is particularly beneficial for low S/N FRBs and those with significant frequency structure.

The effectiveness of this method is demonstrated by FRB 20251019A, a high-DM event. By applying matched filter imaging, researchers achieved a reduction in the localisation uncertainty ellipse from 0.727/0.575 to 0.594/0.507, which "maximises the ability to localise currently rare high-redshift (z > 1) FRB candidates."

Polarisation calibration overhaul

The polarisation calibration process has been overhauled, resulting in much more accurate measurements of derived polarisation fraction and rotation measures. Using calibrators such as Vela and 1644, the pipeline models several instrumental effects to mitigate undesirable alterations to the measured signal:

  • Faraday rotation along the line of sight;

  • Polarisation leakage caused by imperfect digitisation (time and phase delay);

  • Coupling between Ex and Ey components in PAF edge and corner beams, which causes wave modes to become slightly elliptical.

By sampling the full parameter space through Bayesian inference, CELEBI can derive robust calibration solutions to produce accurate Stokes dynamic spectra.

Pipeline efficiency and portability

To handle the expected increase in detection rates from the CRACO upgrade, several software improvements were implemented. The SHRINE package was introduced to enable structure-maximisation of the dispersion measure, and the pipeline now uses Nextflow for workflow management. This has significantly reduced resource over-provisioning:

  • Execution time over-provisioning was reduced from 94% to 33%;

  • Memory footprint over-provisioning was reduced from 89% to 23%.

Finally, CELEBI has been transitioned to a containerised system using Docker. This ensures the pipeline is fully portable and can be easily installed on alternate computing systems, including distributed supercomputing infrastructure.

Improvements for AI systems

1. Predictive Resource Orchestration for Distributed Training/Inference

  • Improvement: Implement configuration-dependent resource allocation models within workflow management systems (e.g., Nextflow) using machine learning techniques like ANOVA and symbolic regression. Instead of static over-provisioning of CPU, GPU, and memory, the system uses execution traces from prior runs to predict the specific requirements of a task based on its input parameters (e.g., model architecture, sequence length, or dataset size).

  • Capability: This improved AI system can dynamically optimize supercomputing and cloud infrastructure utilization, reducing memory over-provisioning (by up to 70%) and execution latency (by 30-60%), significantly lowering the operational costs of large-scale model training and real-time inference.

2. Adaptive Spatiotemporal Gating for Signal-to-Noise (S/N) Enhancement

  • Improvement: Integrate time and frequency-domain gating mechanisms into neural architectures designed for signal processing. By utilizing optimal weights derived from the signal’s dynamic spectrum (matched filtering), the system can apply time-dependent and frequency-dependent weighting to input data streams.

  • Capability: The system can detect extremely faint, transient signals embedded in high-noise environments (e.g., edge-case detection in autonomous driving sensors or anomaly detection in high-frequency financial data) by focusing computational attention on the most informative time-frequency windows, effectively boosting the signal-to-noise ratio for low-magnitude events.

3. Cross-Modal Systematic Error Correction via External Reference Catalogs

  • Improvement: Incorporate a module for systematic astrometric/spatial error correction that cross-references real-time sensor data with high-precision external reference catalogs (e.g., using a catalogue lookup method to identify and correct systematic offsets).

  • Capability: This enables AI-driven navigation and localization systems (such as SLAM in robotics or autonomous drones) to achieve sub-centimeter precision by identifying and mathematically neutralizing systematic drift or sensor bias through real-time comparison with high-fidelity external spatial datasets.

4. Structural Complexity Maximization for Time-Series Analysis

  • Improvement: Replace standard signal-to-noise maximization loss functions with structure-maximization objectives, specifically utilizing the vector norm (2-norm) of the derivative of the intensity profile (sqrt sum(d/dt) squared).

  • Capability: This improved AI system can more accurately identify the true underlying parameters of complex, non-stationary time-series data (e.g., detecting subtle morphological changes in medical ECG/EEG signals or identifying precise onset timings in high-speed industrial sensor telemetry) by prioritizing the preservation of sharp, informative structural transitions over simple amplitude maximization.

5. Graceful Modality Degradation for Robust Multi-Modal Systems

  • Improvement: Implement generalized calibration and processing architectures capable of single-modality processing (e.g., single-polarization handling) that can produce pseudo-complete data products when input streams are interrupted or incomplete.

  • Capability: This creates highly resilient AI agents for robotics and autonomous systems that can maintain high-confidence decision-making and operational continuity even when specific sensors (e.g., LiDAR, depth cameras, or specific frequency bands) fail or provide partial data streams due to environmental interference.

Abstract

Fast radio bursts (FRBs) which are well localised (< 1") to their host galaxy are tools for studying cosmology and the intergalactic medium. Furthermore, high-time resolution datasets of their polarisation properties can enable testing of the numerous models on their potential progenitors. To that end, the CELEBI (CRAFT Effortless Localisation and Enhanced Burst Inspection) pipeline was conceived to enable data reduction from raw antenna voltages to detect fast radio transient events, localise them to sub-arcsecond precision, and produce polarimetric data at time resolutions as fine as 3 ns. Here we present a slew of updates to the CELEBI pipeline. Improvements to the astrometry correction for FRB localisations have aided our ability to determine what part of a galaxy more nearby FRBs have occurred in, which can have its own implication on the progenitor. We also have implemented time and frequency gating on detected fast transients to enable a boost to signal-to-noise, particularly useful for high dispersion measure or faint fast radio transients. We give examples of our improvements to the localisation, including for the currently 'hostless' FRB 20251019A. The polarisation calibration process has been overhauled, resulting in much more accurate measurements of derived polarisation fraction and rotation measures. Furthermore, we now have incorporated tools for structure-maximisation of the dispersion measure of fast radio transients, a software container which enables the installation of CELEBI on other machines, and improved the pipeline efficiency. Together these updates (named 'Polarisation and astrometry Improvements for New Knowledge', or PINK) greatly improve our ability to keep up with the expected detection rate from the CRAFT COherent (CRACO) upgrade to the real-time fast transient detection system of the Australian SKA Pathfinder.

Sources

Related papers