An Open-Access Multi-modal Dataset for Cognitive, Motor, and Cognitive-Motor Tasks

arXiv:2603.22933 · q-bio.NC · Submitted 2026-03-24 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: Today's paper: "An Open-Access Multi-modal Dataset for Cognitive, Motor, and Cognitive-Motor Tasks".

Marcus: The authors present an open-access, multi-modal dataset integrating neurophysiological (EEG, fNIRS), physiological (ECG), behavioral, and subjective measures collected from 30 healthy participants across seven hierarchical cognitive and motor tasks.

Ines: First, who's behind it and why it matters.

Paper summary: Ines: So we've covered a bit about the setup for this open-access resource by looking at its structure and how it addresses previous limitations in the field seventeen <ref:2603.22933#pg1>. The main thesis of "An Open-Access Multi-modal Dataset for Cognitive, Motor, and Cognitive-Motor Tasks" is that existing neuroimaging datasets are too siloed, focusing on single modalities or isolated tasks in controlled settings <ref:2603.22933#pg0>.

Marcus: That's right; they argue that this lack of multi-task datasets restricts our ability to study how cognitive and motor processes actually interact during daily life activities, which is the gap this dataset aims to fill seventeen <ref:2603.22933#pg1>. They propose a continuum from lab-based single-task designs up through multitask and more naturalistic experiments <ref:2603.22933#pg1>.

Yuki: It's interesting because that continuum approach suggests a way to test the robustness and transferability of neurophysiological methods as we move toward more complex, real-world experimental designs <ref:2603.22933#pg1>. This connects the lab work to broader behavioral contexts.

Ines: And what they claim is that by providing this comprehensive platform integrating EEG, fNIRS, ECG, behavior, and subjective measures from thirty participants across seven hierarchical tasks—ranging from simple mental arithmetic to the full cognitive-motor integration—they enable testing multiple hypotheses about these interactions <ref:2603.22933#pg0>.

Marcus: That comprehensive nature is what makes it valuable; it's not just one piece of data, but a synchronized set that allows researchers to test relationships between different brain-behavior functional organizations <ref:2603.22933#pg1>. It’s about seeing the whole system together, not just isolated parts.

Yuki: And from a population genetics standpoint, having this level of integrated data allows us to potentially map out how these complex behaviors and cognitive functions are organized across individuals in a more holistic way seventeen <ref:2603.22933#pg1>. It moves beyond simple correlation into understanding functional organization.

Ines: So, the core message is that we need datasets like this to move past studying specific processes in detail and start exploring the complex relationships between those processes when they happen together <ref:2603.22933#pg0>.

Marcus: Exactly; it provides the necessary data foundation for multivariate analytical methods to get a better interpretability of these complex signals, which is what researchers are hoping to achieve with this resource fifteen <ref:2603.22933#pg1>.

Yuki: It’s about providing the raw material to challenge existing models of how our species organizes its cognitive and motor capabilities in dynamic situations seventeen <ref:2603.22933#pg1>.

Conclusion: Ines: Looking at the paper "An Open-Access Multi-modal Dataset for Cognitive, Motor, and Cognitive-Motor Tasks," the authors highlight that by creating this integrated resource across seven hierarchical tasks, they are offering a new way to test hypotheses about how cognitive functions and motor skills work together in real life scenarios <ref:2603.22933#pg0>.

Marcus: I think the implication for the genomics data science community is that we now have a standardized way to feed complex interaction data into our statistical models, which should allow us to build more sophisticated predictive models than we could with single-modality studies seventeen <ref:2603.22933#pg1>.

Yuki: From a population view, this resource suggests that our understanding of cognitive and motor organization in the species might be better informed when we can anchor findings to these rich, multi-modal behavioral and physiological data points seventeen <ref:2603.22933#pg1>.

Ines: Essentially, the authors are saying that to truly understand how we operate in dynamic environments, we need data that captures those simultaneous interactions between our thinking and our doing <ref:2603.22933#pg0>.

Marcus: And they've provided a publicly available dataset in BIDS format on OpenNeuro, which is a huge step forward for accessibility and reproducibility in this area seventeen <ref:2603.22933#pg1>. It makes this kind of research much more accessible to the wider community.

Yuki: That availability is key because it means other researchers, perhaps even those outside our immediate field, can use this data to explore how these complex processes manifest across different cognitive and motor profiles within a population seventeen <ref:2603.22933#pg1>.

Ines: It really comes down to providing the raw material for testing hypotheses about the real-world coordination of our minds and bodies <ref:2603.22933#pg0>. That’s the big picture here.

Marcus: And it's a testament to how combining different data streams—neurophysiology, physiology, and behavior—can lead to a more robust understanding of complex biological systems seventeen <ref:2603.22933#pg1>.

Zaineb Ajra, Gr´egoire Vergotte, St´ephane Perrey, Lilian Evra, Simon Pla, G´erard Dray

EuroMov Digital Health in Motion · IMT Mines Ales

q-bio.NC

Submitted: 2026-03-24

Updated: 2026-10-02

Comments: published in Scientific Data

Journal ref: Ajra, Z., Vergotte, G., Perrey, S. et al. An Open-Access Multi-modal Dataset for Cognitive, Motor and Cognitive-Motor Tasks. Sci Data (2026)

DOI: 10.1038/s41597-026-08125-y

Code: https://github.com/sccn/labstreaminglayer2https:

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 80/100

The gist: The authors present an open-access, multi-modal dataset integrating neurophysiological (EEG, fNIRS), physiological (ECG), behavioral, and subjective measures collected from 30 healthy participants

Key concepts

Multimodal Dataset: A collection of different types of data—like brain activity (EEG/fNIRS), heart rate (ECG), and actions (behavioral)—collected simultaneously from the same participants. This allows scientists to see how different physical and mental processes relate to each other.
Hierarchical Tasks: Seven experimental conditions were designed in increasing difficulty, starting with simple cognitive tasks like arithmetic and progressing to complex combined tasks that require simultaneous thinking and movement. This structure helps test hypotheses about how these processes interact at different levels of challenge.
BIDS Format: Brain Imaging Data Structure is a standardized, open format for storing neurophysiological data. Using this standard ensures that the raw EEG, fNIRS, and other measurements are organized consistently so that different research tools can easily access and analyze the data.

Terminology

fNIRS — 74× in this paper · explained in 2 episodes
Motor Imagery — 13× in this paper
Mental Arithmetic — 10× in this paper
Cognitive Load — 7× in this paper · explained in 2 episodes
Karolinska Sleepiness Scale — 5× in this paper
Lab Streaming Layer — 4× in this paper
Representational Similarity Analysis — 4× in this paper
Alignment — 3× in this paper · explained in 1 episode
Brain Imaging Data Structure — 3× in this paper
Interpretability — 3× in this paper · explained in 15 episodes
Multimodal Fusion — 2× in this paper · explained in 3 episodes
Robustness — 2× in this paper · explained in 5 episodes
Trace — 2× in this paper · explained in 2 episodes
Transferability — 2× in this paper · explained in 4 episodes
Adaptation — 1× in this paper · explained in 1 episode
Aggregation — 1× in this paper · explained in 1 episode
Benchmarking — 1× in this paper · explained in 3 episodes
Core — 1× in this paper · explained in 2 episodes
Multimodal Dataset — 1× in this paper · explained in 2 episodes
Pruning — 1× in this paper · explained in 4 episodes

Summary

The authors present an open-access, multi-modal dataset integrating neurophysiological (EEG, fNIRS), physiological (ECG), behavioral, and subjective measures collected from 30 healthy participants across seven hierarchical cognitive and motor tasks. This resource is significant because it addresses a gap in existing research by providing a comprehensive platform for analyzing the complex interactions between cognitive and motor processes in real-world scenarios.

The gist: The resulting, raw, multimodal psycho-neuro-physiological dataset - EEG, fNIRS, electrocardiogram (ECG), questionnaires, behavior - is publicly available in Brain Imaging Data Structure (BIDS) format [17].

Dataset Composition and Design

The dataset was constructed around a hierarchical series of seven tasks ranging from single cognitive and motor activities, such as N-back, motor imagery, to combined cognitive-motor interactions simulating real life scenarios. The structure starts from two cognitive tasks (N-back and mental arithmetic) and two motor tasks (passive arm movement and motor imagery), extending to combined conditions that integrate the following roots: N-back arithmetic, active motor, and N-back arithmetic combined with a motor task. This hierarchy allows users to test multiple hypotheses about cognitive-cognitive, motor, and cognitive-motor interactions across increasing levels of task difficulty.

Experimental Paradigm

Participants completed three sessions over which they performed seven different tasks in a randomized order. Each session lasted approximately 30 minutes, including both task and post-task questionnaires. The seven tasks included:

Mental arithmetic (MA): Involving simple additions or subtractions within the 0-9 range, targeting working memory processes.
N-back task (NB): A 2-back version used to impose a moderate level of cognitive workload.
Motor imagery task (MI): Participants imagined performing a movement without actual muscle contraction upon hearing a beep.
Passive motor task (Pass-Mot): Involving the arm being moved by a robotic dynanometer for 60 degrees of motion, with each movement lasting 2 seconds.
Active motor task (Act-Mot): Requiring participants to voluntarily move the Biodex robotised dynanometer arm upon hearing a beep.
N-back arithmetic task (NB-MA): Requiring participants to press a push button when the sum of auditory numbers corresponds to the 2-back condition.
Full task (NB-MA-Act-Mot): The most difficult condition, requiring participants to move their arm when the NB and MA conditions occur simultaneously, simulating real-world coordination.

Data Acquisition and Synchronization

Neurophysiological data were acquired using an integrated EEG-fNIRS system with a 32-channel EEG cap and Octamon systems for fNIRS, recording at 250 Hz for EEG and 10 Hz (or 50 Hz) for fNIRS. Physiological data included Electrocardiography (ECG) recorded at 1986 Hz to monitor cardiac activity, and behavioral data captured via a Biodex System 3 isokinetic device to measure torque during passive movements and voluntary movements. Subjective measures included the Karolinska Sleepiness Scale (KSS) and Likert ratings assessing perceived task difficulty. To ensure precise alignment, LabRecorder from the Lab Streaming Layer (LSL) framework was used to record all data streams, resulting in an XDF file containing synchronized streams.

Technical Validation and Analysis

The dataset underwent technical validation focusing on EEG and fNIRS signals. Subjective data analysis utilized a linear mixed-effects model to assess perceived task difficulty across conditions, showing that multitask conditions (NB-MA, Full) were rated as more demanding than single-component tasks. Neurophysiological validation employed Representational Similarity Analysis (RSA). The analysis compared neural RDMs to a theoretical RDM derived from subjective difficulty ratings. While the whole-head EEG RDM did not show a significant correspondence with the difficulty model, channel-level analyses revealed condition-related structure in the EEG data, indicating that EEG channels provide clearer task-difficulty-related structure than fNIRS channels under the current analysis settings. The dataset is organized according to BIDS standards and is freely available on OpenNeuro.

Data Availability and Usage

The dataset is organized according to the BIDS standard, providing raw data for each participant (sub-001 to sub-030) across three sessions (ses-01, ses-02, ses-03) for all seven tasks. Users are expected to apply their own analysis pipelines as No preprocessed (derivative) data are included. The structure includes modality-specific folders for EEG, fNIRS, and behavioral/physiological data. For example, raw EEG is stored in.edf files with associated sidecar files like 'events.tsv' detailing task events.

Improvements for AI systems

As a fastidious researcher, I have analyzed the provided paper, An Open-Access Multi-modal Dataset for Cognitive, Motor, and Cognitive-Motor Tasks. The core contribution is a rich, raw dataset combining EEG (neurophysiology), fNIRS (hemodynamics), ECG (physiological stress), behavioral measures (push button responses/motor performance), and subjective ratings across a hierarchical set of seven cognitive and motor tasks.

Here are the specific improvements to AI systems that can be made using this dataset, categorized by application:

)

AI Systems for Real-Time Cognitive Load Assessment & Monitoring: The system can be trained to predict the user's perceived mental effort (using Likert scale ratings as a ground truth target).
AI Systems for Neurofeedback and Closed-Loop Motor Control: The system can use EEG/fNIRS signals to provide real-time feedback during motor imagery or active motor tasks, allowing users to learn to modulate their brain activity to improve task performance or achieve specific cognitive states.
AI Systems for Personalized Neurorehabilitation & Injury Prediction: By analyzing the relationship between motor imagery/active movement tasks and physiological markers (ECG), the AI can identify patterns indicative of fatigue, stress, or impaired motor coordination, enabling personalized rehabilitation protocols tailored to individual neural signatures.
AI Systems for Multimodal Brain-Computer Interfaces (BCI) & Decoding: The system can be trained on the hierarchical task structure to decode complex cognitive-motor states.
AI Systems for Biomarker Discovery in Stress/Cognitive Disorders: By correlating specific EEG/fNIRS channel activity patterns with subjective difficulty and physiological stress (ECG), the AI can discover novel, non-invasive biomarkers that distinguish between healthy states and conditions like fatigue or anxiety.

Abstract

The incorporation of neuroimaging techniques such as electroenchephalography (EEG) and functional near-infrared spectroscopy (fNIRS) has provided new opportunities for the analysis of dynamic brain processes involved in cognitive and motor functions. Despite the great contribution of the open-access neuroimaging datasets to neuroscience studies, they have mainly remained on a single modality and isolated task paradigms performed in a controlled environments. These limitations restrict the analysis of multi-task effects in real-world applications, thus creating a gap in the understanding of how cognitive and motor processes interact in daily life activities. To address these limitations, we present a multi-modal dataset containing neurophysiological (EEG, fNIRS), physiological (ECG), behavioral, and subjective measures collected from 30 healthy participants over three sessions. This dataset includes a hierarchical series of seven tasks ranging from single cognitive and motor activities, such as N-back, motor, passive motor, mental arithmetic and motor imagery, to combined cognitive-motor interactions simulating real life scenarios. This raw dataset provides a resource for developing advanced preprocessing methods and analysis pipelines, with potential applications in brain-computer interfaces, neurorehabilitation, and other fields requiring an understanding of multi-tasks brain dynamics. https://doi.org/10.18112/openneuro.ds007554.v1.0.0