Event-driven eligibility propagation in large sparse networks: efficiency shaped by biological realism

Event-driven eligibility propagation in large sparse networks: efficiency shaped by biological realism presents a biologically plausible extension of the eligibility propagation (e-prop) learning rule for recurrent spiking neural networks (SNNs) by translating the time-driven update scheme into an event-driven one, demonstrating its applicability to neuromorphic MNIST and showing that biologically grounded constraints can inform the design of computationally efficient AI algorithms.

The gist: Event-driven updates reduce computation in sparse settings, improving e-prop scalability and enabling large-scale simulations by computing synaptic weight updates as asynchronously as events occur at the corresponding synapse.

Translating Time-Driven to Event-Driven Updates

The original e-prop algorithm was implemented with time-driven weight updates computed synchronously at each computational simulation time step (step). This approach wastes resources by ignoring the sparse nature of spiking activity, which is characteristic of biological networks where connectivity is sparse and neurons spike infrequently. The paper introduces an event-driven algorithm where synaptic weight updates are computed asynchronously and only when an event occurs at the corresponding synapse, exploiting this sparsity for computational advantage. This hybrid approach treats neurons and synapses as distinct entities: Spike communication and synaptic processing are event-driven, while neuron state updates are time-driven.

Mathematical Formulation of the Event-Driven Scheme

The core mechanism involves maintaining an eligibility trace that captures how filtered presynaptic spikes and postsynaptic surrogate gradients influence a synapse's contribution to learning. The update rule for recurrent weights is derived by combining the eligibility trace and the summed learning signal. Specifically, the gradient for recurrent weights is expressed as:

**/dL/dWji ≈ X T t=1 L t j Fκ ψ t j Fα z t−1 i − β aϵ a,t ji (Equation 33). **

Handling Temporal and Transmission Delays

The paper addresses the conflict between instantaneous transmissions assumed in the original derivation and empirical evidence of delays in neurobiological processes. To compensate for these delays, the implementation synchronizes histories of factors involved in weight updates. This is achieved by decoupling transmission delay from the simulation step for conceptual clarity, enabling shorter steps. For recurrent-to-output layer transmission delay (d), a readout-restricted derivative is introduced to account for temporal recurrences within the readout layer while excluding dependencies that propagate back into the recurrent network, allowing for cumulative computation.

Biological Realism Enhancements (e-prop+)

The model is extended with prominent biological features, resulting in the e-prop+ model. Key enhancements include:

1. Dynamic firing rate regularization: Replacing standard regularization with an exponential moving average to emphasize recent spikes and remove explicit dependence on time, making it more responsive to dynamic activity changes.

2. Classification via mean squared error: Employing temporal mean squared error instead of cross-entropy loss for classification tasks to avoid the extra communication step required by the softmax function.

3. Continuous dynamics: The original algorithm's reset mechanism after each weight update is removed, suggesting that they are not critical for maintaining learning performance.

4. Weight updates with every spike: A scheme inspired by truncated algorithms is introduced where each spike triggers a weight update based on the time since the previous spike, aligning with plasticity mechanisms like STDP.

Scalability and Performance Benchmarks

The event-driven e-prop models demonstrate significant scalability, showing super-linear strong scaling and near-ideal weak scaling up to millions of neurons. Performance was benchmarked on tasks such as neuromorphic MNIST (N-MNIST), where the event-driven e-prop model achieved successful learning with convergence speed matching the time-driven implementation. The paper also explores memory management through an auxiliary history vector that monitors weight update progress, ensuring the growth of history is inherently bounded by the number of synapses targeting the neuron.

Algorithmic Implementation and Optimization

The paper details several algorithms for weight updates, including Algorithm 3 (Event-driven weight update) and Algorithm 5 (Optimized event-driven weight update), which utilize a FIFO queue to manage eligibility trace buffering. The implementation utilizes an object-oriented architecture with specialized classes like EpropArchivingNode to manage the histories of surrogate gradients, learning signals, and firing-rate regularization for recurrent neurons versus error signals for output neurons. The Adam optimizer is used for weight updates, following TensorFlow conventions by reordering computations to ensure online gradient computation.

Future Directions in Plasticity Rules

The work provides a foundation for implementing reward-based e-prop and other three-factor learning rules in generic SNN simulations, suggesting potential candidates like Real-Time Recurrent Learning (RTRL) that are practical for neuromorphic hardware. Future work could extend validation to broader tasks, network architectures, and neuron parameters, aiming to advance machine learning for real-world applications by focusing on "biological plausibility and examining how biological constraints affect efficiency and performance.

Here are specific improvements that can be made to AI systems by leveraging the principles and techniques described in this scientific paper:

1. Improve energy efficiency and scalability of large-scale recurrent neural networks (RNNs) by implementing biologically plausible, event-driven eligibility propagation (e-prop) learning rules.

2. Enable the training of massive Spiking Neural Networks (SNNs) with millions of neurons without compromising learning performance, by utilizing event-driven weight updates that exploit the sparsity of spiking activity.

3. Develop AI systems that are inherently more energy-efficient by integrating continuous dynamics and strict locality constraints into recurrent network models, mimicking brain efficiency.

4. Enhance the robustness and generalization of supervised learning algorithms (like pattern generation and evidence accumulation) by incorporating biological features such as dynamic firing rate regularization (using exponential moving averages) instead of fixed time-dependent methods.

5. Create AI systems capable of handling complex, varying input durations by implementing weight updates with every spike, which allows for plasticity rules dependent on precise spike timing (STDP-like mechanisms).

6. Increase the biological plausibility and temporal fidelity of AI models by explicitly incorporating transmission delays between recurrent and output layers into the learning rule, allowing the system to model temporally lagged feedback mechanisms.

7. Improve classification performance on neuromorphic datasets (like N-MNIST) by employing biologically inspired loss functions (e.g., temporal mean squared error instead of standard cross-entropy) and flexible surrogate gradient functions (e.g., exponential surrogate gradient).

8. Create more adaptable and efficient AI architectures by decoupling the eligibility trace filter from the output neuron's time constant, allowing synapses to operate without needing explicit knowledge of downstream dynamics.

Despite remarkable technological advances, AI systems may still benefit from biological principles, such as recurrent connectivity and energy-efficient mechanisms. Drawing inspiration from the brain, we present a biologically plausible extension of the eligibility propagation (e-prop) learning rule for recurrent spiking networks. By translating the time-driven update scheme into an event-driven one, we integrate the learning rule into a simulation platform for large-scale spiking neural networks and demonstrate its applicability to tasks such as neuromorphic MNIST. We extend the model with prominent biological features such as continuous dynamics and weight updates, strict locality, and sparse connectivity. Our results show that biologically grounded constraints can inform the design of computationally efficient AI algorithms, offering scalability to millions of neurons without compromising learning performance. This work bridges machine learning and computational neuroscience, paving the way for sustainable, biologically inspired AI systems while advancing our understanding of brain-like learning.