UltraLIF: Fully Differentiable Spiking Neural Networks
via Ultradiscretization and Max-Plus Algebra

Spiking Neural Networks (SNNs) offer energy-efficient, biologically plausible computation but suffer from non-differentiable spike generation, necessitating reliance on heuristic surrogate gradients. This paper introduces UltraLIF, a principled framework that replaces surrogate gradients with ultradiscretization, a mathematical formalism from tropical geometry providing continuous relaxations of discrete dynamics. The central insight is that the max-plus semiring underlying ultradiscretization naturally models neural threshold dynamics: the log-sum-exp function serves as a differentiable soft-maximum that converges to hard thresholding as a learnable temperature parameter ε → 0. Two neuron models are derived from distinct dynamical systems: UltraLIF from the LIF ordinary differential equation (temporal dynamics) and UltraDLIF from the diffusion equation modeling gap junction coupling across neuronal populations (spatial dynamics). Both yield fully differentiable SNNs trainable via standard backpropagation with no forward-backward mismatch. Theoretical analysis establishes pointwise convergence to tropical LIF dynamics with quantitative error bounds and bounded non-vanishing gradients. Experiments on six benchmarks spanning static images, neuromorphic vision, and audio demonstrate improvements over surrogate gradient baselines, with gains most pronounced in the ultra-low latency regime (T=1) on neuromorphic and temporal datasets. An optional sparsity penalty enables significant energy reduction while maintaining competitive accuracy.

Spiking Neural Networks are brain-inspired AI models that communicate through brief electrical pulses, making them far more energy-efficient than conventional AI. However, a fundamental training challenge has persisted as an open problem: the “spike” is like an on/off switch, and on/off switches have no smooth middle ground for standard learning algorithms to work with. Current methods work around this by teaching the network using a smoothed approximation of the spike during training, then switching to the real discrete spike at test time, creating a mismatch between how the model learns and how it actually runs. This inconsistency is a known limitation of virtually all existing SNN training methods.

This work introduces UltraLIF, a spiking neuron derived from first principles using a mathematical technique from physics called ultradiscretization. Rather than approximating the spike, the smooth trainable operation emerges directly from the neuron’s governing equation, making training and inference identical by construction. A temperature parameter, learned automatically during training, controls how sharp each spike is and adapts to the complexity of the task.

UltraLIF outperforms existing methods across six benchmarks, including spoken word recognition, moving-object detection from event-driven cameras, and standard image classification. It also scales to larger, deeper architectures where conventional spiking neurons permanently switch off and stop learning — a failure mode UltraLIF resolves through its self-adjusting firing sensitivity.