Why discretization matters

Mamba-3 landed this spring (Lahoti et al., ICLR 2026), from a team that includes the original Mamba authors. Its abstract lists three improvements, and the first is easy to read past: “a more expressive recurrence derived from SSM discretization.” Unpacked, it means something striking. Mamba-3 swapped its numerical integrator (the rule that converts a continuous-time model into the discrete steps a GPU actually runs), replacing a first-order update with a second-order trapezoidal rule.

There is no exotic neural machinery in that. The trapezoidal rule is textbook numerical analysis, and the theory that explains why it is the right upgrade was worked out by Dahlquist and the ODE community in the 1950s and ’60s. Yet in 2026 it sits at the center of a frontier sequence-model architecture.

This page is a short tour of why. The claim: for state-space models, discretization is not plumbing. It is a design axis, and stability, memory, and accuracy all hang off it.

A sequence model is an ODE in disguise

A structured state-space model (SSM) starts life as a linear ordinary differential equation:

The state $h(t)$ is a vector of memory; the input $u(t)$ is the signal — for a language model, ultimately the token stream. This is the same object control engineers have studied since the 1960s.

But a GPU does not see continuous time. It sees a sequence $u_1, u_2, \ldots$, so the model must be discretized: pick a step size $\Delta$ and a rule that turns the continuous matrices $(A, B)$ into discrete counterparts $(\bar A, \bar B)$, giving a recurrence

that approximates the flow of the ODE. That conversion rule is a numerical integrator, and everything in this story turns on which one you pick.

S4, the architecture that launched the modern SSM line, discretized with the bilinear transform — a rule we will meet properly below. Later SSMs, Mamba included, settled on discretizations built around the exact exponential $\bar A = e^{A\Delta}$, the zero-order hold (ZOH) and its close cousins: hold the input constant across each step and integrate the rest exactly. The recipe is the same either way, and one detail is crucial: $\Delta$ is a learnable parameter — the network tunes its own timescale by gradient descent. Mamba pushes this further; we will get there. First: what could possibly go wrong?

Stability is geometry

The simplest integrator anyone writes down is forward Euler: follow the local slope, $\bar A = I + \Delta A$. To see why it can fail, look at one mode at a time (take $A$ diagonalizable — every matrix in this story is). If $\lambda$ is an eigenvalue of $A$, the discretization maps it to an eigenvalue $\mu$ of $\bar A$, and the recurrence multiplies that mode by $\mu$ at every step. After a thousand tokens, that is $\mu^{1000}$. The discrete model survives precisely when $|\mu| \le 1$.

So the continuous and discrete worlds have different stability geographies. A continuous mode decays when $\operatorname{Re}(\lambda) < 0$ — the left half-plane. A discrete mode survives when $\mu$ lies in the unit disk. A discretization is a map between these two pictures, and the whole question is where it sends the stable region.

ZOH sends $\lambda \mapsto e^{\lambda\Delta}$, and $|e^{\lambda\Delta}| = e^{\Delta\,\operatorname{Re}(\lambda)} < 1$ whenever $\operatorname{Re}(\lambda) < 0$ — for every step size $\Delta > 0$. Decay in, decay out, unconditionally. Numerical analysts call such schemes A-stable; ZOH earns the label trivially, since $e^{\lambda\Delta}$ is the exact solution map and exactness preserves decay. Forward Euler instead sends $\lambda \mapsto 1 + \lambda\Delta$: a point marching along a straight line away from $1$. For any decaying mode there is a finite threshold (for real $\lambda$, $\Delta > 2/|\lambda|$) beyond which the image exits the disk, and the discrete model blows up even though the continuous system it approximates decays perfectly.

Small steps are safe for everyone. Trouble starts as $\Delta$ grows — drag it:

For SSMs this is not a hypothetical, because $\Delta$ is learned, and in Mamba it is computed per token from the input. Training explores $\Delta$-space freely. With an A-stable scheme, every $\Delta$ the optimizer can reach yields a stable recurrence: HiPPO initialization is built to place $A$‘s eigenvalues in the left half-plane, and an A-stable rule (bilinear for S4, the $e^{A\Delta}$ family for Mamba) keeps their images inside the disk. That stability is a theorem, not a lucky hyperparameter (ch. 4).

Operator stability is the same dichotomy

The slider above moved the time-step $\Delta$ and watched the discretization itself fail. The very same explicit-versus-implicit split lives one level in — inside the recurrence’s own update rule, on the learning step. Two modern sequence-model operators, DeltaNet and Longhorn, both run an online associative memory on a state $S$: each token nudges $S$ toward writing value $v$ at key $k$, one step of regression on a recall loss. The deviation from that fixed point contracts by a per-step factor $\rho_k$ in the key direction, and the two operators differ in exactly one place — whether that write step is taken explicitly or implicitly: forward Euler versus backward Euler again, now on the learning step instead of the time step.

DeltaNet writes with an explicit rate $\beta$, so it is forward Euler on the online-learning flow — safe for gentle writes, unstable once the write is too large:

Longhorn writes implicitly — it solves for the step instead of extrapolating it — which makes the effective rate $\beta^{\mathrm{eff}} = 1/(\alpha + \|k\|^2)$ shrink to meet the key. That is backward Euler’s A-stability guarantee, replayed:

Fix $\beta = \alpha = 1$ and both radii live on one axis — the write strength $\|k\|^2$. Drag it past $\|k\|^2 = 2$ and watch only the explicit operator leave the stable band:

The harder the write, the harder Longhorn contracts it ($\rho_k \to 0$ as $\|k\|^2 \to \infty$), so there is no boundary to cross. The two radii even obey a conservation law, $\rho_k^{\mathrm{Longhorn}} + \beta^{\mathrm{eff}}\|k\|^2 = 1$: whatever contraction Longhorn gives up, it spends on a self-limiting write, so the effective $\beta^{\mathrm{eff}}\|k\|^2 = 1 - \rho_k$ approaches but never reaches $1$ — not even half the explicit boundary of $2$, no matter how large the key. The closed forms, the Rayleigh-quotient drift guard that pins them to the materialized iteration matrix, and the chunkwise-parallel implementations are in ch. 12 of the book (runnable companion code).

Accuracy is a ladder

Staying stable is not the same as being right: the discrete model should also track the continuous one. The standard yardstick is the order of accuracy: error shrinking like $O(\Delta^p)$ as the step refines. ZOH is exact on the unforced system ($e^{A\Delta}$ is the true flow map), but it freezes the input across each step, which costs it accuracy on forced systems: first order, $p = 1$.

Climbing the ladder means letting the input vary across the step, and the classical way to do that is the trapezoidal rule: average the endpoints. In the SSM world it shows up in two guises:

• The bilinear (Tustin) transform: $\mu = \frac{1 + \lambda\Delta/2}{1 - \lambda\Delta/2}$, a Möbius transformation that maps the left half-plane exactly onto the unit disk. A-stable and second-order.
• The exponential-trapezoidal family: keep the exact $e^{A\Delta}$ for the state and apply the trapezoid only to the input integral. Exact on the unforced system and second-order on the forced one: ZOH’s best property and bilinear’s, simultaneously.

Selectivity is adaptive stepping

Through this lens, Mamba’s signature “selectivity” reads as something an ODE person already knows. Mamba computes $\Delta_t$ from the current token: the network runs an adaptive step size, the same idea every production ODE solver has used for decades. A small $\Delta_t$ barely advances the state (hold your memory); a large one integrates far (reset toward the new input); $\Delta$ is simultaneously the stability knob and the memory horizon. The discretization is the gating mechanism.

Which makes Mamba-3’s headline change read like the natural next move from the numerical-analysis playbook. The first-order update was the accuracy bottleneck (the paper pins its predecessors’ rule down as exponential-Euler: exact exponential for the state, Euler for the input), so it is replaced with a generalized trapezoidal rule — in its own terms an exponential-trapezoidal discretization: exact exponential for the state, trapezoid for the input, second-order and A-stable, the very ceiling Dahlquist’s barrier sets in the classical constant-step theory. Mamba-3 combines it with a second, independent upgrade, a complex-valued state, so each mode can rotate as well as decay (ch. 10 develops the scheme in full). The reported payoff is concrete, for the system as a whole: perplexity comparable to Mamba-2 with half the state size, plus gains on retrieval and state-tracking. The paper credits its three changes together; better numerics helped buy that memory.

The theory underneath (A-stability, order conditions, the barrier theorems, the equivalence results that tie consistency and stability to convergence) was finished before “sequence model” meant anything neural. None of this is my invention, and that is the point: the bridge between classical numerical analysis and modern architectures is short, load-bearing, and still being crossed. If this page pulled you across it, the book below walks the whole span, with derivations, exercises, and runnable companions.