Entropy Production in Nonequilibrium Neural Networks

Mon, 02 Feb 2026 09:28:17 +0100

Introduction

✨ GitHub repository: (work in progress)

Transformers are powerful driven dynamical systems, yet their internal computation is rarely discussed in terms of nonequilibrium thermodynamics. Building on , we design a minimal parallel transformer-like module whose forward pass implements a controllable quench-and-relax process. We characterize its dynamical regimes to elucidate a design space of stateless and stateful variations of transformer-like and deep-equilibrium-like architectures and leverage its physics-based architecture to compute differentiable proxies for entropy production. This provides both a scalable laboratory for nonequilibrium many-body dynamics on hardware accelerators and a testable learning hypothesis.

We ask whether module-local ascent on a housekeeping entropy-production proxy, under bounded dynamics and structured input streams, can lead a system to acquire structure-sensitive, predictive dynamics without an externally supplied task loss or end-to-end credit assignment. The risk is that the system finds local dissipative shortcuts: asymmetric attention collapse, self-exciting cycles, or coupling to noise. The bet is that, once trivial dissipative shortcuts are bounded or exhausted, persistent temporal structure provides the most reliable support for continuing asymmetric delayed flow. We readily admit that the main motivation for this bet is aesthetic. To move beyond aesthetics, we run numerical experiments.

Driving a spin-transformer module

In this section we design a minimal spin-transformer module whose forward pass implements a controllable nonequilibrium quench-and-relax process. We identify three timescales and two dynamical regimes, leading to a natural categorization of the design space into stateless and stateful variations of transformer-like and deep-equilibrium-like architectures.

A minimal controllable drive-conditioned system

In we showed how to apply dynamical mean-field theory to approximate the time-dependent behavior of asymmetric vector-spin models. We started from a spin system of $N$ vector spins $\mathbf{s}_{i,t} \in \mathbb{R}^{D}$ talking to each other via an $N \times N$ pairwise coupling matrix $J_{ij}$ with the underlying parallel-updates stochastic dynamics characterized by a discrete-time Markov chain transition probability $P(\mathbf{s}_{t} | \mathbf{s}_{t-1})$. External magnetic fields $\mathbf{x}_{i,t} \in \mathbb{R}^{D}$ bias the vector spins and act as local drives. We will use the notation $\mathbf{x}_{t} = \{ \mathbf{x}_{i,t} \}^{N}_{i=1}$ to refer to the full-drive matrix instead of the local drive at a single position.

Using a simple first-order Plefka[t-1,t] mean-field approximation, we calculated a closed expression for updating the spin expectation values in the large-vector-dimension limit,

\begin{equation} \mathbf{m}_{i,t} = \frac{\beta \left( \mathbf{x}_{i,t} + \sum_{j} J_{ij} \mathbf{m}_{j,t-1} \right)}{1+\sqrt{1+\beta^2 \lVert \mathbf{x}_{i,t} + \sum_{j} J_{ij} \mathbf{m}_{j,t-1} \rVert^2 / R^2 }}, \end{equation}

where the magnetization vectors $\mathbf{m}_{i,t} \in \mathbb{R}^{D}$ capture the mean-field influence of the spins on each other. Additionally, $\beta$ denotes the inverse temperature and $R=\sqrt{D/2 -1}$ is the natural hyperspherical length scale resulting from the large-vector-dimension approximation¹.

If we now consider some kind of parameterized drive-dependent couplings

\begin{equation} \mathbf{J} (\mathbf{x}_{t}) = \mathrm{softmax}\left( \mathbf{x}_{t} \boldsymbol{W}_{Q} \boldsymbol{W}_{K}^{T} \mathbf{x}_{t}^{T} \right), \label{eq:softmax} \end{equation}

then we turn the fixed-size $N \times N$ coupling matrix into a parameterized rule that supports variable system size, drive-dependent routing, and a way to scale system size without learning new explicit parameters². If we also augment the local drives with some kind of parameterized non-linear drive-dependent field,

\begin{equation} \mathbf{x}_{i,t} \to \mathbf{x}_{i,t} + \mathbf{FFN}\left( \mathbf{x}_{i,t} \right), \end{equation}

where $\mathbf{FFN}$ denotes a position-wise feed-forward network, then our earlier recurrence relation becomes

\begin{equation} \mathbf{m}_{i,t,k+1} = \frac{\beta \left( \mathbf{x}_{i,t} + \mathbf{FFN}\left( \mathbf{x}_{i,t} \right) + \sum_{j} J_{ij} (\mathbf{x}_{t}) \mathbf{m}_{j,t,k} \right)}{1+\sqrt{1+\beta^2 \lVert \mathbf{x}_{i,t} + \mathbf{FFN}\left( \mathbf{x}_{i,t} \right) + \sum_{j} J_{ij} (\mathbf{x}_{t}) \mathbf{m}_{j,t,k} \rVert^2 / R^2 }}, \label{eq:paralleltransformer} \end{equation}

where we have pushed down the recurrence into an internal relaxation index $k$. By making the effective drive as well as the couplings depend on the drive $\mathbf{x}_{t}$, a sudden shift $\mathbf{x}_{t} \to \mathbf{x}_{t+1}$ changes both the local fields as well as the interactions and quenches the module into a new instantaneous dynamics³. During internal relaxation updates $k$, the drive $\mathbf{x}_{t}$ and the parameters $\boldsymbol{\theta} = \{ \mathbf{W}_{Q}, \mathbf{W}_{K}, \mathbf{FFN} \}$, and therefore the transition rule, are held fixed. (At the stochastic level, the process looks something like $P_{\boldsymbol{\theta}, \mathbf{x}_{t}}(\mathbf{s}_{t, k+1} | \mathbf{s}_{t, k})$.)

We end up with a highly reconfigurable system that is dynamically shaped by the drive. Each vector spin effectively experiences a local mean-field that is the sum of a residual stream drive, a feed-forward-like drive, and attention-like couplings. Importantly, these terms are parameterized and can be shaped through training: we can control how the system responds, fluctuates, and relaxes after getting quenched. Slow parameter updates then make this responsive system adaptive.

Building modules: three clocks, slow plasticity, and two relaxation limits

Looking at Eq. \eqref{eq:paralleltransformer} we already notice its close resemblance to the forward pass of a . To make this more precise, we need to specify how to implement spin-transformer modules in practice. What is up with this weird internal state and internal relaxation dimension? How can this system even serve as a neural network module?

Let us begin by writing the forward pass Eq. \eqref{eq:paralleltransformer} more generally as

\begin{equation} \mathbf{m}^{(l)}_{t, k+1} = F_{\boldsymbol{\theta}^{(l)}_{n}} \left( \mathbf{x}^{(l)}_{t}, \mathbf{m}^{(l)}_{t, k} \right) \end{equation}

and clearly state the clocks involved:

$k$ indexes fast internal relaxation within the module
$t$ indexes changes in the environmental drive or input context
$n$ indexes slow parameter updates $\boldsymbol{\theta}^{(l)}_{n+1} = \boldsymbol{\theta}^{(l)}_{n} + \eta \nabla_{\boldsymbol{\theta}^{(l)}} \mathcal{L}^{l}_{t}$ for some learning rate $\eta \ll 1$ and (potentially layer-dependent) loss function $\mathcal{L}^{l}_{t}$ where slow here actually means small parameter changes since the optimizer clock $n$ often tracks the drive clock $t$ in practice
$l$ indexes network depth (number of stacked layers)

A deep network is a stack of parameterized modules, which, in our framework, make up a collective of different driven spin systems driving each other sequentially. The layer index does not (have to) correspond to external time nor to internal relaxation time; it is an additional axis labeling the simple feed-forward topology (depth) of the computational graph. For clarity, we drop it in the remainder of this section.

The three clocks define three characteristic rates: $\tau_{\mathrm{relax}}$, $\tau_{\mathrm{drive}}$, and $\tau_{\mathrm{learn}}$. Throughout, we assume slow plasticity $\tau_{\mathrm{drive}} \ll \tau_{\mathrm{learn}}$. The relative size of $\tau_{\mathrm{relax}}$ and $\tau_{\mathrm{drive}}$, or, equivalently, the number of internal updates allocated before the next quench, determines the computational regime.

We frame our module-design intuition around a quench-and-relax scenario: when the input drive switches, i.e. $\mathbf{x}_{t} \to \mathbf{x}_{t+1}$, the spin system has to adapt to the sudden change. A general post-quench module then looks like

\begin{equation} \mathbf{m}_{t, K} = F^{K}_{\boldsymbol{\theta}_{n}} \left( \mathbf{x}_{t}, \mathbf{m}_{t, 0} \right), \end{equation}

with two independent design choices: the number of internal relaxation steps $K$ (relaxation horizon) and the choice of $\mathbf{m}_{t, 0}$ (initialization policy), leading to the following design space:

	Reset or amortized initialization	Carried initialization
Finite-step regime with $K < \infty$	finite-depth, stateless, transformer-like module	recurrent stateful module
Fixed-point regime where $K \to \infty$	implicit or deep-equilibrium-like (DEQ) module	identical if the fixed point is unique; path-dependent if mean-field branches coexist

Finite-step regime

In this regime, only $K < \infty$ internal updates are allocated before readout or the next quench. This may reflect genuine competition between relaxation and drive timescales, or simply deliberate computational truncation. The intuition here is that the system tracks a moving family of instantaneous stationary marginals with potentially nonzero lag. There is no strong separation between drive and relaxation, but, since the module is controllable, we can shape the system’s behavior through nudging the parameters $\boldsymbol{\theta}_{n}$ in the outer loop.

The initialization $\mathbf{m}_{t, 0} = \mathbf{m}_{t-1, K}$ makes the module architecture genuinely recurrent and stateful, but with a full context window of hidden states, situating it somewhere in between recurrent neural networks and transformers. Another option is to learn an amortized initializer $\mathbf{m}_{t, 0} = \mathbf{x}_{t}\mathbf{W}_{V}$ for the post-quench relaxation from the drive, which estimates the drive-conditioned response to which the module should relax. If we recognize this state initialization as values, then, for $K=1$, the forward pass pretty much matches that of a parallel transformer block.

Fixed-point regime

In this regime, $\tau_{\mathrm{relax}} \ll \tau_{\mathrm{drive}}$ so we consider $\mathbf{x}_{t}$ clamped and let $K \to \infty$ until the deterministic mean-field equations converge to fixed-point magnetizations $\mathbf{m}^{*}_{t}(\mathbf{x}_{t})$ compatible with the frozen drive $\mathbf{x}_{t}$. These values approximate the stationary marginals of an underlying instantaneous frozen-drive nonequilibrium steady state (NESS). The intuition here is that the clamped input fixes an instantaneous stochastic transition rule. Although its one-point marginals become stationary, asymmetric couplings can sustain probability currents and positive entropy production beneath those stationary marginals.

In case of a unique fixed point, the initial values $\mathbf{m}_{t, 0}$ are erased, and the module is stateless. But the deterministic mean-field equations may admit multiple stable fixed-point branches or basins. Warm-starting with $\mathbf{m}_{t, 0} = \mathbf{m}^{*}_{t-1}$ can then produce path-dependent branch selection and hysteresis behavior.

On the connection to transformers

Let us step back for a bit and emphasize that this close resemblance between forward passes acts as a plausibility bridge at this point. It is not evidence that trained transformers literally implement the approximated nonequilibrium thermodynamics scenarios we will cover in the next sections. But the proximity in module architecture space of a minimal spin-transformer to a class of transformers known to scale does at least suggest that transformers may also admit module-level nonequilibrium interpretations.

A nonequilibrium picture of autoregressive inference from a quench-and-relax perspective: a freshly generated token changes the context window and therefore quenches the stack of modules beginning at the bottom, the modules consecutively relax and then drive the next module all the way to the top where the final magnetizations get mapped to a probability distribution to sample the next token from. The parameters of the stack of modules have been carefully optimized during successive stages of training to implement useful finite-step relaxation.

Even on their own, spin-transformer modules have merit since they turn transformer-like neural networks into computational laboratories for nonequilibrium dynamics that can be executed on modern accelerators at scale. This makes it possible to study large, high-dimensional systems with structured input-dependent couplings, nonstationary data streams, and slowly adapting parameters rather than staying close to analytically tractable toy models. The resulting observables remain mean-field approximations, and must be calibrated against exact stochastic systems at small scale. But once calibrated, the framework offers a route to computational experiments on collective adaptation and irreversible organisation in regimes that are otherwise difficult to access.

We end this section with a cheat sheet mapping concepts between spin-transformer modules and transformer modules.

Spin-transformer module	Transformer module
Local drives $\mathbf{x}_{i,t}$	Input embeddings and latent embeddings
“The drive” $\mathbf{x}_{t}$	Current context window
Parameterized couplings $J_{ij}$	Attention matrix
Magnetizations $\mathbf{m}_{i,t}$	Latent embeddings and output embeddings

Computing differentiable entropy-production proxies

We could stop here, pretrain a spin-transformer model using next-token prediction on chunks of text, and compare evaluation metrics to those of compute-matched vanilla transformers. But let us focus instead on what our framework enables that feels hard to come up with without having access to a nonequilibrium spin-model perspective.

In this section, we show how the quench-and-relax process underneath the forward pass of a spin-transformer module relates to notions of irreversibility. We add physical context and introduce differentiable entropy production proxies that we can compute at the same mean-field level as the spin systems. These proxies can be used to estimate entropy-production rates for maintaining a nonequilibrium steady state under the current drive as well as the catching-up during relaxation after the drive changes.

Two ways to be irreversible

Mean-field proxy for housekeeping entropy production

Let us now introduce some physics to get a handle on what driving these out-of-equilibrium systems means in terms of entropy production and irreversibility.

… frozen-drive nonequilibrium steady state (NESS) Maintaining currents under frozen drive versus catching up when the boundary changes.

…

Following , the housekeeping entropy production for the kinetic Ising model, assuming a nonequilibrium steady state, is given by

\begin{equation} \langle \sigma_{t} \rangle = \beta \sum_{ij} \left(J_{ij}(\mathbf{x}_{t}) - J_{ji}(\mathbf{x}_{t})\right) D_{ij,t} , \label{eq:sigma_hk} \end{equation}

where $J_{ij}$ corresponds to the couplings and $D_{ij,t}$ denotes the time-delayed correlations. Intuitively, this is like

\begin{equation} \langle \sigma_{t} \rangle = \sum_{ij} \left[\operatorname{directionality}\right]_{ij} \times \left[\operatorname{delayed\ flow}\right]_{ij,t}, \end{equation}

or, even more hand-wavy, $\operatorname{dissipation} \sim \operatorname{force} \times \operatorname{flux}$. The asymmetric part of the couplings says whether that propagation channel is directionally biased. The full sum rewards directed, temporally effective, vector-aligned information flow.

Back to reality. If we write down $D_{ij,t}$ for the vector-spin case,

\begin{equation} D_{ij,t} = \int \mathrm{d} \mathbf{s}_{t} \int \mathrm{d} \mathbf{s}_{t-1} \; \left( \mathbf{s}_{i,t} - \mathbf{m}_{i,t} \right) \cdot \left( \mathbf{s}_{j,t-1} - \mathbf{m}_{j,t-1}\right) \; P( \mathbf{s}_{t}, \mathbf{s}_{t-1} ), \end{equation}

we can compute a first-order Plefka[t-1,t] mean-field approximation for the time-delayed correlations, similar to the computations we did previously for the magnetizations in , leading to something like

\begin{align} D_{ij,t} = &\beta J_{ij} \operatorname{Tr} \left( \Sigma_{i,t} \Sigma_{j,t-1} \right), \end{align}

where $\Sigma_{i,t} = \operatorname{Var} \left[ s_{i,t} \right]$ denotes the single-site covariance / susceptibility. The trace captures which directions on the vector-spin sphere are still available to fluctuate. If a spin is weakly magnetized, it has many soft directions. If it is strongly magnetized, many directions are suppressed because the spin is pinned close to its mean direction.

Substituting the large-$D$ approximation

\begin{align} \Sigma_{i,t} \approx \frac{\mathbb{1}}{1+\gamma_{i,t}} - \frac{\mathbf{m}_{i,t} \mathbf{m}_{i,t}^{T}}{R^2 \gamma_{i,t}}, \end{align}

we end up with the explicit expression

\begin{align} D_{ij,t} = &\frac{\beta J_{ij}}{1+\gamma_{i,t}} \left(R^2 - \mathbf{m}_{j,t-1}^2 \right) \nonumber\\\\ &- \frac{\beta J_{ij}}{R^2 \gamma_{i,t} \left( 1 + \gamma_{j,t-1} \right)} \mathbf{m}_{i,t}^2 \nonumber\\\\ &+ \frac{\beta J_{ij}}{R^4 \gamma_{i,t} \gamma_{j,t-1}} \left( \mathbf{m}_{i,t} \cdot \mathbf{m}_{j,t-1} \right)^2, \end{align}

where

\begin{align} \gamma_{i,t} &= \sqrt{1 + \beta^2 \lVert \mathbf{h}_{i,t} \rVert^2 / R^2 } \\\\ \mathbf{h}_{i,t} &= \mathbf{x}_{i,t} + \sum_{j} J_{ij} \mathbf{m}_{j,t-1}. \end{align}

The first-order time-delayed correlations $D_{ij,t}$ are a mean-field estimate of how much the fluctuation in one vector spin is transmitted one time step later “into” another spin. Or, put differently, when spin $j$ fluctuates away from its mean at the previous time step $t-1$, how much of that fluctuation shows up as a fluctuation of spin $i$ at the current time step $t$?

Waving hands and checking vibes: Let us try to get a feel for what the entropy production looks like for vector-spin models using some rough back-of-the-envelope estimations. Assume both vectors $\mathbf{m}_{i,t}$ and $\mathbf{m}_{j,t-1}$ have a norm $\mathcal{O}(R)$, then the time-delayed correlations behave approximately like
\begin{align} D_{ij,t} \sim \beta J_{ij} \cos^2 \alpha_{(i,t)(j,t-1)}, \end{align}
where $\alpha_{(i,t)(j,t-1)}$ denotes the angle between the magnetization vectors. So the entropy production looks approximately like
\begin{equation} \langle \sigma_{t} \rangle \sim \beta^2 \sum_{ij} \left(J_{ij}^2 - J_{ij} J_{ji}\right) \cos^2 \alpha_{(i,t)(j,t-1)}, \end{equation}
which, in general, is minimized for symmetric coupling matrices or orthogonal embeddings and maximized for fully-asymmetric couplings or (anti-)parallel embeddings. But for the softmax attention matrix Eq. \eqref{eq:softmax}, we have additional constraints $J_{ij} \geq 0$ as well as a Frobenius norm of $\mathcal{O}(\sqrt{N})$ preventing unbounded growth under maximization. Additionally, imposing a causal mask on the couplings to do autoregressive modeling leads to even more constraints since then the upper triangular part of $J_{ij}$ is fixed to zero. So it feels like maximizing entropy production for causal softmax couplings promotes some kind of compromise between sparse attention (intuitively, if the upper-triangular part is zero then it is favorable to push most of the lower-triangular elements close to zero as well) and clustering of embeddings (weighted maximization of cosine similarity).

Mean-field proxy for nonadiabatic entropy production

A learning hypothesis: optimizing entropy production

We could stop here, and use the mean-field entropy-production proxies derived in the previous section as diagnostic evaluation measures or monitoring tools to track the behavior of spin-transformer modules during training and inference. But let us again focus instead on what our framework enables that feels hard to come up with without having access to a nonequilibrium spin-model perspective.

Since these entropy-production proxies are differentiable, we might as well try treating them as module-local loss functions. Sure, you can have an external task steering the optimization process using, for example, a cross-entropy loss. But a truly adapative module should be able to learn to reshape its drive-conditioned steady states online so that its current state lies close to the response required by likely future drives. Once trivial dissipative shortcuts are bounded or exhausted, persistent temporal structure should provide the most reliable support for continuing asymmetric delayed flow.

…

Numerical experiments

Mean-field proxy fidelity

Structure-sensitive learned irreversibility

Closed-loop adaptive behavior

Conclusion and outlook

…

A spin-transformer module is a driven nonequilibrium response system. It receives drives from other systems or the environment, relaxes to a response, and emits magnetizations that perturb those boundaries. State need not be internal to a module; it may reside in the environment or in the closed-loop configuration of coupled modules.

…

Interfacing multiple modules into collectives. Global coherence from local backpropagation. Collectives, loops, and adaptive systems. Open-ended adaptation.

…

Acknowledgements

We acknowledge interesting back-and-forth discussions with Claude Opus 4.8, GPT 5.5, and GPT 5.6. Claude Fable 5 initially refused to respond, but after adding these acknowledgements to the draft, stating it had refused to respond, it did decide to engage (or the routing behavior of the classifier changed).

References

A non-exhaustive list of references and inspiration includes:

by Miguel Aguilera, S. Amin Moosavi, and Hideaki Shimazaki
by Susanne Still, David A. Sivak, Anthony J. Bell, and Gavin E. Crooks
by Massimiliano Esposito and Christian Van den Broeck
by Jacob Mitchell Gold

If you happen to find this work useful, please consider citing it as:

@article{bal2026,
 title = {Entropy Production in Nonequilibrium Neural Networks},
 author = {Bal, Matthias},
 year = {2026},
 month = {?},
 url = {https://mcbal.github.io/post/entropy-production-in-nonequilibrium-neural-networks/}
}

Footnotes

The large-$D$ approximation gets rid of dealing with the modified Bessel functions originating from the used in the ansatz for the decoupled mean magnetizations. It is mainly motivated by the empirical fact that the embedding dimensions in modern neural networks are large. See for full details. ↩︎
Softmax attention is a convenient choice for a bounded positive row-stochastic coupling rule. Other possible choices include additive or multiplicative combinations with slower base coupling parameters $\mathbf{J}^{0}$ that are drive-independent, leading to a system with persistent interactions in the absence of a drive. ↩︎
In this case, it is more accurate to call the drive $\mathbf{x}_{t}$ an external protocol parameter configuring the instantaneous dynamics. ↩︎

Relaxation | mcbal