Entropy Production in Non-Equilibrium Neural Networks

Mon, 02 Feb 2026 09:28:17 +0100

Introduction

✨ Work in progress. Reach out if you want to come join the fun!

✨ GitHub repository:

Modern large-scale autoregressive language models are impressive system engineering artifacts serving billions of users. Powerful in-context learning capabilities can be elicited at inference time through external scaffolding, harnesses, and environment engineering. Yet these models are frozen, with memory externalized into ever-growing transcripts and learning into large-scale offline optimization. This has implications for online continual learning, adaptive model deployment, and real-time closed-loop interaction with live systems.

In this post, we focus on adaptive systems that can reuse a fixed substrate, remain online, and continuously reshape internal dynamics under local constraints. To move in this direction, we treat neural networks as non-equilibrium thermodynamic systems. Building on previous work in , we design transformer-like modules based on mean-field dynamics of a class of vector-spin models where module inputs map to applied fields driving the system and module outputs are mean-field magnetizations.

The physics-inspired backbone of these architectures enables us to write down a proxy for , a thermodynamic quantity measuring irreversibility by quantifying the asymmetry between forward and backward time steps. Since every operation in the computational graph is differentiable, entropy production can be made into a loss steering irreversible flow through the system. For example, maximizing entropy production incentivizes the system to lean into the external drive by nudging its parameters towards asymmetric delayed responses that absorb and transmit structure in the incoming drive. Internally, we imagine the system reshaping itself into ordered structures to enable more efficient dissipation of the tension caused by the incoming data stream.

The risk is that the system finds local dissipative shortcuts: asymmetric attention collapse, self-exciting cycles, or coupling to noise. In the interesting regime of bounded driven system, useless dissipation saturates while structure-sensitive flows remain persistent. For this to happen, environments, as well as the boundary interfaces coupling separate driven systems, need to be engineered so that the most stable way to increase entropy production when flooded by a structured data stream is to latch onto the latent (temporal) structure. Ideally, individual modules locally amplify asymmetric delayed flows in parallel, while module connectivity and environment feedback collectively constrain which flows remain stable and useful for the system as a whole.

The bet is that, embedded in sufficiently structured streams and with a capability to act on its environment, the cheapest way for a bounded local system to keep dissipating is to become predictive, where prediction is a thermodynamic adaptation to ensure continuing support for asymmetric delayed flows. The global computation then emerges from coupled local dissipative systems. We readily admit that the main motivation for this bet is aesthetic. To move beyond aesthetics, we run numerical experiments to find out whether local ascent on a computable entropy-production proxy, under bounded dynamics and structured drive, can lead to local, scaling-compatible learning rules.

Background and intuition

We consider transformer modules as differentiable driven disordered vector-spin systems whose mean-field collective behavior we can control through training, and refer to going back to for earliest instantiations of this intuition. According to our correspondence, the forward pass of a transformer module implements a spin system’s response to getting probed, where inputs map to time-varying applied external fields, asymmetric, sparse attention matrices can be identified with fully-connected spin-spin interactions, and outputs map to spin expectation values or magnetizations. Vigorously waving hands, the forward pass of a spin-transformer module can be designed to mimic that of a vanilla transformer module.

In contrast to physics-oriented literature, we do not specify explicit probability distributions for the external fields and couplings of the disordered many-body system, nor are we interested in Nobel-prize-winning ways to average out the disorder. We instead focus on the very specific stream of quenched disorder realizations induced by a dataset or environment of interest, encoded as sequences of vector embeddings, which we use to drive the system relentlessly. In this framing, training a transformer module corresponds to sculpting the underlying system’s collective response by tuning the parametrized distributions of its external fields and couplings. We are sculpting a spin glass with data.

In , we observed that these systems tend to settle into non-equilibrium steady states (NESSs) as dynamic sweet spots where the “continuous kicking” of the inputs (applied external fields) “sustains” the outputs (magnetizations). This negotiation process tends to happen after just a few iterations. The first iteration already gives a decent guess, which might explain why (1) transformers can get away with stacking modules whose forward passes take just one time step, and (2) why doing a few time steps can improve performance, as done in looping and recursive reasoning approaches. Indeed, repeating the same module can be seen as allowing the underlying non-equilibrium system to settle more snuggly into its steady state for that particular configuration of inputs and parameters. However, as soon as the input drive changes, or the parameters change, the system has to somehow renegotiate a different steady state compatible with what its new configuration dictates the response should be. More on that later (it is more intricate than this; it always is).

Non-equilibrium neural networks

Motivation

Minimal model

When designing neural networks around mean-field vector-spin models, there is a lot of architectural freedom. First of all, we must decide on what mean-field approximation to use to approximate the time-dependent behavior of our vector-spin system. Projecting the dynamics to different ansatz distributions leads to different mean-field equations, which take into account more or less correlations at different time steps.

Mindful of the importance of locality and scaling in distributed systems, we pick the simplest option: a first-order Plefka[t-1,t] approximation. From , we all remember

\begin{equation} \mathbf{m}_{i,t} = \frac{\beta \left( \mathbf{x}_{i,t} + \sum_{j} J_{ij} \mathbf{m}_{j,t-1} \right)}{1+\sqrt{1+\beta^2 \lVert \mathbf{x}_{i,t} + \sum_{j} J_{ij} \mathbf{m}_{j,t-1} \rVert^2 / R^2 }}, \end{equation}

where $\mathbf{m}_{i,t} \in \mathbb{R}^{D}$ denote the magnetizations (outputs) at time $t$, $\mathbf{x}_{i,t} \in \mathbb{R}^{D}$ denote the applied external fields (inputs) at time $t$, $J_{ij}$ are the couplings, $\beta$ is an inverse temperature, and $R=\sqrt{D/2 -1}$ is a natural hyperspherical length scale resulting from the large-$D$ approximation we used to get rid of dealing with Bessel functions. The large-$D$ approximation should be fine since the embedding dimensions in modern neural networks are large.

If we now consider some kind of parametrized drive-dependent couplings

\begin{equation} \mathbf{J} (\mathbf{x}) = \mathrm{softmax}\left( \mathbf{x} \boldsymbol{Q} \boldsymbol{K}^{T} \mathbf{x}^{T} \right), \label{eq:softmax} \end{equation}

we turn the fixed-size coupling matrix into a parametrized rule that supports variable system size, drive-dependent routing, and a way to scale system size without learning new explicit parameters. Softmax attention is a convenient choice for a bounded positive row-stochastic coupling rule. Other choices include additive or multiplicative combinations with slower base coupling parameters $\mathbf{J}^{0}$ that are drive-independent, leading to a system with persistent interactions in the absence of drive.

If we also augment the applied external fields with some kind of parametrized drive-dependent local drive or memory,

\begin{equation} \mathbf{x}_{i,t} \to \mathbf{x}_{i,t} + \mathrm{FFN}\left( \mathbf{x}_{i,t} \right), \end{equation}

then our forward pass looks like the recurrence relation

\begin{equation} \mathbf{m}_{i,t} = \frac{\beta \left( \mathbf{x}_{i,t} + \mathrm{FFN}\left( \mathbf{x}_{i,t} \right) + \sum_{j} J_{ij} (\mathbf{x}_{t}) \mathbf{m}_{j,t-1} \right)}{1+\sqrt{1+\beta^2 \lVert \mathbf{x}_{i,t} + \mathrm{FFN}\left( \mathbf{x}_{i,t} \right) + \sum_{j} J_{ij} (\mathbf{x}_{t}) \mathbf{m}_{j,t-1} \rVert^2 / R^2 }}, \end{equation}

which resembles a parallel transformer block, with the notable difference that the “values” here correspond to the outputs (magnetizations) of the previous time step instead of some linear transformation applied to the inputs at the current time step: current drive routes previous state. Making the applied external fields as well as the couplings drive-dependent leads to a highly-adaptive system where the interaction landscape itself is dynamically shaped by the inputs. Each vector spin effectively experiences a local mean-field that is the sum of a residual stream, a feed-forward-like drive, and attention-like couplings.

But where are the values we know and love from QKV attention? Let us jump ahead and try to figure out what is going on by thinking how we can embed spin-transformer modules in a neural network. We will be forced to grapple with non-equilibrium thermodynamics (audience groans and starts rolling their eyes). There are two main scenarios: (1) a fixed-point module where we clamp the drive $\mathbf{x}_{t}$ and relax the response along an internal time dimension towards a NESS, and (2) a finite-step recurrent module where changing input drives induces quenches or driven transitions between NESSs. These two limits are about timescale separation and adiabaticity: if internal relaxation time becomes comparable to the physical drive timescale, then a fixed point cannot be reached. When a system is driven rapidly, it fails to relax to its instantaneous NESS and the system is forced to keep chasing moving targets in a fast, non-adiabatic regime.

Let us first walk through the fixed-point relaxation scenario. Freezing the drive at $\mathbf{x}_{t}$, we interpret the update equations as an inner-loop relaxation

\begin{equation} \mathbf{m}_{i,k} = \frac{\beta \left( \mathbf{x}_{i,t} + \mathrm{FFN}\left( \mathbf{x}_{i,t} \right) + \sum_{j} J_{ij} (\mathbf{x}_{t}) \mathbf{m}_{j,k-1} \right)}{1+\sqrt{1+\beta^2 \lVert \mathbf{x}_{i,t} + \mathrm{FFN}\left( \mathbf{x}_{i,t} \right) + \sum_{j} J_{ij} (\mathbf{x}_{t}) \mathbf{m}_{j,k-1} \rVert^2 / R^2 }}, \end{equation}

hopefully settling into an instantaneous near-equilibrium steady-state (NESS) fixed point $\mathbf{m}^{*}_{t}(\mathbf{x}_{t})$ for $k \to \infty$ compatible with the current drive $\mathbf{x}_{t}$. If the fixed point is unique, the influence of the initialization is erased. If the landscape is funky, supporting multiple attractor basins or showing signs of hysteresis, then the response $\mathbf{m}^{*}_{t}(\mathbf{x}_{t})$ of the relaxation dynamics depends on the initialization: the system has an implicit memory. In this setup, we can interpret values in QKV attention as amortized feed-forward responses, approximating the internal relaxation process by learning a linear mapping from input drives directly to values. So a transformer block with QKV attention is like a one-shot learned estimator of a fixed-point relaxation response. This approach is also reminiscent of deep equilibrium models (DEQs) and looped, recursive reasoning approaches, but, arguably, less ad hoc since here the looping is done to solve self-consistent mean-field message-passing equations. External time $t$ advances only when the drive changes $\mathbf{x}_{t} \to \mathbf{x}_{t+1}$ (a new sequence arrives) and the nonstationarity messes up everything again, provoking a transition $\mathbf{m}^{*}_{t}(\mathbf{x}_{t}) \to \mathbf{m}^{*}_{t+1}(\mathbf{x}_{t+1})$ between two NESSs.

In the second scenario with a finite-step recurrent module, the driven nonstationary system keeps track of state $\mathbf{m}_{t-1}$ while chasing a moving target with only finite number of internal relaxation steps at every external time step $t$. This is a pretty challenging problem.

About time and clocks: In this section, we have introduced an external physical time index $t$ as well as an internal relaxation index $k$. Yet at this point we are still only discussing a single module, i.e. a single spin system, getting probed by an environment. A deep network is a stack of untied modules, which, in our framework, are part of a collective of different driven spin systems driving each other sequentially. The layer index does not (have to) correspond to external time nor internal relaxation time (except maybe in how DEQ models are sometimes presented). It is an additional axis labeling the simple feed-forward topology (depth) of the computational graph.

Mean-field proxy for entropy production

Following , the entropy production for the kinetic Ising model, assuming a non-equilibrium steady state, is given by

\begin{equation} \sigma_{t} = \sum_{ij} \left(J_{ij} - J_{ji}\right) D_{ij,t} \geq 0, \end{equation}

where $J_{ij}$ corresponds to the couplings and $D_{ij,t}$ denotes the time-delayed correlations. Intuitively, this is like

\begin{equation} \sigma_{t} = \sum_{ij} \left[\operatorname{directionality}\right]_{ij} \times \left[\operatorname{delayed\ flow}\right]_{ij,t}, \end{equation}

or, even more hand-wavy, $\operatorname{dissipation} \sim \operatorname{force} \times \operatorname{flux}$. The asymmetric part of the couplings says whether that propagation channel is directionally biased. The full sum rewards directed, temporally effective, vector-aligned information flow.

Back to reality. If we write down $D_{ij,t}$ for the vector-spin case,

\begin{equation} D_{ij,t} = \int \mathrm{d} \mathbf{s}_{t} \int \mathrm{d} \mathbf{s}_{t-1} \; \left( \mathbf{s}_{i,t} - \mathbf{m}_{i,t} \right) \cdot \left( \mathbf{s}_{j,t-1} - \mathbf{m}_{j,t-1}\right) \; P( \mathbf{s}_{t}, \mathbf{s}_{t-1} ), \end{equation}

we can compute a first-order Plefka[t-1,t] mean-field approximation for the time-delayed correlations, similar to the computations we did previously for the magnetizations in , leading to something like

\begin{align} D_{ij,t} = &\beta J_{ij} \operatorname{Tr} \left( \Sigma_{i,t} \Sigma_{j,t-1} \right), \end{align}

where $\Sigma_{i,t} = \operatorname{Var} \left[ s_{i,t} \right]$ denotes the single-site covariance. The trace captures which directions on the vector-spin sphere are still free to fluctuate. If a spin is weakly magnetized, it has many soft directions. If it is strongly magnetized, many directions are suppressed because the spin is pinned close to its mean direction.

Substituting the large-$D$ approximation

\begin{align} \Sigma_{i,t} \approx \frac{1}{1+\gamma_{i,t}} - \frac{\mathbf{m}_{i,t} \mathbf{m}_{i,t}^{T}}{R^2 \gamma_{i,t}}, \end{align}

we end up with the explicit expression

\begin{align} D_{ij,t} = &\frac{\beta J_{ij}}{1+\gamma_{i,t}} \left(R^2 - \mathbf{m}_{j,t-1}^2 \right) \nonumber\\\\ &- \frac{\beta J_{ij}}{R^2 \gamma_{i,t} \left( 1 + \gamma_{j,t-1} \right)} \mathbf{m}_{i,t}^2 \nonumber\\\\ &+ \frac{\beta J_{ij}}{R^4 \gamma_{i,t} \gamma_{j,t-1}} \left( \mathbf{m}_{i,t} \cdot \mathbf{m}_{j,t-1} \right)^2, \end{align}

where

\begin{align} \gamma_{i,t} &= \sqrt{1 + \beta^2 \lVert \boldsymbol{\theta}_{i,t} \rVert^2 / R^2 } \\\\ \boldsymbol{\theta}_{i,t} &= \mathbf{x}_{i,t} + \sum_{j} J_{ij} \mathbf{m}_{j,t-1}. \end{align}

The first-order time-delayed correlations $D_{ij,t}$ is a mean-field estimate of how much the fluctuation in one vector spin is transmitted one time step later “into” another spin. Or, put differently, when spin $j$ fluctuates away from its mean at the previous time step $t-1$, how much of that fluctuation shows up as a fluctuation of spin $i$ at the current time step $t$?

In the fixed-point implementation, the iteration index used by the solver should not be identified with physical time. Physical time is the index of the external drive. Each forward pass computes the fixed-point magnetizations induced by the current drive. The delayed correlations entering the entropy-production proxy are computed across external time steps, not across internal relaxation time steps.

Vibe check

Let us try to get a feel for what the entropy production looks like for vector-spin models using some rough back-of-the-envelope estimations. Assume both vectors $\mathbf{m}_{i,t}$ and $\mathbf{m}_{j,t-1}$ have a norm $\mathcal{O}(R)$, then the time-delayed correlations behave approximately like

\begin{align} D_{ij,t} \sim J_{ij} \cos^2 \alpha_{(i,t)(j,t-1)}, \end{align}

where $\alpha_{(i,t)(j,t-1)}$ denotes the angle between the magnetization vectors. So the entropy production looks approximately like

\begin{equation} \sigma_{t} \sim \sum_{ij} \left(J_{ij}^2 - J_{ij} J_{ji}\right) \cos^2 \alpha_{(i,t)(j,t-1)}, \end{equation}

which, in general, is minimized for symmetric coupling matrices or orthogonal embeddings and maximized for fully-asymmetric couplings or (anti-)parallel embeddings.

But for the softmax attention matrix Eq. \eqref{eq:softmax}, we have additional constraints $J_{ij} \geq 0$ as well as a Frobenius norm of $\mathcal{O}(\sqrt{N})$ preventing unbounded growth under maximization. Additionally, imposing a causal mask on the couplings to do autoregressive modeling leads to even more constraints since then the upper triangular part of $J_{ij}$ is fixed to zero. So it feels like maximizing entropy production for causal softmax couplings promotes some kind of compromise between sparse attention (intuitively, if the upper-triangular part is zero then it is favorable to push the lower-triangular elements close to zero as well) and clustering of embeddings (weighted maximization of cosine similarity).

✨ The mean-field entropy production proxy captures how much asymmetric attention transports aligned state fluctuations forward in time.

…

Local-learning rules and sparse credit assignment

…

Imagine we want to turn our entropy production proxy into a loss function. One option would be a stop-gradient / local version

\begin{equation} \sigma_{t} = \sum_{ij} \left(J_{ij} - J_{ji}\right) \operatorname{sg}\left(D_{ij,t}\right), \end{equation}

then $\Delta J_{ij} \propto D_{ij} - D_{ji}$ is a temporally asymmetric Hebbian learning rule.

…

Collectives, loops, and adaptive systems

…

Numerical experiments

…

Model behavior in a noisy environment

Cybernetics, interfaces, environments, sensors, controllers, and effectors.

…

Global coherence from local backpropagation

We test a stack of spin-transformer modules in a toy femtoscale online learning setup and try to see if we can make synchronization or specialization happen between the spin-transformer modules when maximizing per-layer entropy-production losses independently. If we detach module outputs after applying each layer, we end up with systems communicating via their boundary interfaces, but without gradients backpropagating through the whole stack. (Pretty unlikely that the entropy-production losses on their own provide enough signal though.)

…

Growing network topologies

…

Conclusion

…

References

If you happen to find this work useful, please consider citing it as:

@article{bal2026,
 title = {Entropy Production in Non-Equilibrium Neural Networks},
 author = {Bal, Matthias},
 year = {2026},
 month = {?},
 url = {https://mcbal.github.io/post/entropy-production-in-non-equilibrium-neural-networks/}
}

A non-exhaustive list of references and inspiration includes:

by Miguel Aguilera, S. Amin Moosavi, and Hideaki Shimazaki
by Jacob Mitchell Gold
by Giovanni Pezzulo and Michael Levin

Steady State | mcbal

Entropy Production in Non-Equilibrium Neural Networks

Introduction

Background and intuition

Non-equilibrium neural networks

Motivation

Minimal model

Mean-field proxy for entropy production

Vibe check

Local-learning rules and sparse credit assignment

Collectives, loops, and adaptive systems

Numerical experiments

Model behavior in a noisy environment

Global coherence from local backpropagation

Growing network topologies

Conclusion

References

Acknowledgements

Footnotes

Steady State | mcbal

Entropy Production in Non-Equilibrium Neural Networks

Introduction

Background and intuition

Non-equilibrium neural networks

Motivation

Minimal model

Mean-field proxy for entropy production

Vibe check

Local-learning rules and sparse credit assignment

Collectives, loops, and adaptive systems

Numerical experiments

Model behavior in a noisy environment

Global coherence from local backpropagation

Growing network topologies

Discussion and related work

Conclusion

References

Acknowledgements

Footnotes