Mean-Field Theory

Nonequilibrium Dynamics in Spin-Model Transformers featured image
Artificial Intelligence Associative Memories Attention +18

Nonequilibrium Dynamics in Spin-Model Transformers

A design space for controllable quench-and-relax architectures

avatar
Matthias Bal
•
Read more
Spin-Model Transformers featured image
Artificial Intelligence Associative Memories Attention +12

Spin-Model Transformers

A nonequilibrium statistical mechanics perspective on transformers

avatar
Matthias Bal
•
Read more
Deep Implicit Attention: A Mean-Field Theory Perspective on Attention Mechanisms featured image
Artificial Intelligence Associative Memories Attention +13

Deep Implicit Attention: A Mean-Field Theory Perspective on Attention Mechanisms

Can we model attention as the collective response of a statistical-mechanical system?

avatar
Matthias Bal
•
Read more