Blog posts

Spin-Model Transformers

A non-equilibrium statistical mechanics perspective on transformers

Last updated on Dec 7, 2023 51 min read

Spin-Model Transformers

Transformers Are Secretly Collectives of Spin Systems

A statistical mechanics perspective on transformers

Last updated on Nov 29, 2021 15 min read

Transformers Are Secretly Collectives of Spin Systems

Transformers from Spin Models: Approximate Free Energy Minimization

How far can we push the idea of transformers as physical systems?

Last updated on Oct 12, 2021 25 min read

Transformers from Spin Models: Approximate Free Energy Minimization

Deep Implicit Attention: A Mean-Field Theory Perspective on Attention Mechanisms

Can we model attention as the collective response of a statistical-mechanical system?

Last updated on May 11, 2021 20 min read

Deep Implicit Attention: A Mean-Field Theory Perspective on Attention Mechanisms

Matthias Bal

About me

Interests

Machine learning, data science, and computational modeling
Statistical physics, many-body systems, and tensor networks
Literature, language, and the stories we tell ourselves

Education

PhD in Physics, 2018

Ghent University