Blog posts

Transformers Are Secretly Collectives of Spin Systems

A statistical mechanics perspective on transformers
Last updated on Nov 29, 2021 14 min read
Transformers Are Secretly Collectives of Spin Systems

Transformers from Spin Models: Approximate Free Energy Minimization

How far can we push the idea of transformers as physical systems?
Last updated on Oct 12, 2021 25 min read
Transformers from Spin Models: Approximate Free Energy Minimization

Deep Implicit Attention: A Mean-Field Theory Perspective on Attention Mechanisms

Can we model attention as the collective response of a statistical-mechanical system?
Last updated on May 11, 2021 20 min read
Deep Implicit Attention: A Mean-Field Theory Perspective on Attention Mechanisms

Attention as Energy Minimization: Visualizing Energy Landscapes

Can we swap softmax attention for energy-based attention?
Last updated on Mar 18, 2021 19 min read
Attention as Energy Minimization: Visualizing Energy Landscapes
See all posts
Matthias Bal

Matthias Bal

About me

Interests

  • Machine learning, data science, and computational modeling
  • Statistical physics, many-body systems, and tensor networks
  • Literature, language, and the stories we tell ourselves

Education

  • PhD in Physics, 2018

    Ghent University

  • MSc in Physics and Astronomy, 2013

    Ghent University

  • BSc in Physics and Astronomy, 2011

    Ghent University