A statistical mechanics perspective on transformers
How far can we push the idea of transformers as physical systems?
Can we model attention as the collective response of a statistical-mechanical system?
Can an energy-based perspective shed light on training and improving Transformer models?