In this week’s

Arxiv Dive with <http://Oxen.ai|Oxen.ai>

we are going to go deeper into transformers and “Mechanistic Interpretability” with the paper: “_A Mathematical Framework for Transformer Circuits_”. They break down transformers into small little zero layer transformers, single layer attention only, and two layer attention only models. Breaking the models into small components allows them to build up and to try to understand what the heck these models are doing. As always, reading is optional, and we will dive as deep as need be Find the doc here: https://transformer-circuits.pub/2021/framework/index.html and signup to join our discussion 👉👉👉 https://lu.ma/oxenbookclub