📌 Understanding Transformer Architecture by Building GPT
In Part 2, we built a simple MLP that generates names one character at a time, trained on 32k popular names. In this lecture, Andrej Karpathy walks through the transformer architecture one piece at a time. We will start by refactoring that model, then add each transformer component in turn and watch the loss drop as we go. ...