In Part 2, we built a simple MLP that generates names one character at a time, trained on 32k popular names. In this lecture, Andrej Karpathy walks through the transformer architecture one piece at a time. We will start by refactoring that model, then add each transformer component in turn and watch the loss drop as we go. ...
I have been learning dbt recently and wanted to get a better understanding of data warehousing methodologies. Two of the most well-known ones are the Kimball and Inmon approaches. In this post, I will explain the key differences between the two and why Kimball is generally considered better for analytics. Kimball (Dimensional Modeling) Kimball’s approach uses a star schema with two main types of tables: Fact Tables: Store measurable events (sales amount, quantity sold, order totals). These are the “what happened” of your business. Dimension Tables: Store descriptive context (customer info, product details, dates). These answer “who, what, where, when.” The star schema looks like this: ...
This cheat sheet covers the fundamentals of dbt (data build tool)—a popular data transformation tool used in modern data engineering and analytics workflows. It includes key concepts, commands, and best practices to get you started with dbt. Fundamentals (Must Know) What is dbt? dbt (data build tool) is a transformation tool that enables analysts and engineers to transform data in their warehouse using SQL. Key concept: dbt handles the T in ELT (Extract, Load, Transform). It doesn’t extract or load data—it transforms data that’s already in your warehouse. ...
Why I’m Learning dbt as an Analyst You can find the project on GitHub here: zhugejun/learn-dbt-by-building I’ve been an Institutional Research Analyst in higher education for almost a decade. For my day-to-day job, I can wrangle enrollment with SQL, automate reports with R, build prediction models with Python, and visualize data with Power BI. There are times that I need to run queries to pull data from CAMS directly, download data from ZogoTech (our third-party OLAP vendor), save it as CSV, and load it to R for aggregation, visualization, and further analysis. Sometimes, I need to ingest the enrollment history data from National Student Clearinghouse (NSC) and combine it with data from multiple resources to create a superintendent report. ...
In Part 1, we built a single-layer neural network – no hidden layer at all – that generates words one character at a time. It worked well enough: the network reproduced exactly what the simple counting model had produced. But a bigram model is limited by construction, since it assumes each character depends only on the one immediately before it. If a character starts just one bigram, the model will always emit that same next character, no matter what came earlier or how likely the alternatives are. That missing context is what holds bigram models back. In this lecture, Andrej Karpathy shows how a deeper network fixes it. ...