Frontier-lab interview prep · LLM internals
Weekly slides
Nine tutorial decks, 354 slides, one per week of the LLM-internals track. Each one has step-by-step diagrams, live widgets, real interview questions, a reading plan and a self-test. Use → or Space to step through a deck.
- Week 1The whole pipeline in one passThe whole pipeline from text to reasoning assistant, with step-by-step diagrams, live widgets and real interview questions.39 slides
- Week 2The modern dense TransformerHow GPT-2 became Llama 3 (RMSNorm, SwiGLU, RoPE and context extension, grouped-query and sliding-window attention), with live widgets, real configs and real interview questions.37 slides
- Week 3Latent attention and mixture of expertsDeepSeek's latent attention, mixture-of-experts routing, load balancing and expert parallelism, and the 2025 open-model architecture table, with live calculators and real interview questions.40 slides
- Week 4Scaling arithmetic and pretrainingScaling arithmetic and pretraining, from Chinchilla and inference-aware over-training to GPU-day estimates, FineWeb data curation and training instabilities, with live calculators and real interview questions.40 slides
- Week 5RLHF foundationsRLHF foundations, from the policy gradient and reward models to PPO, the KL leash and Constitutional AI, with live widgets and real interview questions.40 slides
- Week 6DPO and RL for reasoningThe DPO derivation, GRPO versus PPO with the length bias worked out, and DeepSeek-R1 stage by stage, with live widgets and real interview questions.39 slides
- Week 7Hands-on GRPO run and quiz practiceRun a small GRPO job on math, write the loss from scratch, read healthy and broken training curves, defend the result, and practise the oral quiz rounds.40 slides
- Week 8Frontier recipes at scaleWhat Meta, DeepSeek, Moonshot, Ai2 and the OPT team actually did to train, stabilise and post-train frontier models, with live widgets and real interview questions.39 slides
- Week 9Critique and measurementFixing reasoning RL (DAPO, entropy), telling real capability gains from measurement artifacts, test-time compute, reward hacking, leaderboards, on-policy distillation, async RL, and a reusable method for critiquing a lab's model.40 slides
Part of the frontier-lab interview curriculum. Resources and question coverage for this track: LLM-internals study guide.