Frontier-lab interview prep · LLM internals

Weekly slides

Nine tutorial decks, 354 slides, one per week of the LLM-internals track. Each one has step-by-step diagrams, live widgets, real interview questions, a reading plan and a self-test. Use → or Space to step through a deck.

  1. Week 1The whole pipeline in one passThe whole pipeline from text to reasoning assistant, with step-by-step diagrams, live widgets and real interview questions.39 slides
  2. Week 2The modern dense TransformerHow GPT-2 became Llama 3 (RMSNorm, SwiGLU, RoPE and context extension, grouped-query and sliding-window attention), with live widgets, real configs and real interview questions.37 slides
  3. Week 3Latent attention and mixture of expertsDeepSeek's latent attention, mixture-of-experts routing, load balancing and expert parallelism, and the 2025 open-model architecture table, with live calculators and real interview questions.40 slides
  4. Week 4Scaling arithmetic and pretrainingScaling arithmetic and pretraining, from Chinchilla and inference-aware over-training to GPU-day estimates, FineWeb data curation and training instabilities, with live calculators and real interview questions.40 slides
  5. Week 5RLHF foundationsRLHF foundations, from the policy gradient and reward models to PPO, the KL leash and Constitutional AI, with live widgets and real interview questions.40 slides
  6. Week 6DPO and RL for reasoningThe DPO derivation, GRPO versus PPO with the length bias worked out, and DeepSeek-R1 stage by stage, with live widgets and real interview questions.39 slides
  7. Week 7Hands-on GRPO run and quiz practiceRun a small GRPO job on math, write the loss from scratch, read healthy and broken training curves, defend the result, and practise the oral quiz rounds.40 slides
  8. Week 8Frontier recipes at scaleWhat Meta, DeepSeek, Moonshot, Ai2 and the OPT team actually did to train, stabilise and post-train frontier models, with live widgets and real interview questions.39 slides
  9. Week 9Critique and measurementFixing reasoning RL (DAPO, entropy), telling real capability gains from measurement artifacts, test-time compute, reward hacking, leaderboards, on-policy distillation, async RL, and a reusable method for critiquing a lab's model.40 slides

Part of the frontier-lab interview curriculum. Resources and question coverage for this track: LLM-internals study guide.