Sebastian Raschka

Founder, Principal AI & LLM Research Engineer — RAIR Lab LLM Research Engineer — Lightning AI Assistant Professor (Statistics) — University of Wisconsin-Madison

Listen — profile

0:00 / 3:14

Profile

Sebastian Raschka occupies a niche almost nobody else fills well: he is the person who shows you the code. While much of the AI commentariat traffics in benchmark charts and vibes, Raschka’s default move is to reimplement the thing in plain PyTorch, on a laptop, with every tensor shape annotated. That discipline is why Build a Large Language Model (From Scratch) became the standard on-ramp for developers who wanted to stop treating transformers as a black box — and why its companion repo, rasbt/LLMs-from-scratch, has passed 100,000 GitHub stars, putting it in the same rarefied tier as Andrej Karpathy’s nanoGPT as a canonical teaching artifact.

The path here was unusually circuitous. Raschka did his PhD in computational biology at Michigan State, wrote Python Machine Learning in 2015 as a grad student — a book that quietly became a bestseller and spawned the still-excellent Machine Learning with PyTorch and Scikit-Learn — and spent years as an assistant professor of statistics at the University of Wisconsin–Madison. He also built mlxtend, a scikit-learn-adjacent utility library that a lot of people used without knowing whose it was. In 2022 he joined Lightning AI as an LLM research engineer and eventually resigned the professorship outright. He now runs RAIR Lab (Raschka AI Research), his own independent research-and-consulting shop, which is the rare case of an educator monetizing without diluting: the free material stayed free and got better.

The other half of his output is Ahead of AI, a newsletter read by well over 150,000 people that has become the closest thing the field has to a trustworthy quarterly review. Pieces like The Big LLM Architecture Comparison — a side-by-side dissection of what actually differs between DeepSeek, Llama, Qwen, Gemma and GPT-OSS at the block level — do the work that most model announcements deliberately avoid. He is consistently more interested in which components changed and why than in leaderboard placement, and he’s willing to say when a widely hyped technique is a marginal delta. His 2026 writing has followed the field into reasoning models, RLVR/GRPO, and local coding agents, again with runnable implementations attached.

For a developer learning AI today, the argument for Raschka is simple. Most explanations sit at one of two altitudes: a blog post with an analogy, or a paper with no code. He lives in between, and he is honest about the gap between a clean from-scratch implementation and what a production training run actually requires. Read him alongside Aurélien Géron for classical ML foundations, and treat his repos as the thing you type out yourself rather than clone. His 2026 follow-up, Build a Reasoning Model (From Scratch), does the same trick for inference-time scaling, GRPO, and distillation — the machinery behind the reasoning models that now dominate the frontier.

Books

Build a Large Language Model (From Scratch)

2024

Code a GPT-2-class LLM end to end in PyTorch — tokenization, attention, pretraining, and finetuning — on hardware you already own.

Build a Reasoning Model (From Scratch)

The 2026 sequel: takes a pretrained LLM and adds evaluation, inference-time scaling, self-refinement, GRPO-based RL, and distillation.

Machine Learning Q and AI

Thirty short chapters answering the intermediate-level questions that tutorials skip, from embeddings to multi-GPU training.

Machine Learning with PyTorch and Scikit-Learn

The comprehensive classical-ML-through-deep-learning textbook, and still one of the best single volumes for building real foundations.

Python Machine Learning

The 2015 original that made his reputation and taught a generation of developers scikit-learn.

Key Articles & Papers

The Big LLM Architecture Comparison 2025 — A component-by-component teardown of DeepSeek, Llama, Qwen, Gemma and GPT-OSS — the single best answer to 'what actually changed since GPT-2?' Understanding Reasoning LLMs 2025 — Clear taxonomy of how reasoning models are built: inference-time scaling, RL, and distillation, written right after the DeepSeek-R1 shock. Understanding Large Language Models 2023 — The reading-list post that oriented thousands of engineers entering LLM work — still a solid curriculum. Understanding and Coding the Self-Attention Mechanism From Scratch 2023 — Self-, multi-head, causal and cross-attention implemented line by line; the reference explainer for anyone who has read the paper but not the code. Practical Tips for Finetuning LLMs Using LoRA 2023 — Hundreds of controlled experiments distilled into actionable rules on rank, alpha, QLoRA trade-offs, and which layers to adapt. Understanding Multimodal LLMs 2024 — Cross-attention versus unified-embedding approaches to vision-language models, explained without hand-waving. The State Of LLMs 2025: Progress, Problems, and Predictions 2025 — His annual field survey — the one to read if you want a year of developments compressed without marketing. Components of A Coding Agent 2026 — Deconstructs what a coding agent actually is beneath the product layer: tool loops, context handling, and harness design. Recent Developments in LLM Architectures 2026 — KV sharing, compressed attention and related efficiency work — where architecture research is heading after the MoE wave. Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning 2018 — His most-cited academic work: a rigorous, readable treatment of cross-validation and benchmarking that remains relevant to anyone reporting numbers.

Videos

Notes on what I verified and what I’d flag:

Video IDs: quh7z1q7-uc (3-hour workshop), kPGTx4wcm_w (Developing an LLM), nDl6Aj9aPAI (PyTorch Conference 2025), and EV7WhVT270Q (Lex Fridman #490, with Nathan Lambert) are confirmed across multiple sources. The last four — Vanishing Gradients, MAD Podcast, TWIML, and the Manning author interview — came from single search-index results with matching titles. Drop any of them if your script’s API check fails.
No Controversies section: I found no notable public controversy. He’s a low-drama figure; the closest thing is ordinary technical disagreement in his newsletter comments.
RAIR Lab has no standalone domain — the canonical page is sebastianraschka.com/rair-lab/, which is what I linked.