A course about how large language models are served. It is built on the open
textbook “LLM Inference Systems: Principles, Mathematics, and Practice” and redesigned
here as interactive lecture-blogs you can study on your own, at your pace.
convenience
Easy to find, learn or recap the material: every lecture carries its derivations, worked
numbers, and pitfalls with it.
clarity
Each page is a result of care not only about what to say, but how to say and,
especially, how to show something.
you
These materials exist so that you (yes, you!) could study what you like, at your pace.
The purpose is to help you enter your own very personal adventure. For you.
A Guide to Your Adventure
Lecture-blogs
Which we tried to make: intuitive, clear and engaging;
complete (the full lecture and more); and honest about trade-offs, because in serving there
is no single right answer, only workloads. Each lecture carries Research Thinking cards,
worked numeric examples, and a Have Fun! corner.
Research Thinking
Learn to think as a systems researcher: find the flaw in an approach, think why/when something
can help, come up with ways to improve, then look at previous attempts.
It is well-known that you learn something easier if you are not given the
answer right away, but think about it first. In every card you will see ? a question.
Think about it for a minute, a day, a week... and only then open the possible answers. You are
not supposed to reinvent a paper that took its authors months; it's the habit of thinking that
counts.
Interactive parts
Often we ask you to walk through a figure step by step, to play with a
small playground, or just to think with a pen. The roofline is only truly yours after you've
derived it once with your own numbers. Lecture 1's playground is a good place to start.
What's inside: the course map
One spine, nine lectures. The spine is the decode arithmetic
intensity \(I \approx \frac{2PB}{2P + B\cdot kv}\) from Lecture 1. Every later lecture is an
intervention on one of its terms.