0114 min read
What Actually Happens When You Send a Prompt to an LLM?
Follow one LLM request from the API gateway to the GPU and back. Learn what prefill, decode, and the KV cache do along the way.
Backend · distributed systems · infrastructure
Learn how modern systems work.
Practical explanations of backend engineering, distributed systems, Kubernetes, and AI infrastructure. Read a deep dive, follow a guide, or practice a system design.
Recommended reading
All articles →Start with these articles
Build it step by step
All guides →Practical guides
GUIDE01
New // beginner
Kubernetes from Zero: Build and Operate Your First Local Cluster
Learn why Kubernetes exists. Then build a local cluster and practice deploying, scaling, fixing, updating, and rolling back an application.
Find your path
All topics →Explore by topic
Recently published
Latest articles
Article48 min read
Writing Kubernetes Controllers
Learn how Kubernetes controllers work, from watches and queues to reconciliation, scaling, testing, and production use.
Article12 min read
Prefill and Decode Need Different Resources
Prefill processes the prompt. Decode generates one token at a time. Learn when running them on separate workers improves speed and capacity—and when it does not.
Engineering intelligence
Open Radar →On the radar
Models
NVIDIA pairs a 3B-active MoE model with agent workload routing
NVIDIA released Nemotron 3.5 Lightning, a 30B mixture-of-experts model that uses 3B parameters at a time. NeMo Switchyard can send each agent task to a model based on quality and cost instead of using one large model for everything.
Why it matters
An agent does not need the same model for every step. A strong model can plan while smaller models handle repeated tool calls and checks. The router now becomes an important part of the infrastructure because it affects quality, speed, cost, privacy, and failures.
Architecture practice
View all →System designs
System designintermediate
Core System Design Concepts: A Practical Guide
Learn the main ideas behind system design, including scale, data, failures, security, cost, and common architecture choices.
About the author
Written from engineering experience.
BackendServices, APIs, and data systems
InfrastructureCloud native platforms and networking
AI systemsInference, scheduling, and reliability
I’m Tejo. I explain complex systems in plain English, from the first request to recovery after a failure. I include enough detail for you to use the ideas in real systems.
More about me