Guided learning path
Local LLM Inference
Run and measure open-weight models locally, connecting quantization, hardware limits, latency, throughput, and output quality.
Who this path is for
Developers evaluating local model serving on constrained hardware.
What you will learn
Define a serving workload; measure latency and throughput separately; evaluate a quantization choice against both memory and quality constraints.
Prerequisites
A command-line environment, basic Python familiarity, and an understanding of prompts and tokens. Check each lab for its own hardware and software requirements.
Difficulty and workload
Intermediate and hands-on. Begin with the benchmark methodology, then follow the CPU-serving and quantization experiments where your hardware permits.
Suggested reading sequence
- Benchmark LLM Serving Without Lying to Yourself
- Serve a 3B LLM on CPU With llama.cpp
- When Q4 Passes Latency but Fails Quality
- The KV Cache Is Your Real LLM Serving Budget
This reading sequence uses existing articles. New guided lessons will appear in the published-lessons list below. Return to all learning paths to explore a related subject.
Curriculum
Published lessons
- No lessons have been published in this path yet.