Before the Pod Exists: How Kueue Decides Whether an AI Job May Run
Learn how Kueue admits Kubernetes AI and batch jobs, manages scarce GPU quota, and hands workloads to kube-scheduler.
Read noteTopic
Distributed Platforms
When an Agent Calls a Tool, What Happens to Its KV Cache?
Understand how paused agent requests trade accelerator memory against KV-cache transfer, recomputation, resume latency, and system goodput.
Read noteThe KV Cache Is Your Real LLM Serving Budget
Derive KV-cache memory from model architecture, then connect retained tokens and active sequences to real serving capacity.
Read noteWhen Q4 Passes Latency but Fails Quality
See why a Q4 model that fits and meets latency can still fail a release gate—and why incomplete evidence must not select a quantization.
Read noteServe a 3B LLM on CPU With llama.cpp
Understand CPU inference from first principles: autoregressive decode, memory bandwidth, KV state, context allocation, scheduling, and admission control.
Read noteBenchmark LLM Serving Without Lying to Yourself
Understand inference performance as a workload-dependent surface, then test saturation and goodput with a controlled local benchmark.
Read note