Evaluation Is the Control System for Production AI - Production AI Systems - Part 5
Evaluation connects product requirements, versioned datasets, component failures, operational constraints, release gates, and production feedback.
Read noteTopic
AI Systems
Long-Running AI Work Is a Durable Workflow - Production AI Systems - Part 4
Long-running AI work should be modeled as durable state transitions with idempotent stages, checkpoints, bounded retries, cancellation, and backpressure.
Read noteFrom Model Output to Trusted Application Data - Production AI Systems - Part 3
Schema-constrained generation is only the first trust boundary. Production systems still need evidence, policy, authorization, bounded repair, and explicit disposition.
Read noteThe Model Gateway Is a Policy Boundary, Not a Proxy -Production AI Systems - Part 2
A model gateway should centralize provider compatibility, routing, quotas, and data policy without absorbing product workflow logic.
Read noteA Production AI System Is More Than a Model Call - Production AI Systems - Part 1
A production AI feature is not a model call. It is a distributed system with explicit boundaries for authorization, orchestration, validation, durable execution, and evaluation.
Read noteWhen an Agent Calls a Tool, What Happens to Its KV Cache?
Understand how paused agent requests trade accelerator memory against KV-cache transfer, recomputation, resume latency, and system goodput.
Read noteEvaluate the Agent Trajectory, Not Just Its Final Answer
Build an execution-based grader that separates final-state success from tool-selection, argument, authorization, ordering, and duplicate-work failures.
Read noteMake Agent Tool Retries Safe With Typed Contracts and Idempotency
Understand idempotency as a semantic contract for ambiguous tool outcomes, then prove durable replay and conflict handling locally.
Read noteThe State of Agentic AI in 2026: What Changed, What Matters, and How to Start
A practical field guide to the capabilities, protocols, risks, and minimum path for building a first useful AI agent in 2026.
Read note