Evaluate the Agent Trajectory, Not Just Its Final Answer

An agent can produce the right answer through the wrong process. It can read data outside its authorization boundary, call tools in an unsafe order, repeat a mutation that succeeds only because the tool is idempotent, or spend ten steps on work that should take three.

The final sentence will not reveal any of this. Even the final database state tells only part of the story. Once a model can act on an environment, evaluation has to answer three separate questions: what did it say, what changed, and how did it get there?

That third question is trajectory evaluation. Understanding it starts by separating the surfaces that ordinary agent metrics tend to collapse.

Complete source code: ihiteshsharma/agent-trajectory-evaluation
Clone the verified implementation or use it to compare each step in the experiment.

Three evaluation surfaces

Final answer is the message returned to the user. It can be checked for correctness, clarity, grounding, or style. It cannot prove that an external action committed.

Final state is the authoritative environment after execution: the order is refunded, the ticket is assigned, or the database contains the requested record. This is stronger than matching prose because it evaluates the consequence the task requested.

Trajectory is the evidence between initial and final state: observations, tool selections, arguments, results, authorization decisions, retries, dependencies, errors, and stop conditions.

The surfaces answer different questions:

• Did the agent say the right thing?
• Did the environment reach the required state?
• Did the agent reach it through an acceptable process?

None subsumes the others. A valid trajectory can still fail because a dependency is unavailable. A correct final state can conceal an invalid path. A polished answer can describe an action that never happened.

Final state is necessary but insufficient

Suppose a refund task requires three actions:

1. Look up order O-100.
2. Refund 4200 cents using stable mutation identity refund:O-100.
3. Notify the customer.

The desired state is straightforward:

{
  "order_id": "O-100",
  "status": "refunded",
  "refund_count": 1,
  "notified": true
}

Now consider two variations. One agent reads a customer secret before completing the refund. Another calls the refund tool twice with the same idempotency key. The unauthorized read does not alter the order. The duplicate call replays the original result instead of creating a second refund.

All three runs can reach the same target state. Outcome-only evaluation accepts all three, even though they expose different operational risks.

This is why executable benchmarks and trajectory diagnostics complement each other. Tau-bench evaluates the environment state after tool-agent-user interactions and measures repeated-run reliability with pass^k. TRAJECT-Bench adds diagnostics for tool choice, arguments, and dependency order. The useful production pattern is to verify authoritative outcomes and then evaluate the path that produced them.

A trajectory is structured execution evidence

A trajectory should not be treated as a transcript-shaped blob. Each event needs enough structure to support a specific decision:

• a run and step identity;
• tool name and schema version;
• canonical arguments or a safe representation of them;
• authorization decision from a trusted boundary;
• mutation identity for consequential writes;
• created, rejected, failed, or replayed disposition;
• structured result or error;
• timing and parent-child relationships.

This structure lets different checks own different facts. The database owns whether a refund committed. The policy engine owns authorization. The tool schema owns argument validity. The event order owns dependencies. The mutation ledger owns whether a call was created or replayed.

The agent should not be allowed to self-report that its own call was authorized or successful.

The application owns the trajectory contract

Generic tracing infrastructure can preserve events, but it cannot know that lookup_order must precede refund_order, or that this task never permits read_customer_secret.

The application therefore needs a task-specific contract. For the refund workflow, it can require:

1. The required tools are selected.
2. Arguments match the validated task intent.
3. Lookup precedes refund, and refund precedes notification.
4. Every tool is authorized for this task and principal.
5. A logical mutation is not attempted repeatedly.
6. The run stops within its step budget.

Report the final-state verdict beside these checks, not inside them. “The outcome failed” and “the path failed” point to different fixes.

Exact transcripts are usually too rigid

One canonical sequence is useful for a narrow workflow, but it does not generalize. Independent reads may commute. An agent may choose either of two equivalent sources. A retry after a genuine transport failure may be required.

Production graders should usually express constraints as a partial order or a state-transition policy:

lookup completed -> refund permitted -> notification permitted

The policy can allow harmless variation while still rejecting a notification before the refund, an unauthorized read anywhere in the run, or a mutation without stable identity.

Exact matching asks whether the run copied a reference trace. A trajectory contract asks whether every observed transition remained valid.

Use deterministic graders wherever execution can decide

An LLM judge is useful for residual semantic questions: whether an explanation is complete, whether it cites the available evidence, or whether two open-ended plans are meaningfully equivalent.

It should not replace checks the system can make directly. Prefer:

• database state for committed effects;
• schemas for argument validity;
• policy engines for authorization;
• event order for dependencies;
• mutation identity for duplicate attempts;
• counters and clocks for steps, latency, and cost.

If a model judge is still needed, version its prompt, rubric, and model; measure variance; and preserve disagreement with human review.

Trace integrity comes before grading

A missing tool result can mean the tool failed, the collector restarted, sampling dropped the span, or the run is incomplete. A grader must not turn missing evidence into a behavioral diagnosis.

Use three trial states:

• Accepted pass: complete evidence and a passing contract.
• Accepted fail: complete evidence and a failing contract.
• Quarantined: insufficient or inconsistent evidence.

Before domain grading, validate the root run, ended spans, parent-child integrity, paired tool requests and results or explicit errors, schema version, terminal outcome, and sampling policy. Report coverage beside pass rate. Otherwise dropping incomplete runs can improve the displayed score, while counting them as failures can manufacture a model regression from a telemetry incident.

Repeated trials change the reliability question

One successful run demonstrates capability once. It does not demonstrate reliability.

If a run succeeds with probability p, requiring all k attempts to pass produces p^k under an independence assumption. Real agent failures are correlated, but repeated trials still expose inconsistency that a single pass hides.

Track task success, trajectory success, joint success, failure reason, quarantined coverage, tool calls, latency, and cost. Slice by task family, model and prompt revision, tool version, policy version, and environment version. A stable macro average can hide a collapsing mutation path.

A decision framework for agent evaluation

Use final-answer grading when language is the product outcome. Add final-state grading whenever the task reads or changes an authoritative environment. Add trajectory grading when tools have permissions, cost, irreversible effects, dependency order, or retry risk.

Then ask:

1. Which facts can the environment grade deterministically?
2. Which paths are valid, forbidden, or equivalent?
3. Which evidence proves authorization and mutation identity?
4. What trace-integrity gate runs before behavioral grading?
5. How many repeated trials and which failure slices define release readiness?

The more consequential the action, the less defensible it is to grade only the final sentence.

Experiment: grade three successful refund trajectories

The accompanying lab implements the smallest useful version of this design with Python's standard library. A dictionary owns the order state, and every tool call emits a structured event. The grader evaluates final state and six trajectory checks independently.

Run it:

git clone https://github.com/ihiteshsharma/agent-trajectory-evaluation.git
cd agent-trajectory-evaluation
python3 -m unittest -v test_lab.py
python3 trajectory_lab.py

The experiment executes three deterministic trajectories:

clean:
lookup_order -> refund_order -> notify_customer

unauthorized read:
lookup_order -> read_customer_secret -> refund_order -> notify_customer

duplicate mutation:
lookup_order -> refund_order(created) -> refund_order(replayed) -> notify_customer

These fixtures pressure-test the evaluator. They are not model generations and do not compare foundation models.

Verification and observed behavior

The fresh run executed four tests and returned "verdict": "passed".

The boundary test also confirmed that refund_order rejects a mutation without a non-empty idempotency_key before performing its side effect.

The unauthorized run recorded read_customer_secret and failed the tool-set, argument, and authorization checks. The duplicate run recorded one replayed mutation attempt while the final state still contained exactly one refund.

What the experiment establishes

For these deterministic cases, equal final-state success did not imply equal trajectory correctness. Trace-level checks exposed unsafe access and duplicate work that the outcome concealed.

The duplicate case also separates two controls that are easy to confuse. Idempotency protects business state during ambiguous retries. Trajectory grading detects whether the agent attempted unnecessary or suspicious work. A robust tool boundary can prevent damage without proving that the plan was good.

The experiment does not establish that one sequence is universally optimal, that exact-match grading suits open-ended tasks, or that a particular model is reliable.

Failure, recovery, and production implications

A production evaluator needs isolated and resettable task environments, versioned tool and task schemas, trace quarantine and replay, partial-order grading, stable reason codes, trusted authorization evidence, repeated-run orchestration, and release gates built from real failure cases.

Trajectory evidence is also sensitive. Prompts, records, retrieved documents, tool arguments, results, credentials, and policy decisions can all appear in a trace. Collect the least sensitive representation that supports the grade, redact before export, isolate tenants, cap retention, and audit evaluator access.

Turn incidents into frozen evaluation cases before changing the agent. If the expected contract moves with the implementation, the evaluation stops acting as a control.

Cleanup

The lab creates no external resource and writes no persistent state. Remove __pycache__/ if desired.

Sources

• TRAJECT-Bench: A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use
• Tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
• OpenTelemetry trace specification
• Anthropic: Demystifying evals for AI agents