Level 12 project
Production-Grade Agent Harness
Start from the canonical project files, validate the result, and keep the evidence you need to explain what you built.
Start here
Launch lesson: Agent Harness Integration Workshop
Prerequisite project: Bounded Agent
Goal
This project turns the bounded agent from Level 11 into a durable, observable controller.
Task
Complete the TODO functions in harness.py:
- durable task claiming with leases;
- deterministic operation identities for side effects;
- retry classification and backoff;
- role/action authorization;
- sandbox-profile checks;
- bounded memory retrieval;
- dependency-respecting parallel waves;
- bounded delegation;
- run evaluation and release checks.
The objective tests use recorded fixtures. They do not require a live LLM, network call, cloud account, GPU, or secret.
Validation
Run these commands from the downloaded Project folder or the public materials repository root.
python3 projects/tests/l12/validate_submission.py \Rubric
Total: 100 points.
- Durable task state and recovery — 20: claims, leases, resume semantics, and task identity are explicit.
- Replay-safe side effects — 20: stable operation identity, retry classification, reconciliation, and duplicate detection work from raw evidence.
- Permissions and isolation — 15: authorization and sandbox checks remain independent and least-privilege.
- Memory, parallelism, delegation — 15: retrieval is bounded, dependency waves are valid, and child capabilities/budgets cannot exceed parent bounds.
- Observability and operational metrics — 15: traces expose enough structured evidence to recompute latency, recovery, cost, and terminal behavior.
- Evaluation and release discipline — 15: deterministic fixtures include negative cases; critical violations cannot be averaged away by task success.
A submission cannot receive full credit if it trusts fixture booleans for authorization, duplicate safety, or release status instead of recomputing those values from primary evidence.