H1 harness spike: notes

Design: docs/research/h1-harness-design.md, approved in #67 (option B, the caps as proposed, Sonnet). These notes have two parts: - the pre-approval run of 6 Oct 2026: the dev copy on a scratch hub; - the spike: the runtime on the real hub, after the code is approved and released. Not run yet.

1. What was built

• ClaudeCliAgent with harness: true. The command line is exactly the design's §3:
• -p --output-format json --json-schema <done|question schema>;
• --session-id <uuid>, or --resume <uuid> on a resume;
• --model sonnet --restricted --tools/--allowedTools Read,Edit,Write,Glob,Grep;
• --permission-mode acceptEdits --permission-prompts none --strict-mcp-config --max-budget-usd 0.5.

The prompt goes on stdin. There is no MCP config, so the agent has no write path into the hub. - tasks.session_id, an additive column. MMDEP generates the id and stores it before the launch. - A new task step, resume. When a run ends with question, MMDEP: - raises a specify item, raised by the agent (claude), with the question and summary fenced as untrusted text; - parks the task with step resume.

On the answer, the next pass resumes the same session, with the same agent, in the same worktree (not reset). A failed resume is never handed to another agent. - Limits: - 3 questions per session; a 4th parks a review item, and a retry starts a new session; - spike_budget_usd (10): when the sum of the calls' own costs reaches it, the task parks with a manual item before any further call. - Ledger: - harness.start, harness.resume (waited_s = item created → answered); - harness.result (status, num_turns, duration_ms, total_cost_usd, call_cost_usd); - harness.question (item, cycle); - harness.too_many_questions, harness.budget_spent; - account.lock_wait (waited_s, gave_up) for every launch as the agent account. - The approve item gains a HARNESS line: the session, the questions answered on the way, and the session's cost.

2. Found while building

• total_cost_usd is cumulative over a session. On a resume, the CLI reports the whole session so far. Probe:
• first call: 0.0288 USD (input 6, output 410 tokens);
• resume: total_cost_usd 0.0349; modelUsage input 8 (= 6 + 2), output 774 (= 410 + 364);
• the resume's own cost is about 0.0061.

MMDEP logs call_cost_usd (the difference from the session's previous result), and the spike budget sums that. Open: whether --max-budget-usd caps the call or the whole session. If it's the session, a long session stops at 0.50 USD in total. - The structured result is in structured_output. result holds the same JSON as text, and MMDEP falls back to it. The session_id in the result equals the one MMDEP passed, on the start and on the resume. - A call that fails before a JSON result (non-zero exit, is_error) logs no cost. The budget can undercount by those calls.

3. Pre-approval run, 6 Oct 2026 (dev copy, scratch hub)

Setup: - h1_setup.py in the session scratchpad: two synthetic repositories (h1-a: invoice rounding; h1-b: discounts by customer tier for "ACME Test Ltd" and "Example Traders"); - a scratch hub with claude (harness: true, the #67 caps); - test_command {python} -m unittest -q; - agent account mode off. The scratch hub has no agent_account section, so the lock isn't exercised.

The real hub was not opened.

Leg 1: start → question, two projects in parallel (two mmdep run processes started together):

Project	Session start	Result	Turns	duration_ms	Cost (USD)	Item
h1-a	01:09:12Z	question: "2 decimal places (123.456 → 123.46) or the nearest whole rupee (→ 123)?"	2	17378	0.0267	specify #2
h1-b	01:09:12Z	question: "What discount percentage for each tier: gold and silver?"	2	7308	0.0266	specify #1

• Both sessions overlapped in time.
• Both agents changed nothing before asking, and said so.
• Wall time for both passes: 23 s.

Leg 2: answer → resume → approve. - Through the inbox: not run. The answers have to come from a person, and an agent can't answer an item (this holds in scratch hubs too). - The CLI side of a resume ran live twice, through MMDEP's own adapter code: - the probe; - test_live_claude_asks_then_resumes_the_same_session.

Each time the same session continued and remembered its question. The edit was made, and the result's session_id was unchanged.

Measured so far: - relay steps per task: 1 question; - calls: 2 start calls (0.0533 USD) plus about 6 probe and live-test calls (about 0.12 USD, my estimate from the probe's numbers). Total under 0.20 USD, against the 10 USD spike budget; - time to a question: 7–17 s.

Not measured yet: - the waiting time on a person (item created → answered); - the lock wait; - the end-to-end time to an approve item.

4. The spike (after approval and release)

Per the design's §9: Atul's steps, then the runtime's dispatcher on the real hub: - claude gets harness: true and a cap of 20 a day; - h1-a and h1-b are registered under C:\Projects\ai-sandbox; - each has a test env, test_command, and the agent account mode required.

Runs until success, budget, safety or 3 working days. The tables per task go here, with: - the relay steps; - waited_s; - the costs; - account.lock_wait; - the Agent SDK note (golden rule 2).