DR-018 — Review of the 3-tier vetting¶

Converted from docs/decisions/DR-018-review-of-3-tier-vetting.md to MADR (backlog B25). The text is unchanged; its sections are grouped under the MADR headings below, in their original order within each group.

Context¶

1. Local model pool (Tier 2)¶

Vetting says	Review	Our plan
Ollama on every inventory machine, exposed on the network	Yes, but bind to the LAN address and firewall the port to the harness machine. Ollama has no authentication, so it must never be open to the internet or guest Wi-Fi	One model per machine, chosen by its RAM
LiteLLM proxy as one OpenAI-compatible endpoint with load balancing	Yes, adopt. It gives one endpoint for the harness and Aider, routing by model name, retries and fallbacks, per-key budgets and spend logs, and it can carry ZDR headers for OpenRouter. The harness already talks the OpenAI format, so this is a config change	LiteLLM on the primary laptop now; on the server later (§2), serving both local and cloud models
llama3.3:70b and qwen2.5-coder:32b on the inventory machines	Not realistic for low-configuration, CPU-only machines. A 4-bit 70B model needs about 40+ GB of RAM and a 32B about 20 GB, and both run at well under 1 token a second on CPU	8 GB machine: qwen2.5-coder 1.5B or 3B. 16 GB: qwen2.5-coder 7B. 32 GB: a 14B coder. Anything larger needs a GPU (DR-010)
"Seamlessly pool resources of different machines" (Atul's question)	Distributed inference that splits one model across machines (llama.cpp RPC, exo, distributed-llama) exists, but over home LAN and CPUs it's slower than one machine. Not recommended. Pool requests, not models: LiteLLM sends each request to a free machine running a suitable model	LiteLLM request pooling

2. Program-management server (MMDEP app, MCP gateway, server harness)¶

Vetting says	Review
4 cores / 8 threads, 16 GB RAM, 500 GB NVMe, Ubuntu 24.04	Oversized to start. The server runs no models; it calls cloud APIs. Its load is: the MMDEP app and MCP gateway (light); git worktrees for 5–6 repositories; test suites for one or two items at a time; MkDocs builds; LiteLLM.

Recommended sizing (SRV-1):

Stage	Size	Example cost
Start	2 vCPU, 4 GB RAM, 80 GB SSD, Ubuntu 24.04	Lightsail about $24 a month
If test suites queue up	2 vCPU, 8 GB RAM	about $44 a month
Only if measured	4 vCPU, 16 GB RAM	—

Separate from the production Lightsail server. Never co-host with api.moneymarche.com.
Back up the MMDEP database daily (snapshot).
Measure CPU, RAM and queue length from day one (collector), and resize on evidence (DR-004 "measure before it grows").

3. Cloud harness on the server (Tier 3)¶

Vetting says	Review	Our plan
LangGraph or a FastAPI webhook harness	Not LangGraph. We retired a LangGraph orchestrator (`legacy/`) for simplicity. Same code as the desktop harness (DR-012: tiers 1 and 3 share code); only config differs	The existing `mmdep harness`, run on the server with cloud models through LiteLLM. Polling as now, plus an optional webhook endpoint (the server has a public address) to start a run immediately
Token gate, worktree spawner, headless Aider, ZDR headers, auto MR	Yes, and most of it already exists: routing gates, worktrees, exact-string, placeholder and scope gates, auto PR/MR plus merge by risk. Aider and ZDR are B19 and B22	Reuse
Keys from environment variables	Yes. On the server, store keys in a root-only env file or secret store, never in the repository. LiteLLM holds the provider keys; the harness gets one LiteLLM key with a budget	—

4. The three guardrails¶

Harness 1 (attended browser agents) must not write to branches the harness scans. Already true: the harness only creates and reads auto/* branches, agents use their own prefixes (sonnet/*, lane-s/*…), and B20 gives every agent its own worktree. Results go in the issue and routing block (rules §21). The generated dependency matrix replaces tasks-matrix.json (DR-017).
Harness 2: on merge conflict or failure, drop the branch, record why, flag for help. Mostly already true: a failed item is recorded in the ledger, commented on the issue (S3), and skipped until the issue changes.
Change: escalate before a human. After 2 failures on local or free models, route to a stronger model (Gemini Flash, then Claude Sonnet). Only after that add the label needs-attention, which the actions sync shows to the owning agent first and to Atul only if it's business-relevant.
Harness 3: fail over to CI minutes automatically if the local pool doesn't answer within 90 seconds. Disagree with the trigger. A slow model call isn't a reason to spend rationed compute minutes. Model failover belongs inside the harness's model chain (local, then Gemini, then OpenRouter ZDR, then Claude), which already exists. CI minutes run tests, not models, so ration them by rule:
the laptop heartbeat is INACTIVE for more than 4 hours and the item is labelled critical; or
a critical item has waited more than 24 hours;
always within the per-group monthly budget (DR-011, FBK-1).

5. Other points¶

Claude credits running out: keep Claude (subscription) for design and cx:3 work in Harness 1. In the server harness, use Claude's API only for cx:3 critical items, with a LiteLLM budget. Gemini Flash-Lite (paid) and OpenRouter (ZDR) carry the volume.
ChatGPT Go: stays a Harness 1 tool for design, preflight and GitLab/GitHub edits. Its prompts go into issues as routing blocks, which the harnesses execute.

Options¶

Not stated separately in the original record.

Decision¶

6. Decisions for Atul¶

POOL-1: LiteLLM as the single model endpoint (local pool plus cloud keys, with budgets). One right-sized model per inventory machine; no cross-machine model splitting.
SRV-1: start the server at 2 vCPU / 4 GB / 80 GB (Lightsail, about $24 a month), separate from production; resize on measured load.
Needed: the specs of the 2–3 inventory machines (RAM, CPU, any GPU, OS), so each gets the right model.

Consequences¶

Not stated separately in the original record.