DR-015 — Tools and model routing¶
Converted from
docs/decisions/DR-015-tools-worktrees-fabric-aider-model-routing.mdto MADR (backlog B25). The text is unchanged; its sections are grouped under the MADR headings below, in their original order within each group.
Context¶
1. Worktrees: one per agent or harness (Atul, 20 Oct)¶
Already in use:
- the harness works in C:\Projects\_mmdep-harness-work\issue-N worktrees;
- the jobs runner runs every job in a fresh worktree at the exact commit;
- the hub (mmdep/workspaces.py) has a worktree per task.
Extend (backlog B20): mmdep worktree add <agent> <repo> [branch] creates C:\Projects\_worktrees\<agent>\<repo> on its own branch prefix (for example lane-s/…, harness/…, sonnet/…). Every local or terminal agent and the harness get their own worktree, so no two agents ever share a working copy or a checked-out branch. mmdep worktree list shows them; prune removes merged ones.
2. Fabric for deployment and audit¶
Fabric (Python, built on Invoke and Paramiko) runs named tasks over SSH. It fits two jobs:
- TEA audit tasks: the read-only scripts in tea/scripts/ become Fabric tasks (fab -H moneymarche-aws tea.stgapi-status). Output goes to the inbox or the relay.
- Staging deploys: fab deploy.staging --ref <sha> does artefact upload, systemd restart, and smoke tests. The GitLab runner calls it with the SSH key from a CI secret.
Production stays manual: a Fabric task may prepare a production deploy, but running it needs Atul's approval (high risk, DR-012).
It's a new dependency (fabric), justified because it replaces hand-written SSH steps. It goes in a separate optional extra ([deploy]), not the core package. Backlog B18 is a prototype with three read-only TEA tasks plus a staging status task; it's medium risk and lands in the integration branch.
3. Aider¶
Planned in DR-010 §3 as an optional engine inside the harness for multi-file cx:2 items, after the built-in engine was measured. The pass-rate table (B2) now gives that measurement.
Plan (backlog B19, sprint 2 spike):
- an engine: aider option per routing block or per config;
- the harness still owns routing, scope, budget, tests, lint gates, the regression gate, PRs and merging;
- Aider only edits files in the worktree, and runs with --no-auto-commits and the scope files only;
- models: Gemini (paid) first, then OpenRouter, per §5.
Measure: run the same 10 cx:2 items with the built-in engine and with Aider, compare pass rate, attempts, cost and minutes, then decide the default.
4. Python tools in use today¶
| Tool | Where | Used for |
|---|---|---|
| pytest | tests/, all projects' CI |
Unit and regression tests; harness focused runs plus a full-suite "no new failures" gate |
| ruff | pre-commit, harness gate, GitHub Actions CI | Linting (only findings in changed files fail) and ruff format |
| pre-commit | MMDEP repository | Runs check-yaml, ruff + ruff-format, and gitleaks (secret scan) before every commit |
| gitleaks (binary) | pre-commit | Blocks committing secrets |
| GitHub Actions | .github/workflows/ci.yml |
pytest + ruff on every PR and on main |
| GitLab CI on the laptop runner (Docker) | API, MM-Cron, sites | Project tests without using GitLab minutes |
| PyYAML | registers, routing blocks | Parsing jobs.yaml and issue routing blocks |
| mcp | mmdep/mcp_server.py |
The MMDEP hub's MCP server |
| pywin32 + ctypes | agent_account.py, tools/keys |
Windows agent account, hidden key storage |
| urllib, subprocess, sqlite3 (stdlib) | collector, harness, store | HTTP checks, gh/glab/git through mmdep/shell.py, the hub database |
| Our own tools | tools/quality/dupes.py, route_lint.py, mmdep laptop check, tools/keys/set_key.py, tools/tea/ |
Duplicate-code report, routing-block lint, laptop health, key storage, TEA script library |
| Optional, legacy: langgraph, langchain-ollama, google-genai | legacy/ |
The earlier orchestrator (not in the core path) |
Gaps worth adding (backlog B21, low risk, as CI steps only): coverage reporting, plus pip-audit (dependency vulnerabilities) and bandit (Python security lint) in report-only mode first.
5. Model routing: Gemini, OpenRouter free, local Ollama (and later machines)¶
The governing rule is the data class (DR-008/009), then cost, then quality.
| Tier | Models | Allowed data | Used for |
|---|---|---|---|
| Gemini API (paid tier) | Flash-Lite (first), Flash | public, indicative, internal (no client data) | Harness first choice for internal code (MMDEP, API, MM-Cron, PE, MCRM); knowledge extraction (DR-014); failure triage; actions-sync free-text parsing |
| OpenRouter free | openrouter/free router; specific :free models (Qwen Coder, Llama, Nemotron, Gemma…) |
public and synthetic only, unless MOD-1 allows more: free providers may log or train on prompts | Open work packages and factsheet parsers on public PDFs; fallback for public-data repositories; second opinions on public code |
| OpenRouter paid / Claude API / OpenAI | as configured | per provider terms; never client data | Only with allow_paid and within the ₹3,000 cap |
| Local Ollama (laptop, low memory) | qwen2.5-coder:1.5b today |
any, including restricted, because nothing leaves the machine | Redaction double-check before cloud calls; preflight classification (cx/ctx/risk); commit and PR text; small cx:1 edits when offline |
| Local, larger machines (later, DR-010) | qwen2.5-coder:7b on 16 GB; bigger models on a GPU box |
any | cx:1–2 code with tests, offline; nightly batch runs; the same harness, a different url |
| Claude (subscriptions) | Sonnet (coding project), Opus (PM and design) | internal | cx:2–3 coding, reviews, design, PM; no API cost |
How the harness applies it (backlog B17): every source (repository) carries data_class (public or internal). The model chain filters out any model whose allowed classes don't include it. For example, openrouter-free is skipped for internal repositories unless MOD-1 allows it, and local models are always allowed. The ledger records the data class per run.
7. Atul's additions (20 Oct 23:09): evaluation¶
7.1 OpenRouter free models with Zero Data Retention: adopt; this replaces MOD-1 options A–C¶
- Layer A (Atul, once, in the OpenRouter web settings):
- Privacy settings → enable ZDR enforcement for every model group (Anthropic, OpenAI, Google, xAI, "All other models");
- turn off "allow providers that may train on inputs".
- Guardrail fields:
enforce_zdr_anthropic,enforce_zdr_openai,enforce_zdr_google,enforce_zdr_xai,enforce_zdr_other. - Layer B (code): every OpenRouter request from the harness, knowledge extraction and Aider sends
"provider": {"zdr": true, "data_collection": "deny"}. A per-request setting can only add restrictions, never remove the account's. - Layer C (client routing): if no ZDR endpoint serves the model ("no endpoints"), the harness falls through to the next model; it never retries without ZDR. The ledger records
zdr: onand the provider that served the call. - Effect: internal code may go to OpenRouter free models only through ZDR endpoints. Client data and secrets still never go to any cloud model.
- Unknown until measured: how many free models keep a ZDR endpoint. OpenRouter's docs don't say whether
openrouter/freesurvives ZDR. B22 measures it in the first week.
7.2 MkDocs + Material: adopt for the Documentation module¶
- What it gives: one static site built from the Markdown in our repositories (
docs/,docs/knowledge/, the registers, the decision records), with search and navigation. - Hosting: not public. Build it as a CI artefact (GitHub Actions / laptop runner) and view it locally (
mkdocs serve), or host it behind authentication later (sprint 3, alongside the MMDEP service). GitHub Pages for private repositories needs a paid plan. - Backlog: B23.
7.3 Aider: the suggestions mapped to our harness¶
| Suggestion | Verdict | How it fits |
|---|---|---|
1. Edit-apply-commit, /undo |
Partly | The harness already applies, tests and commits. Keep commits with the harness (one owner of branches, PRs and merges): Aider runs with --no-auto-commits. The harness may reuse Aider's commit-message text |
| 2. Architect mode (strong model plans, cheap model edits) | Yes, high value | Gemini Flash or Claude as architect; a local or free model as editor. That's how small Ollama models become useful. Test it in the B19 spike |
3. Closed-loop self-correction (--test-cmd, --lint-cmd, --auto-test) |
Yes, bounded | Aider's inner loop fixes quickly. The harness keeps the outer loop: budget, scope check, full-suite regression gate, PR, merge. Cap Aider's inner attempts so the two loops don't compete |
| 4. Repository map (tree-sitter) | Yes, high value | Our built-in engine sends whole scope files. Aider's compact map gives small models the right context with fewer tokens |
5. --chat-history-file audit trail |
Yes | Matches rule §21. Write it to docs/sessions/harness/<date>-issue-N.md in the worktree, so it's committed with the change |
| Search/replace and diff edit formats (against "lazy" or tired models) | Yes, but measure | DR-010 chose whole-file output because small models handled diffs badly in 2025. Aider's formats are the best-tested way to do diffs. B19 compares whole-file and diff per model |
| Multi-response stitching | Useful for Gemini and Claude | Comes free with Aider |
| "No placeholders" directive | Adopt now, without Aider | Add the directive to the harness prompt and a gate that rejects output containing placeholder patterns (# rest of code, ... on its own line, pass # TODO in new code). Backlog B24 |
/drop and context limiting |
Yes | Aider gets only the scope files with --file, plus the repository map; nothing else |
Corrections to the sample command:
- --cov-fail-under=80 would fail today, because coverage isn't measured yet (B21 reports it first). Use "coverage must not drop" once a baseline exists.
- --yes-always is safe only inside a disposable worktree with the harness checking afterwards that no file outside scope changed (it already does).
- Model IDs such as qwen-2.5-coder-32b:free come and go. Take them from config, not scripts.
Order: B24 now (cheap). B22 (ZDR) as soon as Atul has set Layer A. B19 (Aider spike, now including architect mode, repository map, edit-format comparison, chat history and ZDR parameters). Then the decision on Aider as the default engine for cx:2.
Options¶
Not stated separately in the original record.
Decision¶
6. Decision for Atul (superseded by §7.1: internal code allowed on OpenRouter free models only with ZDR)¶
- MOD-1: may OpenRouter free models see our internal code? The MMDEP tooling, API, MM-Cron and PE code hold no secrets or client data, but are proprietary. Some free providers log or train on prompts.
- Option A (recommended): allow free models only for
publicrepositories and MMDEP's own tooling. Keep the API, MM-Cron, PE and MCRM code on Gemini paid or local models. - Option B: allow all internal code, without secrets or client data, on free models.
- Option C: public only.
Today the harness already falls back to
openrouter-freeon MMDEP issues, which is Option A behaviour for MMDEP.
Consequences¶
Not stated separately in the original record.