LLD: API write-route security (keys, log-only, enforce)¶

status: draft (MMDEP Design, 27 Oct). High-risk design written by Claude; the API agent checks it against src/routes.py (Lane P step P1) and builds it. sprint: S2 (log-only on staging and production), S3 (enforce) decisions: TD-061, TD-062 overall: api/overall.md §6; phase: api/phase-1.md §3 stream A brief: relay 19 Oct (Atul, 10:28 IST), rows "Secure the API write routes" and "Notice: the API write routes will require a key"

Context¶

Every write route on api.moneymarche.com and stgapi.moneymarche.com accepts calls from anyone today. The only key, for /save-nav, is written into the code of two repositories (KN), so it has to be treated as public. Atul's brief asks for: - keys for machine writers; - a log-only period, so no consumer breaks; - enforcement on staging first, then production with his approval; - rotation and an audit log.

Writers come in three kinds, and one key scheme can't serve them all:

Class	Who calls	Examples (UNVERIFIED until the route inventory)	Can it hold a secret?
M: machine	Server-side code we run	`/save-nav`, `/jobs/upload`, `/api/ingest/*`, `/api/home-page-categories` (admin panel, PHP on the server), staging loader	Yes, in a server env file or CI secret
F: public form	A visitor's browser	enquiry, quote, booking, callback, lead	No: anything in browser JavaScript is public
U: user data	A signed-in client (webapp-sakai)	customer, KYC, bank details, nominee, family, address, income, portfolio	Needs a user session, not a shared key

This LLD builds M and puts guards on F. U gets its own LLD, because a session design is a separate piece of work. Before that LLD, the inventory must answer one question first: does any U route accept writes today with no session check? If one does, that's a client-data exposure, and it jumps the queue: its fix is a production change Atul approves the same day.

Decision¶

1. Every write route declares its class¶

Each POST, PUT, PATCH and DELETE handler carries exactly one decorator:

@require_key(scope="nav:write")       # class M
@public_form(limit="10/min")          # class F
@user_session(scope="customer:write") # class U (placeholder until the U LLD; behaves as log-only)

A guard test walks app.url_map and fails if any write route has none of the three. No route can be added later without a class (§Test plan).

2. Keys (class M)¶

Header: X-MM-Key: <key_id>.<secret>, where key_id is like mmcron-prod-01 and secret is 32 random bytes in base64url.
Server side: the API reads MM_KEYS_FILE, a JSON file (root, mode 600) at /opt/bitnami/api/api-secrets/keys.json for production and /opt/bitnami/api/staging-secrets/keys.json for staging. The file holds only the SHA-256 of each secret, never the secret itself:

{"keys": [{"id": "mmcron-prod-01", "consumer": "mm-cron", "env": "production",
           "scopes": ["nav:write", "ingest:*"], "sha256": "<hex>", "created": "2026-11-03", "status": "active"}]}

Check: split the header; look up key_id; hmac.compare_digest(sha256(secret), stored). The key's env must equal the server's ENV, so a staging key is refused on production. The route's scope must be in the key's scopes (ingest:* matches ingest:holdings).
Rotation: a consumer can hold two active keys at once (-01, -02), so a key is changed with no downtime. Keys are rotated quarterly (SEC-2) and at once on suspicion. Changing status to revoked and restarting takes effect immediately.
Issuing: tools/keys/new_key.py <consumer> <env> <scopes> (in the api repo) prints the secret once to the terminal and the JSON entry to add. A person runs it on the server, adds the entry, and gives the secret to the consumer's env file or CI secret variable. The secret never goes into a repo, chat, issue or log (brief rules).
The legacy key: the hard-coded /save-nav value is loaded as key id legacy-savenav with scope nav:write and status: legacy. It is accepted in log mode, refused in enforce mode, and logged as legacy every time it is used. Consumers that still send it show up by name in the log.

3. Modes¶

MM_WRITE_AUTH = off | log | enforce, set per environment in its env file.

Mode	Class M without a valid key	Class F	Class U
`off`	allowed, not logged (today's behaviour; only for rollback)	as today	as today
`log`	allowed, logged as `would-deny`	rate-limited (§4); over-limit logged, not blocked	logged
`enforce`	401 (missing or unknown key) or 403 (wrong env or scope), with a JSON error body	rate-limited; over-limit gets 429	logged (enforced by the U LLD)

4. Guards for public forms (class F)¶

Rate limit per client IP and route, in process (a token bucket in memory; the API runs as one python3 app.py gevent process (RB §1), so no shared store is needed; if it ever runs several workers, the limit applies per worker). Default 10 a minute and 100 a day, configurable per route. The client IP comes from Apache's X-Forwarded-For, trusting only the local proxy.
Origin check: Origin or Referer must be one of https://www.moneymarche.com, https://app.moneymarche.com, or https://staging.moneymarche.com (staging only). A mismatch is logged in log mode and refused in enforce mode.
Phase 2 (not here): a CAPTCHA token or a server-side relay through WordPress, if spam shows up in the log. Lead capture must not drop real leads (R1 value model), so nothing stricter is added without evidence.

5. Audit log¶

Every write call, in every mode, gets one JSON line in logs/write-auth.log: - time, env, route, method, class; - key_id (never the secret), consumer, decision (allow, would-deny, deny, rate-limited, legacy); - remote IP, user agent, status code.

A daily summary (tools/keys/summary.py) counts calls per consumer and decision. The collector reads only the counts. The enforcement gate is: zero would-deny and zero legacy for class M on 7 consecutive days.

6. Rollout¶

Step	Where	When	Who
1	Route inventory: every write route, its class, its consumers (from code plus the consumers' own lists, relay 19 Oct "Notice" row)	S2 day 1–2	API agent
2	Code merged with `MM_WRITE_AUTH=log` on staging	S2 day 3	API agent (low risk on staging)
3	Keys issued: MM-Cron (staging, production), admin panel (production), staging loader (staging), MMDEP collector (staging, read-only checks, no write scope)	S2 day 3–5	a person (Atul or Ajay), steps from the API agent
4	Release with `log` on production	S2 release (Atul approves)	Lane P
5	Consumers send their keys	S2 day 5 → S3	each project
6	`enforce` on staging	after one day of clean log on staging	API agent
7	`enforce` on production	after 7 clean days (gate in §5)	Lane P prepares, Atul approves
8	Legacy key removed from code and from the keys file	S4	API agent, person

Rollback: set MM_WRITE_AUTH=log (or off) in the env file and restart. No code change is needed.

C4 element¶

apiSecurity
apiIngest
apiCatalogue
apiApp

Evaluation¶

	Assessment
V value	High. It closes open writes to NAV and catalogue data, and the log gives the full writer inventory the "APIs only" work needs
S simplicity	One decorator per class, a JSON file, an env switch; standard library only (`hmac`, `hashlib`, `secrets`)
C cost	About 2–3 agent sessions plus a person's half hour per key install. No new service, no new dependency
D data	Secrets stay on the server and in CI secret variables; only hashes are on disk; logs carry key ids and IPs (operational data, internal class, kept 90 days)
L legal	Rate limits and origin checks don't change what personal data is collected. The U class (client data) needs its own DPDP-aware design (roadmap §3)

Test plan¶

Guard: tests/test_write_route_classes.py builds the app, lists every rule whose methods include POST, PUT, PATCH or DELETE, and fails if any lacks a class decorator. It also fails if a route has two.
Keys: a valid key → allowed. An unknown id, a wrong secret, a wrong env, or a missing scope → 401 or 403 in enforce and would-deny in log. The comparison uses hmac.compare_digest (checked by monkeypatching it and asserting it was called).
Legacy key: legacy decision in log; 401 in enforce.
Modes: each mode × each class gives the expected status and log decision (a table-driven test).
Rate limit: the 11th call in a minute from one IP gets rate-limited (log) or 429 (enforce); another IP is unaffected; the X-Forwarded-For header is trusted only when the immediate peer is 127.0.0.1.
No secrets in logs: run a request with a known fake secret, then grep the log output for it: none.
Keys file: a file with a plain secret instead of a hash is rejected at start-up (the length and hex check). A missing file in enforce mode stops start-up with a clear error; in log mode it starts and logs that every M call is would-deny.
Staging smoke after deploy: the existing smoke routes (all GET) still return 200; one POST /save-nav with the staging key returns 200, and the same call without a key returns 401 once enforce is on.

Risk¶

risk: high

Security and production. The failure that matters most: enforcement blocks an unknown writer (the admin panel, a forgotten cron job, a website form) and data or leads stop. Mitigations: - log-only first; - the 7-day zero gate; - per-environment switch with a restart-only rollback; - forms are never key-gated.

A second risk is a class U route that is unauthenticated today; the inventory's first output answers that.

Work items¶

#	Title	Risk	Model	Scope paths	Depends on	Cx	Tests
1	Route inventory: every write route × class × consumers; flag any class U route without a session check	high	chatgpt-api-mmcron	api: docs/security/write-routes.md		3
2	`require_key`, `public_form`, `user_session` decorators; keys file loader; modes; audit log	high	chatgpt-api-mmcron	api: src/security/, src/config.py, tests/test_write_auth.py	1	3	tests/test_write_auth.py
3	Guard test: every write route has exactly one class	medium	gemini-flash-lite	api: tests/test_write_route_classes.py	2	1	tests/test_write_route_classes.py
4	Apply the decorators to every route from the inventory	high	chatgpt-api-mmcron	api: src/routes.py	2,3	2	tests/test_write_route_classes.py
5	In-process rate limiter and origin check for class F	medium	gemini-flash-lite	api: src/security/ratelimit.py, tests/test_ratelimit.py	2	2	tests/test_ratelimit.py
6	`tools/keys/new_key.py` and `tools/keys/summary.py` with a runbook for the person	medium	gemini-flash-lite	api: tools/keys/, docs/runbooks/write-keys.md, tests/test_keys_tools.py	2	2	tests/test_keys_tools.py
7	Staging: deploy with `MM_WRITE_AUTH=log`; issue staging keys (person); then `enforce`	medium	chatgpt-api-mmcron	api: deploy notes; stgapi env (person)	4,5,6	2	staging smoke
8	Production release with `log`; 7-day gate report; `enforce` with Atul's approval	high	chatgpt-api-mmcron	api: release MR; production env (person)	7	3	gate report
9	Remove the hard-coded `/save-nav` key from api and mm-crons code	medium	gemini-flash-lite	api: src/config.py; mm-crons: lambda/navdata/lib/navdata.py	8	1	existing tests