Skip to content

DR-004 — Hosting growth path, including running AI models

Converted from docs/decisions/DR-004-hosting-growth-and-models.md to MADR (backlog B25). The text is unchanged; its sections are grouped under the MADR headings below, in their original order within each group.

Context

Why this order

  • Cost: a GPU running all month costs ₹36,000 to ₹60,000. The same money buys a large volume of API calls, and our current AI use (agents, classification, summaries) is small.
  • Compliance: client data (PAN, holdings, policies) should stay in India and with providers you have approved. The India regions of AWS, GCP and E2E all qualify; Hetzner does not. Check SEBI/IRDAI guidance with your compliance adviser before any client-facing AI feature.
  • Simplicity: one cloud (AWS) for servers keeps credentials, backups and monitoring in one place. GPUs can live elsewhere because they are rented by the hour and hold no permanent data.
  • Check later: the IndiaAI Mission offers subsidised GPU access to Indian companies; worth checking eligibility before any long GPU commitment.

Options

Not stated separately in the original record.

Decision

Recommendation

  1. Now (October): keep AWS on hold for the two weeks agreed (D12). The laptop carries the collector and runner meanwhile.
  2. Tools server (November): one small AWS server in Mumbai (ap-south-1), next to production on Lightsail. Same account family, same region as production, India data residency.
  3. AI features: use hosted model APIs first (Claude, Gemini, OpenAI), routed by data class. Public and internal data can go to them on paid terms; client data only to a provider and region you approve. No GPU server until there is a measured need.
  4. Own models (later, only on a trigger): rent GPUs by the hour in India and switch them off when idle. Cheapest found: E2E Networks (Indian provider); GCP Mumbai next; AWS Mumbai if staying in one cloud matters more than price.

Triggers for step 4: monthly API spend for one feature above roughly ₹40,000; a data class you won't send to an API provider; or a latency or offline need.

Consequences

Prices found (one NVIDIA L4 GPU, 24 GB, enough for 7–13B models)

Provider Region Per hour 24×7 per month (approx.) Notes
E2E Networks India ₹49 + GST ₹35,800 + GST Also L40S ₹102, A100 80 GB ₹189, H100 ₹256 per hour; small CPU VM from ₹894 a month
Google Cloud g2-standard-4 Mumbai $0.74 $537 1-year commitment saves only about 8%
AWS g6.xlarge Mumbai $0.97 $705 Spot from $0.53 (can be interrupted)
Hetzner GEX45 Europe only — €214 + €209 setup Cheapest dedicated GPU, but outside India: not for client data

Running a GPU only during working hours (10 hours a day, 22 days) cuts the monthly figures to about 30%.

Sources