Skip to content

Google Cloud · GKE Standard

What would self-hosting cost?

Compare selected open-weight models on Google Cloud with managed APIs. Start with a typical workload, then adjust your volume.

USD budget estimates · Prices checked 2026-10-12 · Quality and speed need a pilot.

Your workload

Advanced settings

Example: 8 people × 20 requests × 22 workdays, 2,000 input / 600 output tokens each.

Smallest first, in parameters (B = billion, T = trillion). Mixture-of-experts models need GPU memory for every weight but compute each token with only the active ones. Test each model on your tasks.

Leave blank to use the workload example. Count every call in an agent workflow.

A shorter window may need more GPUs. The result explains billed running time.

Pay for running hours. Commitments bill every hour of the term, even while idle.

Output budgets include reasoning. Use advanced settings for longer answers or heavy thinking.

Reset

Share the URL to save this scenario.

Start with an API on cost grounds

Claude Haiku 5.5 starts at $1.76/month for this token budget. Self-hosting may still be worth paying for control, residency, or customization. Test task quality before choosing.

Small team · 5–10 people · 3,520 calls/month over 22 active days.

GKE infrastructure / month

≈ $3.4k

GPU compute + cluster fee. Operations and additional services excluded.

Managed APIs / month

$1.76–$176

Same token budget across the APIs below. Model quality and actual token usage vary.

Illustrative setup

1 GPU

G4 · 1× RTX PRO 6000 · 96 GB. 1 replica (complete model copies).

Assumes 100 generated tokens/s per model copy, with 65% usable. This is a budget scenario; GPU speed, response time, and quality need a pilot.

Managed APIs · same token budget

Standard text rates, ordered by price. These models have different capabilities.

API monthly costs for this workload
ModelPer month
Claude Haiku 5.5 $1.76
Gemini 3.8 Flash $13.20
Claude Sonnet 5.5 $35.20
Claude Opus 5.5 $70.40
GPT-6 Astra $176.00

When could self-hosting cost less?

Break-even holds this setup, token mix, and usage window fixed. More GPUs or operating costs change the threshold. Quality still needs testing.

Claude Haiku 5.5
Beyond this setup's assumed capacity
Gemini 3.8 Flash
Beyond this setup's assumed capacity
Claude Sonnet 5.5
Beyond this setup's assumed capacity
Claude Opus 5.5
Beyond this setup's assumed capacity
GPT-6 Astra
About 67.2k calls/month

Method and limitations

  • Infrastructure only: whole GPU nodes and a $0.10/hour GKE cluster fee. Disks, network, load balancers, licenses, taxes, and operations are extra. US reference pricing; regional availability and prices vary.
  • Model weights plus 25% reserve select a compatible checkpoint and machine. Capacity assumes 100 output tokens/s per model copy, with 65% usable, regardless of GPU. This is an illustrative input, not a benchmark. Measure prompt processing, reasoning, KV cache, concurrency, and latency before sizing production.
  • Monthly running hours and each active-day window constrain capacity separately. Simple mode sizes complete model copies; advanced mode keeps your chosen setup fixed. Scheduled examples assume nodes run only during active windows. Commitments bill 730 hours/month, including idle time, and require GPU reservations.
  • API prices include input and output, including reasoning. The same token budget does not establish equal quality or actual usage. Break-even holds the setup fixed; more replicas and operating costs change it.

Sources · checked 2026-10-12

Snapshot checked 2026-10-12. Prices and runtime support change; follow the linked authority before committing spend.