GPU Cloud Cost Calculator
Estimate the cost of renting cloud GPUs like A100, H100, and RTX 4090 for AI training and inference.
GPU Type
Number of GPUs
Hours per month
hrs/mo
Cost per Month
$238.40
Cost per Hour
$1.49
Cost per Year
$2,860.80
Compare across GPU types
| GPU | Tier | $/hr (total) | $/month |
|---|---|---|---|
| NVIDIA T4 (16GB) | Budget / Inference | $0.20 | $32.00 |
| NVIDIA RTX 4090 (24GB) | Community Cloud | $0.34 | $54.40 |
| NVIDIA L4 (24GB) | Budget / Inference | $0.39 | $62.40 |
| NVIDIA A10 (24GB) | Community Cloud | $0.75 | $120.00 |
| NVIDIA A100 40GB | Community Cloud | $1.10 | $176.00 |
| NVIDIA A100 80GB | Community Cloud | $1.49 | $238.40 |
| NVIDIA H100 80GB PCIe | Community Cloud | $1.99 | $318.40 |
| NVIDIA H100 80GB SXM | Community Cloud | $2.99 | $478.40 |
| NVIDIA A100 80GB (8-GPU instance) | AWS / Azure per-GPU | $3.40 | $544.00 |
| NVIDIA H100 80GB (8-GPU instance) | AWS per-GPU | $6.88 | $1,100.80 |
How to estimate GPU cloud costs
Cloud GPU pricing depends heavily on the provider tier. "Community cloud" marketplaces (RunPod, Vast.ai, Lambda) offer the cheapest on-demand rates because they pool spare capacity, while hyperscalers (AWS, Azure, GCP) charge more per GPU but bundle enterprise SLAs, networking, and support. Spot/preemptible instances can cut costs further (often 40-70% off on-demand) but can be reclaimed with little notice, so they're best for fault-tolerant training jobs rather than production inference.
Which GPU do I actually need?
T4 / L4 — fine for small-model inference, embeddings, and light fine-tuning.
RTX 4090 / A10 — solid for most fine-tuning jobs and mid-size model inference on a budget.
A100 40GB/80GB — the standard for training and fine-tuning models up to ~13-70B parameters.
H100 — needed for large-scale pretraining or serving the largest open-weight models with high throughput; the extra cost over A100 is usually only worth it at scale.
FAQs
Is this pricing accurate?
These are approximate on-demand rates checked against public provider pricing as of August 2026. GPU cloud pricing is volatile and varies by region, availability, and contract length — treat this as a starting estimate, not a quote.
Should I rent or buy a GPU?
As a rough rule of thumb, if your projected usage exceeds roughly 6-9 months of continuous rental cost, it's often worth pricing out owning hardware instead — though renting still wins for spiky or short-term workloads since it avoids upfront capital and depreciation risk.