TL;DR
- The H100 gives 80 GB at 3.35 TB/s, the H200 141 GB at 4.8 TB/s, the B200 192 GB at 8 TB/s.
- On PowerGPU the H100 SXM rents at $1.428 per hour, the H200 at $2.791 and the B200 at $5.425.
- Pick the H200 for long context and memory-bound work: its 43 percent bandwidth jump shows up almost one-to-one.
- The B200 delivers 2,250 FP16 TFLOPS, roughly 2.3× the H100, so it buys deadlines rather than cheaper FLOPS.
Specs that matter, side by side
| A100 SXM4 | H100 SXM | H200 | B200 | |
|---|---|---|---|---|
| Architecture | Ampere | Hopper | Hopper | Blackwell |
| VRAM | 80 GB HBM2e | 80 GB HBM3 | 141 GB HBM3e | 192 GB HBM3e |
| Memory bandwidth | 2.0 TB/s | 3.35 TB/s | 4.8 TB/s | 8 TB/s |
| FP16 tensor (dense) | 312 TF | 990 TF | 990 TF | 2,250 TF |
| Low-precision extras | — | FP8 | FP8 | FP8 + FP4 |
| NVLink | 600 GB/s | 900 GB/s | 900 GB/s | 1.8 TB/s |
Public NVIDIA figures, dense (non-sparsity) numbers.
Rental economics today
Specs age; the ratio of spec to price is the actual decision. Live from the sheet (snapshot 2026-09-14):
| Card | On-demand | Interruptible | $/hr per TB/s | $/hr per 100 FP16 TF |
|---|---|---|---|---|
| A100 SXM4 | $0.560 | $0.280 | $0.280 | $0.179 |
| H100 SXM | $1.428 | $0.714 | $0.426 | $0.144 |
| H200 | $2.791 | $1.395 | $0.581 | $0.282 |
| B200 | $5.425 | $2.712 | $0.678 | $0.241 |
Read the last two columns as "price of bandwidth" and "price of compute". The H100 usually wins compute per dollar; the H200 wins bandwidth per dollar; the B200 buys time.
Pick the H100 when…
- your model + optimizer fits in 80 GB per shard (most ≤34B full fine-tunes, all LoRA work);
- throughput per dollar is the metric — FP8 on Hopper remains the efficiency sweet spot;
- you want depth of supply: H100 pools are the deepest of the three, so interruptible slots are nearly always available.
Pick the H200 when…
- context length is the product: KV-cache at 128k eats 80 GB cards alive, 141 GB breathes;
- the workload is memory-bound (MoE inference, stencil HPC, giant batch serving) — the 43% bandwidth jump shows up almost 1:1;
- you can trade a $1.363/hr premium for fewer, fatter shards (less inter-GPU traffic, simpler parallelism).
Pick the B200 when…
- the deadline is the budget: ~2.3× H100 FP16 throughput means Monday's checkpoint on Friday;
- FP4/FP8 inference at scale — Blackwell's low-precision path is where its $/token leadership lives;
- the model shard needs 192 GB — some 100B+ configurations simply do not fit anything else per card.
The A100 dark horse
Four years old and still the best $/GB of HBM on the sheet: $0.560/hr for 80 GB. For ≤13B training runs, LoRA farms and FP64 simulation it routinely beats the newer cards on total job cost — the software stack is bulletproof and interruptible A100 supply is deep. Run the break-even math before assuming newer is cheaper.
Put the numbers to work
Every price in this guide is our live rate — fixed, ≥30% under the market median, billed per second. Deploy the exact setup above from the console in about 30 seconds, paid in crypto, no card and no KYC.
