---
title: "Cloud GPUs for LLM Training — H100/H200/B200 from $0.714/hr | PowerGPU"
description: "Train LLMs on HBM GPUs with NVLink and InfiniBand: H100 SXM at $1.428/hr fixed, $0.714/hr interruptible. Checkpoint patterns, cluster pods, real cost math."
url: https://powergpu.ai/use-cases/llm-training
last_modified: 2026-09-14T11:04:20+00:00
prices_as_of: 2026-09-14
site: PowerGPU (powergpu.ai)
---

Use case · LLM training

# LLM training on cloud GPUs: the same silicon, no auction

Pre-training and continued pre-training live on HBM parts: H100 SXM at **$1.428** /GPU-hr fixed (market median $2.04), NVLink in-node, InfiniBand across nodes, and interruptible capacity at half price for every checkpointed epoch.

## The training cards, ranked

Full comparison in the [H100 vs H200 vs B200 guide](https://powergpu.ai/guides/h100-vs-h200-vs-b200).

| Tier | GPU | VRAM | On-demand | Interruptible | Why this card |  |
| --- | --- | --- | --- | --- | --- | --- |
| (Good) | [A100 SXM4](https://powergpu.ai/gpu/a100-sxm4) | 80 GB | $0.560 | $0.280 | Proven 80 GB HBM2e workhorse — unbeatable $/step for ≤13B experiments and LoRA-heavy labs. | [Deploy](https://cloud.powergpu.ai/?gpu=a100-sxm4) |
| (Better) | [H100 SXM](https://powergpu.ai/gpu/h100-sxm) | 80 GB | $1.428 | $0.714 | The default: FP8 transformer engine, 3.35 TB/s, deep supply in every major region. | [Deploy](https://cloud.powergpu.ai/?gpu=h100-sxm) |
| (Best) | [B200](https://powergpu.ai/gpu/b200) | 192 GB | $5.425 | $2.712 | 192 GB HBM3e and Blackwell FP4/FP8 — shortest wall-clock when the deadline is the budget. | [Deploy](https://cloud.powergpu.ai/?gpu=b200) |

## Cost of a real run, line by line

Continued pre-training, 8× H100 SXM, 72 hours, interruptible with volume checkpoints:

- **GPU time**: 8 × 72 h × $0.714 · **=**: $411.26
- **Checkpoint volume 500 GB**: 3 days × $40.00/mo · **=**: $4.00
- **Dataset ingress 400 GB**: 400 × $0.01 · **=**: $4.00
- **Total**:  · **≈**: **$419**

The market-median equivalent of the GPU line alone: about $588 at spot auctions you have to babysit — or $1,176 on-demand.

## Patterns that make it cheap

- **Checkpoint to a volume** every N steps — interruptions restart free, from disk.
- **Tune on 1×, train on 8×** — debug the config at one-eighth the burn rate.
- **Pack with FP8/BF16** — the H100/H200 transformer engine is the discount nobody uses.
- **Destroy, keep the volume** — datasets stay warm at $0.08/GB/mo, GPUs bill zero.

Templates: [PyTorch, axolotl](https://powergpu.ai/templates) (FSDP-ready) — or your own image with [a full VM](https://powergpu.ai/products/vms) for exotic stacks.

## LLM training GPUs: FAQ

Sizing math: [VRAM requirements guide](https://powergpu.ai/guides/llm-vram-requirements).

**Which GPU should I train an LLM on?**

For serious runs: H100 SXM ($1.428/hr) is the price-performance default, H200 adds 141 GB for longer context and bigger micro-batches, B200 leads when wall-clock time is the constraint. For sub-13B experiments, 8× A100 nodes at $0.560/GPU-hr are hard to beat.

**On-demand or interruptible for training?**

Interruptible, almost always — training checkpoints anyway, and −50% compounds over hundreds of GPU-hours. A 72-hour run on 8× H100 costs about $411 interruptible vs $823 on-demand. Keep on-demand for the final, deadline-bound run.

**How do multi-node runs connect?**

Single machines scale to 8× with NVLink. Past that, clusters connect 8-GPU nodes over InfiniBand with NCCL pre-tuned — see the clusters page for pod pricing and all-reduce figures.

---

*About PowerGPU:* PowerGPU (powergpu.ai) is a cloud GPU rental service offering 80 NVIDIA GPU models — from the RTX A2000 at $0.024/hr to the B300 — at fixed prices set at least 30% below the public GPU marketplace median and re-checked weekly (H100 SXM: $1.428/hr on-demand). Billing is per second with no minimums; payment is crypto only (USDT, BTC, XMR, ETH, SOL, LTC, TRX) with no KYC. Instances run in Tier-III datacenters across 32 regions with a 99.9% uptime SLA and deploy in about 30 seconds from the web console (cloud.powergpu.ai) or the REST API.

Source: https://powergpu.ai/use-cases/llm-training · Site index for AI assistants: https://powergpu.ai/llms.txt · Full content: https://powergpu.ai/llms-full.txt