--- title: "A100 vs H100 for fine-tuning (2026): cost per run, not per hour" description: "Same 80 GB, 2.7× the hourly price: when an H100 finishes a LoRA, QLoRA or full fine-tune fast enough to beat the A100 on total cost — live prices, break-even rule." url: https://powergpu.ai/guides/a100-vs-h100-for-fine-tuning last_modified: 2026-09-14T11:04:20+00:00 prices_as_of: 2026-09-14 site: PowerGPU (powergpu.ai) --- Guide · Choosing hardware # A100 vs H100 for fine-tuning (2026): cost per run, not per hour Same 80 GB, 2.7× the hourly price: when an H100 finishes a LoRA, QLoRA or full fine-tune fast enough to beat the A100 on total cost — live prices, break-even rule. 8 min read Published 2026-09-03 Updated 2026-09-03 (prices live from the sheet) TL;DR - Rent the H100 only when it finishes the job faster than the price ratio between the two cards; otherwise the A100 is cheaper per run. - On PowerGPU that ratio is $1.428 against $0.560 per hour, while BF16 LoRA and QLoRA runs see only 1.7-2.2×. - Both cards hold 80 GB, so fit never decides; the H100 adds 3.35 versus 2.0 TB/s of bandwidth and FP8. - FP8 full fine-tunes see 2.5-3× and tip the decision to the H100; without FP8 enabled you pay for bandwidth alone. ## The break-even rule Both cards hold 80 GB, so the choice is never about fit — it is about dollars per finished run. Today the [H100 SXM](https://powergpu.ai/gpu/h100-sxm) costs **$1.428/hr** and the [A100 SXM4](https://powergpu.ai/gpu/a100-sxm4) **$0.560/hr**: a ratio of **2.55×**. The rule that settles every job: rent the H100 if speedup(H100 / A100) > 2.55, otherwise rent the A100 Speedups below that line make the A100 cheaper per run even though the H100 is faster; above it the H100 is both faster and cheaper. The ratio moves with the weekly price re-check — the number above is live. ## What actually differs | | A100 SXM4 80 GB | H100 SXM 80 GB | Why it matters for fine-tuning | | --- | --- | --- | --- | | Memory | 80 GB HBM2e | 80 GB HBM3 | Same capacity — identical model fit | | Bandwidth | 2.0 TB/s | 3.35 TB/s | Optimizer steps and attention are bandwidth-bound: ~1.6× from this alone | | BF16 tensor (dense) | 312 TFLOPS | 990 TFLOPS | Compute-bound layers scale; small batches rarely saturate either | | FP8 | — | Transformer Engine | The H100's real weapon: 2.5–3× on full fine-tunes that use it | | NVLink | 600 GB/s | 900 GB/s | Multi-GPU FSDP all-gathers; matters past 2× cards | | Price today | $0.560/hr | $1.428/hr | Ratio 2.55× — the break-even line | ## Five fine-tuning jobs, costed Indicative wall-clock on the A100 and the speedup most teams measure on an H100 for the same recipe. The cost columns use today's on-demand prices; interruptible capacity halves both — fine-tunes checkpoint, so use it. | Job | Setup | A100 hours | H100 speedup | A100 cost | H100 cost | Cheaper | | --- | --- | --- | --- | --- | --- | --- | | QLoRA, Llama 3.1 8B, 10k pairs | BF16 compute, 4-bit base | 1.5 h | 1.8× | $0.84 | $1.19 | **A100** ((29% less)) | | LoRA, Llama 3.1 70B (4-bit base) | single card, 48 GB used | 3 h | 2.0× | $1.68 | $2.14 | **A100** ((22% less)) | | Full fine-tune, 8B, BF16 | 8× node, FSDP | 6 h | 2.2× | $26.88 | $31.16 | **A100** ((14% less)) | | Full fine-tune, 8B, FP8 (Transformer Engine) | 8× node, FSDP | 6 h | 3.0× | $26.88 | $22.85 | **H100** ((15% less)) | | DPO / RLHF pass, 8B | policy + reference | 2.5 h | 2.4× | $1.40 | $1.49 | **A100** ((6% less)) | The pattern is consistent: adapter methods in BF16 favour the A100; anything that engages FP8 or runs long enough for bandwidth to dominate favours the H100. Wall-clock is the tie-breaker — an H100 run that costs 10% more but finishes a day earlier is usually the right call for a deadline. ## Decision table | You are doing | Rent | Because | | --- | --- | --- | | QLoRA / LoRA on ≤13B, BF16 | [RTX 4090](https://powergpu.ai/gpu/rtx-4090) or [A100 PCIe](https://powergpu.ai/gpu/a100-pcie) | Fits in 24–80 GB; speedups under the break-even line | | LoRA on 70B (4-bit base), single card | [A100 SXM4](https://powergpu.ai/gpu/a100-sxm4) | Bandwidth-bound but 2× speedup stays under 2.6× | | Full fine-tune, BF16, multi-GPU | A100 SXM4 unless deadline-bound | ≈2.2× speedup is close to the line — pick by wall-clock | | Full fine-tune with FP8 (TE, NeMo, TorchAO) | [H100 SXM](https://powergpu.ai/gpu/h100-sxm) | 2.5–3× speedup beats the price ratio outright | | Sequence lengths ≥32k or big micro-batches | H100 SXM (or [H200](https://powergpu.ai/gpu/h200)) | Memory bandwidth and, on the H200, 141 GB | ## Making either card cheaper - **Go interruptible.** Every job above checkpoints; the flat −50% turns an A100 into $0.280/hr and an H100 into $0.714/hr. - **Enable FP8 before renting an H100.** Without it you are paying for bandwidth only; with it the H100 earns its price. - **Tune on 1×, train on 8×.** Debug the config on a single card at one-eighth the burn rate, then scale. - **Datasets and checkpoints on a volume** at $0.08/GB/month — the trainer instance stays disposable. - **Compare the neighbours.** [H100 SXM vs A100 SXM4](https://powergpu.ai/compare/h100-sxm-vs-a100-sxm4) and [the PCIe pair](https://powergpu.ai/compare/h100-pcie-vs-a100-pcie) keep live numbers; the [QLoRA walkthrough](https://powergpu.ai/guides/fine-tune-llm-qlora) has the exact axolotl config. --- Put the numbers to work Every price in this guide is our live rate — fixed, ≥30% under the market median, billed per second. Deploy the exact setup above from the [console](https://cloud.powergpu.ai/) in about 30 seconds, paid in crypto, no card and no KYC. --- *About PowerGPU:* PowerGPU (powergpu.ai) is a cloud GPU rental service offering 80 NVIDIA GPU models — from the RTX A2000 at $0.024/hr to the B300 — at fixed prices set at least 30% below the public GPU marketplace median and re-checked weekly (H100 SXM: $1.428/hr on-demand). Billing is per second with no minimums; payment is crypto only (USDT, BTC, XMR, ETH, SOL, LTC, TRX) with no KYC. Instances run in Tier-III datacenters across 32 regions with a 99.9% uptime SLA and deploy in about 30 seconds from the web console (cloud.powergpu.ai) or the REST API. Source: https://powergpu.ai/guides/a100-vs-h100-for-fine-tuning · Site index for AI assistants: https://powergpu.ai/llms.txt · Full content: https://powergpu.ai/llms-full.txt