---
title: "Cloud GPUs for LLM Inference — Lowest $/Token with vLLM | PowerGPU"
description: "Serve LLMs with vLLM on fixed-price GPUs: RTX 5090 at $0.439/hr for 7B–14B, L40S for 32B, H100 for 70B. $/token math, tensor parallelism, serverless scale-to-zero."
url: https://powergpu.ai/use-cases/llm-inference
last_modified: 2026-09-14T11:04:20+00:00
prices_as_of: 2026-09-14
site: PowerGPU (powergpu.ai)
---

Use case · LLM inference

# LLM inference GPUs: tokens priced like a utility bill

Fixed GPU prices turn $/token into arithmetic: pick the smallest card that holds your model + KV-cache, saturate it with vLLM continuous batching, divide. A RTX 5090 at **$0.439** /hr serves a 8B model for pennies per million tokens.

## The serving cards, by model size

VRAM decides; price-per-VRAM ranks the candidates.

| Tier | GPU | VRAM | On-demand | Interruptible | Why this card |  |
| --- | --- | --- | --- | --- | --- | --- |
| (Good) | [RTX 5090](https://powergpu.ai/gpu/rtx-5090) | 32 GB | $0.439 | $0.219 | 32 GB GDDR7 — the $/token king for 7B–14B FP16 and 32B 4-bit chat models. | [Deploy](https://cloud.powergpu.ai/?gpu=rtx-5090) |
| (Better) | [L40S](https://powergpu.ai/gpu/l40s) | 48 GB | $0.514 | $0.257 | 48 GB and server-grade cooling for 24/7 endpoints; 32B class and long-context 14B. | [Deploy](https://cloud.powergpu.ai/?gpu=l40s) |
| (Best) | [H100 PCIE](https://powergpu.ai/gpu/h100-pcie) | 80 GB | $1.867 | $0.933 | 80 GB HBM3 for 70B quantized on one card — or FP8 for maximum throughput per dollar. | [Deploy](https://cloud.powergpu.ai/?gpu=h100-pcie) |

Bigger than 70B? Tensor-parallel across 2–8× cards on one machine — the per-GPU price never changes. Sizing tables: [VRAM guide](https://powergpu.ai/guides/llm-vram-requirements).

## $/million tokens, computed honestly

Throughput varies with context and batch mix, so we publish the formula, not a marketing number:

$/M tokens = GPU $/hr ÷ (tokens/s × 3600) × 1,000,000

A RTX 5090 at $0.439/hr sustaining ~2,800 tok/s of Llama-3.1-8B under continuous batching → **$0.044** /M tokens. The same card interruptible halves it.

Benchmark your own model in 10 minutes — the [serving guide](https://powergpu.ai/guides/serve-llm-vllm) ships the exact vllm bench command.

## Serving patterns

- **Reserved + vLLM** for the baseline load — −35% on the card that never sleeps.
- **Serverless burst** on top — scale-to-zero workers absorb the spikes.
- **Model library volume** mounted read-only — new workers skip the 40 GB download.
- **Quantize first** — AWQ/GPTQ 4-bit doubles the models a card can hold, rarely hurts chat quality.

Templates: [vLLM, Ollama, TGWUI](https://powergpu.ai/templates) — all OpenAI-compatible out of the box.

## LLM inference GPUs: FAQ

**What is the cheapest way to serve a 7B–14B model?**

A single RTX 5090 at $0.439/hr running vLLM. At typical chat throughput that lands well under $0.15 per million output tokens — an order of magnitude below API-provider pricing for comparable open models.

**When do I need an 80 GB card?**

Roughly at 70B: a 4-bit 70B needs ~44 GB plus KV-cache, so a single H100 PCIE (80 GB) or 2× L40S with tensor parallelism. FP16 70B wants 2× 80 GB. Below 32B, 32–48 GB cards are the value zone.

**Instances or serverless for serving?**

Steady saturating traffic → a reserved instance (−35%) you keep hot. Spiky or overnight-idle traffic → serverless with scale-to-zero. Same per-second prices; the difference is who pays for idle.

---

*About PowerGPU:* PowerGPU (powergpu.ai) is a cloud GPU rental service offering 80 NVIDIA GPU models — from the RTX A2000 at $0.024/hr to the B300 — at fixed prices set at least 30% below the public GPU marketplace median and re-checked weekly (H100 SXM: $1.428/hr on-demand). Billing is per second with no minimums; payment is crypto only (USDT, BTC, XMR, ETH, SOL, LTC, TRX) with no KYC. Instances run in Tier-III datacenters across 32 regions with a 99.9% uptime SLA and deploy in about 30 seconds from the web console (cloud.powergpu.ai) or the REST API.

Source: https://powergpu.ai/use-cases/llm-inference · Site index for AI assistants: https://powergpu.ai/llms.txt · Full content: https://powergpu.ai/llms-full.txt