--- title: "Cheapest cloud GPU for Ollama (2026): 8B to 70B models, by the hour" description: "Which rented card runs each Ollama model size at Q4, indicative tokens per second per card, and what an always-on private assistant costs per month." url: https://powergpu.ai/guides/cheapest-gpu-for-ollama last_modified: 2026-09-14T11:04:20+00:00 prices_as_of: 2026-09-14 site: PowerGPU (powergpu.ai) --- Guide · Costs & pricing # Cheapest cloud GPU for Ollama (2026): 8B to 70B models, by the hour Which rented card runs each Ollama model size at Q4, indicative tokens per second per card, and what an always-on private assistant costs per month. 8 min read Published 2026-09-03 Updated 2026-09-03 (prices live from the sheet) TL;DR - An RTX 3060 12 GB is the cheapest card that runs Ollama well, covering every 7B-8B model and most 14B models at Q4. - Ollama pulls 4-bit weights by default, so budget about 0.6 GB per billion parameters plus 1-3 GB for context. - That puts 8B on 12 GB cards, 27B-32B on 24 GB, and 70B on a 48 GB RTX A6000 at $0.281 per hour on PowerGPU. - Ollama is single-stream: expect about 120 tokens per second for 8B on an RTX 4090, and use vLLM when many users share the card. ## How much VRAM each Ollama model needs Ollama pulls 4-bit (Q4_K_M) weights by default, so the rule of thumb is **about 0.6 GB per billion parameters, plus 1–3 GB for context**. That puts 8B models on 12 GB cards, 14B on 12–16 GB, 27B–32B on 24 GB, and 70B on 48 GB. Go up a class if you want 32k+ context or FP16 weights (:fp16 tags need 2 GB per billion). ## The table: model → cheapest card → speed → cost Tokens per second are indicative single-stream numbers for Ollama with the whole model on the GPU (llama.cpp CUDA backend); your prompt length and quantization move them. Prices are today's on-demand rates; interruptible halves them. | Ollama model | Weights | Card | On-demand | ~tok/s | $ / M tokens | Note | | --- | --- | --- | --- | --- | --- | --- | | llama3.1:8b | 4.9 GB | [RTX 3060](https://powergpu.ai/gpu/rtx-3060) (12 GB) | $0.042 | ~35 | $0.33 | The cheapest card that runs 8B comfortably | | llama3.1:8b | 4.9 GB | [RTX 4090](https://powergpu.ai/gpu/rtx-4090) (24 GB) | $0.327 | ~120 | $0.76 | Snappy chat; room for 32k context | | qwen2.5:14b | 9.0 GB | [RTX 4070](https://powergpu.ai/gpu/rtx-4070) (12 GB) | $0.075 | ~45 | $0.46 | 12 GB is enough at Q4 | | qwen2.5:14b | 9.0 GB | [RTX 5090](https://powergpu.ai/gpu/rtx-5090) (32 GB) | $0.439 | ~110 | $1.11 | 1.79 TB/s GDDR7 shows in tokens/s | | gemma3:27b | 17 GB | [RTX 3090](https://powergpu.ai/gpu/rtx-3090) (24 GB) | $0.108 | ~25 | $1.20 | 24 GB with context headroom at the lowest rate | | qwen2.5:32b | 20 GB | [RTX 4090](https://powergpu.ai/gpu/rtx-4090) (24 GB) | $0.327 | ~40 | $2.27 | Tight but fine on 24 GB with 8k context | | qwen2.5:32b | 20 GB | [RTX 5090](https://powergpu.ai/gpu/rtx-5090) (32 GB) | $0.439 | ~62 | $1.97 | 32 GB gives long-context breathing room | | llama3.3:70b | 43 GB | [RTX A6000](https://powergpu.ai/gpu/rtx-a6000) (48 GB) | $0.281 | ~15 | $5.20 | Cheapest single card that holds 70B | | llama3.3:70b | 43 GB | [L40S](https://powergpu.ai/gpu/l40s) (48 GB) | $0.514 | ~20 | $7.14 | Datacenter card for an always-on assistant | | llama3.3:70b | 43 GB | [H100 PCIE](https://powergpu.ai/gpu/h100-pcie) (80 GB) | $1.867 | ~38 | $13.65 | HBM bandwidth — the fastest single-card 70B | Single-stream $/M tokens looks expensive next to API providers because one chat at a time leaves most of the card idle — Ollama is a private-assistant tool, not a serving engine. For many users, [vLLM](https://powergpu.ai/templates/vllm) batches requests and drops the cost per token by 10–30×. ## An always-on private assistant, per month | Assistant | Card | 24/7 on-demand | Reserved (3 mo) | 8 h/day, workdays | | --- | --- | --- | --- | --- | | 8B chat (llama3.1:8b) | [RTX 3060](https://powergpu.ai/gpu/rtx-3060) | $31 | $20 | $7 | | 14B–32B (qwen2.5:32b) | [RTX 4090](https://powergpu.ai/gpu/rtx-4090) | $239 | $155 | $58 | | 70B (llama3.3:70b) | [RTX A6000](https://powergpu.ai/gpu/rtx-a6000) | $205 | $133 | $49 | | 70B, fast (llama3.3:70b) | [H100 PCIE](https://powergpu.ai/gpu/h100-pcie) | $1,363 | $885 | $329 | Add a volume for the model library (/root/.ollama, 30–100 GB at $0.08/GB/month) so pulled models survive stop/start and instance swaps. ## Setup in one command *Ollama on the cheapest 12 GB card* ``` powergpu launch --gpu rtx-3060 --template ollama --disk 40 --volume ollama:/root/.ollama # ✓ instance i-2b7e11c0 running (26.4s) · $0.042/hr # https://i-2b7e11c0.powergpu.ai:11434 (Ollama API, TLS) curl https://i-2b7e11c0.powergpu.ai:11434/api/pull -d '{"name":"llama3.1:8b"}' ``` Want a chat UI on top? The [Open WebUI template](https://powergpu.ai/templates/open-webui-ollama) bundles Ollama with a multi-user interface, RAG and model management behind TLS. ## Five ways to spend less - **Match the card to the model, not the ambition.** An 8B assistant on an H100 is 40× the price of the same assistant on a 3060 and does not answer better. - **Stop it when nobody is talking.** Per-second billing means a 9-to-5 assistant costs a third of a 24/7 one; the disk keeps the models. - **Reserve if it never sleeps.** −35% from three months — $0.027/hr for the 3060, $0.182/hr for the A6000. - **Q4 over FP16.** Half the VRAM class, nearly the same answers for chat; keep FP16 for evaluation runs. - **Two cheap cards can beat one expensive one.** 2× RTX 4090 (48 GB) at $0.654/hr runs 70B for less than one L40S — slower per token, cheaper per month. --- Put the numbers to work Every price in this guide is our live rate — fixed, ≥30% under the market median, billed per second. Deploy the exact setup above from the [console](https://cloud.powergpu.ai/) in about 30 seconds, paid in crypto, no card and no KYC. --- *About PowerGPU:* PowerGPU (powergpu.ai) is a cloud GPU rental service offering 80 NVIDIA GPU models — from the RTX A2000 at $0.024/hr to the B300 — at fixed prices set at least 30% below the public GPU marketplace median and re-checked weekly (H100 SXM: $1.428/hr on-demand). Billing is per second with no minimums; payment is crypto only (USDT, BTC, XMR, ETH, SOL, LTC, TRX) with no KYC. Instances run in Tier-III datacenters across 32 regions with a 99.9% uptime SLA and deploy in about 30 seconds from the web console (cloud.powergpu.ai) or the REST API. Source: https://powergpu.ai/guides/cheapest-gpu-for-ollama · Site index for AI assistants: https://powergpu.ai/llms.txt · Full content: https://powergpu.ai/llms-full.txt