--- title: "Run llama.cpp on a cloud GPU from $0.021/hr | PowerGPU" description: "Llama.cpp on a fixed-price cloud GPU: one-click template, running in ~30 s, from $0.021/hr on RTX 3060. Best GPUs, volumes, per-second billing." url: https://powergpu.ai/templates/llama-cpp last_modified: 2026-09-14T11:04:20+00:00 prices_as_of: 2026-09-14 site: PowerGPU (powergpu.ai) --- Template · LLM serving & chat · CUDA 12.9 # Run llama.cpp on a cloud GPU, in 30 seconds GGUF inference with a built-in server — the lightest way to run quantized models. llama.cpp runs GGUF models with CUDA offload and ships a lightweight OpenAI-compatible server. It is the most memory-frugal way to serve quantized models and the natural home for Q4/Q5/Q6 GGUF files from Hugging Face — including models that do not fit vLLM's formats. From **$0.021** /hr on an interruptible RTX 3060. Llama.cpp powergpu/llama-cpp (CUDA 12.9) (ARM) (SSH) (Jupyter) Pinned image, rebuilt on upstream releases. Override env, ports and disk at deploy. ### Running in ~30 s Pre-cached image, ports mapped, TLS terminated — the stack is working before you finish reading this. ### Template is free You pay the GPU price only: Llama.cpp on a RTX 3060 is $0.042/hr on-demand, $0.021/hr interruptible, billed per second. ### Volumes for state Models, datasets and outputs live on a $0.08/GB/mo volume; the instance stays disposable. ### Private by default Dedicated GPU, encrypted disk, crypto payments, no KYC, no stored IPs — Jupyter and SSH behind your own credentials. ## Best GPUs for Llama.cpp Three price points that run this template well — Good, Better, Best. Every model in the catalogue works; these are the value picks. | Tier | GPU | VRAM | On-demand | Interruptible | Why this card | | | --- | --- | --- | --- | --- | --- | --- | | (Good) | [RTX 3060](https://powergpu.ai/gpu/rtx-3060) | 12 GB | $0.042 | $0.021 | 12 GB is enough for 7B–8B Q4 GGUF — the cheapest chat endpoint on the sheet. | [Deploy](https://cloud.powergpu.ai/?gpu=rtx-3060) | | (Better) | [RTX 4090](https://powergpu.ai/gpu/rtx-4090) | 24 GB | $0.327 | $0.163 | 24 GB runs 14B–32B quantized models with full GPU offload. | [Deploy](https://cloud.powergpu.ai/?gpu=rtx-4090) | | (Best) | [RTX A6000](https://powergpu.ai/gpu/rtx-a6000) | 48 GB | $0.281 | $0.140 | 48 GB for 70B Q4 GGUF on a single card. | [Deploy](https://cloud.powergpu.ai/?gpu=rtx-a6000) | Need more VRAM? The [full catalogue](https://powergpu.ai/gpus) lists all 80 models with live availability; the [VRAM guide](https://powergpu.ai/guides/llm-vram-requirements) sizes models to cards. ## Deploy Llama.cpp from the console, CLI or API Pick the template in the [console](https://cloud.powergpu.ai/) deploy bar, or script it: - **Console** — filter by GPU, choose *Llama.cpp* in the template picker, set disk and env, deploy. - **CLI** — pip install powergpu, then the command on the right. [CLI reference](https://powergpu.ai/cli). - **API** — POST /v1/instances with "template": "llama-cpp". [REST reference](https://powergpu.ai/api). - **Own image** — any OCI reference works too; we inject the NVIDIA runtime. [Template docs](https://powergpu.ai/docs/templates). *deploy — llama-cpp* ``` $ powergpu launch --gpu rtx-3060 --template llama-cpp \ --disk 100 --volume models:/workspace/models ✓ instance i-7a41c0e2 running (27.9s) # Llama.cpp · RTX 3060 · $0.042/hr · per second # https://i-7a41c0e2.powergpu.ai:8888 (TLS) $ powergpu stop i-7a41c0e2 # billing ends this second ``` ## What is inside the Llama.cpp template - **Image**: powergpu/llama-cpp - **CUDA**: CUDA 12.9 - **Access**: also builds for ARM hosts · SSH shell · JupyterLab on a mapped port - **Category**: [LLM serving & chat](https://powergpu.ai/templates#llm) - **Storage**: Instance NVMe disk (sized at deploy) + optional [network volumes](https://powergpu.ai/products/volumes) - **Billing**: GPU price only, per second — no template fee, no setup fee ## Other llm serving & chat templates - [vLLM on a cloud GPU](https://powergpu.ai/templates/vllm) — Production LLM serving with PagedAttention — an OpenAI-compatible endpoint from any HF model. - [vLLM Omni on a cloud GPU](https://powergpu.ai/templates/vllm-omni) — vLLM extended for multimodal models — vision and audio inputs on the same fast server. - [SGLang on a cloud GPU](https://powergpu.ai/templates/sglang) — High-throughput serving with RadixAttention — excels at structured and agentic workloads. - [Ollama on a cloud GPU](https://powergpu.ai/templates/ollama) — Pull and run quantized models in one command, with a clean REST API. - [Open WebUI (Ollama) on a cloud GPU](https://powergpu.ai/templates/open-webui-ollama) — A full chat UI over Ollama — conversations, RAG and model management in the browser. - [Oobabooga Text Gen on a cloud GPU](https://powergpu.ai/templates/oobabooga-text-gen) — The text-generation WebUI & API — load GPTQ, EXL2 or GGUF with extensions. - [All 37 templates](https://powergpu.ai/templates) ## Llama.cpp on a cloud GPU: FAQ Environment variables, ports and custom images are covered in the [template docs](https://powergpu.ai/docs/templates). **How much does it cost to run Llama.cpp on a cloud GPU?** Only the GPU price — the template is free. From $0.021/hr on an interruptible RTX 3060, $0.042/hr on-demand on a RTX 3060. Billing is per second, so an hour of tinkering costs an hour, not a day. Storage is $0.08/GB/month. **How long does Llama.cpp take to start?** About 30 seconds from the deploy click: the image is pre-cached on hosts, ports are mapped and TLS is terminated for you. Restarting a stopped instance is faster, and your disk is exactly as you left it. **Can I keep my models and outputs between sessions?** Yes — attach a volume at deploy. Everything on the volume survives instance destruction and mounts on the next instance in the region in seconds, at $0.08/GB/month. The instance disk itself survives stop/start but not destroy. **Can I split a model between GPU and CPU RAM?** Yes — set the number of offloaded layers (-ngl) at launch. Machines with large RAM let you run models bigger than VRAM at reduced speed, which is handy for occasional 70B use on a cheap card. --- *About PowerGPU:* PowerGPU (powergpu.ai) is a cloud GPU rental service offering 80 NVIDIA GPU models — from the RTX A2000 at $0.024/hr to the B300 — at fixed prices set at least 30% below the public GPU marketplace median and re-checked weekly (H100 SXM: $1.428/hr on-demand). Billing is per second with no minimums; payment is crypto only (USDT, BTC, XMR, ETH, SOL, LTC, TRX) with no KYC. Instances run in Tier-III datacenters across 32 regions with a 99.9% uptime SLA and deploy in about 30 seconds from the web console (cloud.powergpu.ai) or the REST API. Source: https://powergpu.ai/templates/llama-cpp · Site index for AI assistants: https://powergpu.ai/llms.txt · Full content: https://powergpu.ai/llms-full.txt