Glossary · 85 terms · prices live from the sheet

Cloud GPU glossary: the vocabulary, defined

Every term you meet when renting GPUs — billing models, memory types, interconnects, quantization, serving stacks — defined in one or two sentences that stand on their own, with the current numbers where a number helps.

Full price sheet GPU catalogue

Pricing & billing 17

Cost per image: The right unit for diffusion workloads: the hourly rate divided by images generated per hour. A faster card at a higher hourly price frequently produces cheaper images, which is why hourly price alone is a poor guide for Stable Diffusion or Flux. More
Cost per million tokens: The unit that makes self-hosted inference comparable to an API: the hourly GPU rate divided by the tokens the GPU produces in an hour. It falls sharply with continuous batching, so a card that looks expensive per hour is often the cheapest per token. More
Egress (bandwidth): Data leaving the provider network, and the fee charged for it. Egress is a common source of surprise bills because hyperscalers meter it in tiers and can charge more than the compute for data-heavy jobs. PowerGPU charges a flat $0.01 per GB in and out, in every region. More
GPU cloud: A service that rents access to GPU servers by the hour or the second instead of selling the hardware. It covers hyperscalers, specialist providers and peer-to-peer marketplaces, which differ mainly in who owns the machines, how prices are set and what uptime is promised. More
GPU marketplace: A platform where independent hosts list their own machines and renters bid or buy directly. Prices are the lowest in the market and move constantly, but hardware quality, uptime and support vary host by host, so the renter carries the reliability risk. More
GPU-hour: One GPU running for one hour: the unit almost every rental price is quoted in. It is per GPU, not per machine, so an 8-GPU instance running one hour consumes eight GPU-hours. On PowerGPU the per-GPU rate is identical from 1x to 8x, with no multi-GPU premium. More
Hyperscaler: One of the very large general-purpose clouds that sell GPUs alongside hundreds of other services. They offer deep integration and enterprise contracts, and typically charge several times the specialist rate for the same NVIDIA card, with metered egress and committed-use discounts. More
Instance storage pricing: What the disk attached to a rented GPU costs, usually per GB per month and billed whether or not the instance is running. PowerGPU charges $0.08 per GB per month, metered per second, at the same rate running or stopped; the meter ends when the instance is destroyed. More
Interruptible instance (spot, preemptible): A discounted instance the provider may pause or reclaim when capacity is needed elsewhere. Most clouds run it as an auction whose price and eviction risk move hourly. PowerGPU charges a flat 50% of on-demand with no bidding, keeps the disk and re-queues the job automatically: $0.714 per hour for an H100 SXM. More
Market-median pricing: Setting a price as a published ratio of the middle market rate for the same hardware, rather than by list, auction or negotiation. PowerGPU uses the public GPU marketplace median for each model multiplied by 0.70 and rounded down, re-checked weekly, so the discount is at least 30% and verifiable by arithmetic. More
Neocloud: A cloud provider built specifically around AI accelerators rather than general-purpose compute. Neoclouds buy GPUs in bulk, run them in leased datacenter space and compete on price and availability instead of breadth of services. More
On-demand instance: Capacity reserved for you the moment you ask for it and held until you release it, at a rate that cannot be outbid or reclaimed. On PowerGPU an on-demand GPU costs the public marketplace median times 0.70 — an H100 SXM at $1.428 per GPU-hour — and the rate is locked for the life of the instance. More
Per-second billing: Charging for the exact runtime rather than rounding up to the hour. A 47-minute job costs 47 minutes. It matters most for bursty work such as render frames, batch inference and experiments, where hourly rounding can inflate a bill by a third. PowerGPU bills GPU time, storage and bandwidth per second. More
PowerScore: PowerGPU's index of dense FP16 tensor throughput, normalised so an RTX 3090 scores 100. It gives a single comparable number across generations when raw TFLOPS figures are quoted under different sparsity and precision conventions. It measures compute only, not memory capacity or bandwidth. More
Price floor: A published ceiling on what a provider will charge, expressed against an external reference. PowerGPU commits to never exceeding 70% of the public marketplace median for any model; because the reference is public and the sheet is served as JSON, the commitment can be audited at any time without an account. More
Rent-versus-buy break-even: The utilisation level at which owning a GPU becomes cheaper than renting one. It compares the purchase price plus hosting, power and cooling against the rental rate times the hours actually used. Because most teams run far below full utilisation, the honest break-even is usually years rather than months. More
Reserved instance: Capacity committed for a fixed term in exchange for a lower hourly rate. PowerGPU reserves at 65% of on-demand on a three-month term, with the rate locked for the whole term: $0.928 per hour for an H100 SXM. It pays off above roughly two-thirds utilisation. More

GPU hardware 19

CUDA core: The general-purpose shader unit of an NVIDIA GPU, used for graphics, simulation and any code that is not a matrix multiply. Counts are useful for rendering and scientific workloads but are a poor predictor of AI performance, which is dominated by tensor cores and memory bandwidth. More
Dedicated GPU: A physical GPU assigned to one tenant for the life of the instance, rather than time-sliced or partitioned between customers. It removes the noisy-neighbour effect on throughput and keeps memory contents private to the renter. Every PowerGPU instance is dedicated. More
FP16 and BF16: Half-precision formats used for most modern training and inference. FP16 has more mantissa precision, BF16 a wider exponent range that makes training more stable without loss scaling. Both need about 2 GB of memory per billion model parameters, before activations and cache. More
FP32: 32-bit floating point, the traditional precision for scientific computing and rendering. It is largely obsolete for neural network training, where reduced precisions run several times faster at equivalent quality, but it still governs simulation, CAD and many HPC codes. More
FP8: An 8-bit floating-point format supported natively from the Hopper generation onward. It roughly doubles throughput and halves memory against FP16 with a small quality cost, which is why H100, H200 and Blackwell cards serve large models far more cheaply than their FP16 numbers alone suggest. More
GDDR6X / GDDR7: The graphics memory used on consumer and workstation cards: cheaper and lower capacity than HBM, with lower bandwidth. The RTX 4090 carries 24 GB of GDDR6X, the RTX 5090 32 GB of GDDR7. It is the reason consumer cards win on price per token but lose on very large models. More
HBM3 / HBM3e: High Bandwidth Memory, stacked vertically beside the GPU die to reach bandwidths several times higher than graphics memory. The H100 SXM pairs 80 GB of HBM3 at about 3.35 TB/s; the H200 carries 141 GB of HBM3e at about 4.8 TB/s. Bandwidth, not capacity, usually sets inference speed. More
InfiniBand: A low-latency network fabric used to join GPU servers into a training cluster, with remote direct memory access between nodes. Distributed training is bound by gradient exchange, so fabric quality decides whether adding a second node speeds a run up or barely helps. More
INT8 and INT4: Integer precisions used to shrink models after training. INT8 needs about 1 GB per billion parameters and INT4 about 0.55 GB, so quantizing a 70B model to 4-bit brings it from roughly 150 GB down to around 44 GB — the difference between two flagship cards and one. More
Memory bandwidth: How fast the GPU can move data between its memory and its compute units, in GB/s or TB/s. Token generation in a language model reads the whole weight set for every token, so bandwidth — not raw FLOPS — is usually what limits single-stream inference speed. More
Multi-GPU instance: One machine with several GPUs attached to the same host, from 2x to 8x. It is the simplest way to run a model too large for one card, using tensor parallelism. On PowerGPU an 8x machine costs exactly eight times the per-GPU price, with no premium for the larger configuration. More
NVLink: NVIDIA's high-speed interconnect between GPUs inside one machine, an order of magnitude faster than PCIe. It matters when a model is split across cards, because tensor-parallel inference and multi-GPU training exchange activations constantly. Multi-GPU SXM machines on PowerGPU carry it. More
NVSwitch: A switch chip that lets every GPU in a server talk to every other at full NVLink speed, instead of only to its neighbours. It is what makes an 8-GPU HGX node behave like one large accelerator for training runs that span all eight cards. More
PCIe versus SXM: Two physical forms of the same datacenter GPU. PCIe cards slot into standard servers, draw less power and cost less; SXM modules mount directly on the board with higher power limits and full NVLink, giving more bandwidth and better multi-GPU scaling. The same chip performs differently in each. More
TDP: Thermal design power: the sustained heat a GPU is built to dissipate, in watts, and a good proxy for how much electricity it draws. It explains why an efficient card such as the L4 at 72 W can be cheaper to run continuously than a faster card that spends most of its time idle. More
Tensor core: A specialised unit inside modern NVIDIA GPUs that multiplies small matrices at reduced precision. Tensor cores do the heavy lifting for training and inference, and the generation of tensor core — Ampere, Ada, Hopper, Blackwell — matters more for AI throughput than the CUDA core count. More
TFLOPS: Trillions of floating-point operations per second, the headline measure of compute throughput. Figures are only comparable when the precision and the sparsity convention match: PowerGPU quotes dense, non-sparsity FP16 tensor throughput everywhere, which is roughly half the sparsity numbers some vendors publish. More
Tier-III datacenter: A facility designed for concurrent maintainability: redundant power and cooling paths so any component can be serviced without taking servers down. It is the practical dividing line between professional hosting and a machine in someone's spare room. Every PowerGPU GPU sits in a Tier-III or equivalent facility. More
VRAM: The memory soldered onto the GPU, holding model weights, activations and cache. It is the first constraint in any GPU decision: if the model does not fit, throughput is irrelevant. Rentable cards range from 6 GB up to 288 GB on the newest datacenter parts. More

Inference & serving 14

Cold start: The delay before an idle service can answer its first request, spent pulling the image, allocating a GPU and loading weights into VRAM. It is the main cost of scaling to zero, and it is why weights are usually kept on a fast local volume rather than fetched from object storage. More
Context window: The maximum number of tokens a model can attend to at once, prompt and generated output together. Long windows are a memory problem rather than a compute one: the KV cache scales with context, so serving 128k tokens can need several times the VRAM of the weights alone. More
Continuous batching: A serving technique that admits new requests into a running batch as earlier ones finish, instead of waiting for a whole batch to complete. It keeps the GPU saturated under uneven traffic and is the single biggest lever on cost per token, which is why vLLM and similar servers default to it. More
GGUF: A single-file format for quantized language models, used by llama.cpp and Ollama. It packages weights, tokenizer and metadata together at a chosen quantization level, which makes models easy to move between machines and is the usual way to run a model on a small rented GPU. More
KV cache: The stored attention keys and values for tokens already processed, kept so each new token does not re-read the whole conversation. It grows with context length and with the number of concurrent requests, and it is the memory cost most sizing estimates forget until a long-context request runs out of VRAM. More
LLM inference: Running a trained language model to produce output, as opposed to training it. It divides into a compute-heavy prefill pass over the prompt and a bandwidth-bound decode loop that emits one token at a time, which is why memory bandwidth usually decides the speed users feel. More
Ollama: A tool that downloads and runs quantized language models locally with one command, using GGUF files. It favours simplicity over maximum throughput, which makes it well suited to a single rented GPU used as a private assistant or a development backend. More
Quantization: Storing model weights at lower precision — 8-bit or 4-bit instead of 16 — to cut memory and increase speed. Modern 4-bit methods lose little measurable quality on most tasks and roughly quarter the memory, so quantization is often what decides whether a model fits on one affordable card. More
Scale to zero: Shutting all capacity down when there is no traffic, so an idle endpoint costs nothing but storage. It suits spiky or overnight-idle workloads; a service with steady traffic is cheaper on a reserved instance kept warm, because it never pays the cold-start penalty. More
Serverless GPU endpoint: A hosted HTTP endpoint that allocates GPUs per request and releases them afterwards, billing only for time actually spent computing. It removes capacity planning at the cost of cold starts, and works best for bursty inference rather than long-running training. More
Tensor parallelism: Splitting each layer of a model across several GPUs so they compute one forward pass together. It is how a model larger than one card is served, and it depends heavily on interconnect speed: NVLink-connected cards scale well, PCIe-only cards lose more to communication. More
Throughput versus latency: The central trade-off in serving. Throughput is total work per second across all users and drives cost; latency is how long one request waits and drives experience. Larger batches raise throughput and lengthen individual responses, so the right setting depends on whether you are serving a chat UI or a backlog. More
Tokens per second: The throughput measure for language models. Two different numbers hide behind it: per-stream speed, which one user perceives as responsiveness, and aggregate speed across all concurrent requests, which sets the cost per million tokens. Batching raises the second dramatically while barely changing the first. More
vLLM: An open-source inference server for language models, built around paged attention and continuous batching, exposing an OpenAI-compatible HTTP endpoint. It is the usual choice for self-hosted serving because it saturates a GPU far better than naive generation loops. More

Training & fine-tuning 9

Checkpoint: A saved snapshot of model weights and optimiser state mid-run, so training can resume after an interruption. Frequent checkpointing to a persistent volume is what makes cheap interruptible capacity usable for long training jobs: an eviction costs minutes rather than the whole run. More
Data parallelism: Replicating the whole model on each GPU and giving every replica a different slice of the batch, then averaging gradients. It is the simplest way to scale training when the model fits on one card, and it is bound by how fast gradients can be exchanged between GPUs. More
Epoch: One complete pass of the training process over the dataset. Fine-tunes usually run a small number of epochs, and epoch time multiplied by the hourly rate is the honest way to compare two GPUs: a card at twice the price that finishes in a third of the time is cheaper per run. More
Fine-tuning: Continuing training on a pre-trained model with your own data, to teach it a domain, a format or a behaviour. It costs orders of magnitude less than training from scratch and is the usual reason a team rents GPUs for days rather than months. More
Full fine-tune: Updating every weight in the model rather than a small adapter. It needs memory for weights, gradients and optimiser states — often more than ten times the weights alone in FP16 — so it usually means a multi-GPU node, and it is worth the cost only when LoRA demonstrably falls short. More
LoRA: Low-Rank Adaptation: freezing the original weights and training small added matrices instead. It cuts trainable parameters by orders of magnitude, so a fine-tune fits on one consumer card, and the result ships as a small adapter file that can be swapped or merged at load time. More
Mixed precision: Running most operations in 16-bit while keeping sensitive accumulations in 32-bit. It roughly halves memory and doubles throughput at no practical quality cost, and it is the default in every modern training stack; BF16 avoids the loss-scaling that FP16 requires. More
Pre-training: Training a model from random initialisation on a very large corpus. It is the most expensive thing done on GPUs, measured in node-months on interconnected clusters, and it is why HBM capacity, NVLink and InfiniBand matter far more here than the hourly price of a single card. More
QLoRA: LoRA applied on top of a 4-bit quantized base model, so the frozen weights occupy a quarter of the memory while the adapters train in higher precision. It is what makes fine-tuning a 70B model on a single 48 GB or 80 GB card practical. More

Platform & access 18

API key: A secret token that authenticates programmatic requests, sent as a bearer credential. PowerGPU keys are shown once and stored only as a hash, act with the full permissions of the account, and should be scoped per machine so a leaked key can be revoked without disturbing anything else. More
Bare metal: A whole physical server rented with no virtualisation layer between the tenant and the hardware. It gives maximum performance and full control at the cost of flexibility and minimum term, and it is normally reserved for large sustained workloads rather than experiments. More
CLI: A command-line client that wraps the API, so machines can be searched, launched and stopped from a shell or a CI pipeline without writing HTTP calls. It is the usual way to make GPU rental reproducible inside a build script. More
Cluster: Several GPU servers joined by a low-latency fabric and scheduled as one pool, for training runs too large for a single node. Beyond one machine the network becomes the bottleneck, so InfiniBand or equivalent, and the topology between nodes, matter as much as the GPUs themselves. More
Container instance: A GPU workload running in a container that shares the host kernel, with the NVIDIA runtime injected. Containers start in about thirty seconds because the image is pre-cached, which makes them the right unit for anything that does not need its own kernel or custom drivers. More
Docker / OCI image: A packaged filesystem and configuration that defines exactly what runs on the instance, identified by a registry reference. Any OCI-compliant image works on PowerGPU with the NVIDIA runtime injected, so an environment validated locally runs identically on rented hardware. More
JupyterLab: A browser-based notebook environment commonly pre-installed on GPU templates, exposed on a mapped port. It suits interactive experimentation and data inspection; long training runs are better launched from a shell so they survive a closed browser tab. More
llms.txt: A plain-text file at the root of a website that gives language models a curated index of its content and its key facts, in the way robots.txt gives crawlers a policy. PowerGPU publishes one, plus a full-content Markdown version, and serves any page as Markdown on request. More
MCP (Model Context Protocol): An open protocol that lets an AI assistant call external tools and read external data through a standard interface, instead of relying on what it memorised during training. PowerGPU runs a public read-only MCP server so an assistant can quote today's GPU prices, compare cards and cost a job. More
Region: The geographic location of the datacenter running your instance. It governs network latency to your users, which jurisdiction the hardware sits in, and sometimes availability of a specific GPU. PowerGPU operates in 32 regions across five continents at one uniform price. More
REST API: An HTTP interface for driving the service from code. On PowerGPU the pricing and inventory endpoints are public and need no key, so the whole price sheet can be read before creating an account; deploying, stopping and destroying instances uses the same API with a bearer key. More
SDK: A typed client library that maps the API onto native objects and handles authentication, retries and errors. It matters most when GPU provisioning is part of an application rather than an operator task, such as spinning up a worker per job. More
SLA: A service level agreement: the uptime a provider commits to and what it owes when it misses. A commitment without a remedy is marketing. PowerGPU publishes a 99.9% monthly target on on-demand and reserved instances, with service credits when it is not met. More
SSH access: An encrypted shell session on the rented machine, authenticated by key. It is the lowest common denominator for working on a GPU instance: install anything, run long jobs under a terminal multiplexer, and copy data in and out with rsync or rclone. More
Template: A pre-built container image with a stack already installed, launched in one click so the environment is ready in seconds rather than after an afternoon of driver debugging. PowerGPU publishes 37 templates from official upstream images — PyTorch, vLLM, ComfyUI, Ollama and others — and the template itself is free. More
Uptime credit: Compensation paid in account balance when a provider misses its uptime commitment, usually as a percentage of the affected spend and usually claimed rather than granted automatically. It is the only part of an SLA with teeth, so the credit table matters more than the headline percentage. More
Virtual machine: A full guest operating system with the GPU passed through, booting its own kernel with root access. It takes minutes rather than seconds to start, and it is what you need for custom kernel modules, desktop environments or software that refuses to run in a container. More
Volume: Network-attached storage that outlives the instance using it, so models, datasets and outputs survive when the machine is destroyed. Keeping weights on a volume turns instances into disposable compute and cuts cold starts, since the download happens once. PowerGPU charges $0.08 per GB per month. More

Payment & identity 8

KYC: Know Your Customer: the identity verification a provider performs before accepting a customer, typically an identity document, a card in a matching name or a company registration. It is a regulatory requirement for financial services and a business choice for compute providers. More
Monero (XMR): A cryptocurrency whose amounts and participants are hidden by default at the protocol level, rather than merely pseudonymous. It is the privacy-preserving option among the assets PowerGPU accepts, and confirmations typically arrive within minutes. More
No-KYC rental: Renting compute without submitting identity documents, which becomes possible when settlement is prepaid rather than credit-based. On PowerGPU an account is an email address and a password, funded by a crypto top-up; no documents, no card and no stored IP addresses. More
Prepaid credit: A balance funded before use and drawn down as resources run, instead of an invoice at the end of the month. It caps exposure — a runaway job stops when the balance does — and removes the credit check that card billing implies. PowerGPU balances are held in USD and refundable on request. More
Stablecoin: A cryptocurrency designed to hold a constant value against a fiat currency, usually the US dollar. It is the practical way to pay for compute in crypto, because the amount owed does not move between sending the transaction and its confirmation. More
Top-up: Adding funds to a prepaid balance. On PowerGPU a top-up generates a one-time deposit address for the chosen asset; credit appears after network confirmation, from seconds on TRON or Solana to roughly half an hour on Bitcoin. The minimum is $40 and credits never expire. More
TRC-20 and ERC-20: The token standards on the TRON and Ethereum networks respectively. The same stablecoin exists on both, but the addresses are not interchangeable: sending an ERC-20 transfer to a TRC-20 address loses the funds, which is why deposit pages always name the network. More
USDT: Tether, the most widely used dollar stablecoin, issued on several blockchains. PowerGPU accepts it on TRON (TRC-20) and Ethereum (ERC-20); the network chosen changes only the fee and the confirmation time, not the credited amount. More

The three questions behind most of these terms

Longer answers live in the guides; the numbers behind every price are on the methodology page.

What is a GPU-hour?

One GPU running for one hour. It is the unit every cloud GPU price is quoted in, and it is per GPU rather than per machine: an 8-GPU instance running for one hour consumes eight GPU-hours. On PowerGPU the per-GPU price is identical from 1× to 8×, and billing is per second, so a 12-minute job is billed as 0.2 GPU-hours.

What is the difference between on-demand and interruptible GPUs?

On-demand capacity is guaranteed for as long as you keep the instance: nobody can take it from you and the price is locked at deploy. Interruptible capacity is cheaper because it can be paused when the hardware is needed elsewhere. On PowerGPU interruptible is a flat 50% off on-demand with no bidding, the disk is kept and the instance is automatically re-queued — so it fits checkpointed training, batch inference and render queues.

How much VRAM does a model need?

For an LLM, roughly 2.4 GB per billion parameters in FP16 and about 0.62 GB per billion at 4-bit, plus room for the KV-cache, which grows with context length and concurrency. A 70B model therefore needs about 44 GB at 4-bit — one 80 GB card — or two 80 GB cards in FP16. Full tables are in the VRAM requirements guide.