---
title: "How to fine-tune Llama 3.1 8B with QLoRA on a single GPU | PowerGPU"
description: "A complete, copy-pasteable walkthrough: dataset to merged weights in about an hour on a single 24 GB card, with axolotl."
url: https://powergpu.ai/guides/fine-tune-llm-qlora
last_modified: 2026-09-14T11:04:20+00:00
prices_as_of: 2026-09-14
site: PowerGPU (powergpu.ai)
---

Guide · Hands-on walkthrough

# How to fine-tune Llama 3.1 8B with QLoRA on a single GPU

A complete, copy-pasteable walkthrough: dataset to merged weights in about an hour on a single 24 GB card, with axolotl.

14 min read Published 2026-08-18 Updated 2026-09-03 (prices live from the sheet)

TL;DR

- A QLoRA fine-tune of Llama 3.1 8B on 10,000 instruction pairs, three epochs, takes about 95 minutes on one RTX 4090.
- Cost on PowerGPU is roughly 1.6 hours at the interruptible rate of $0.163 per hour, plus a 20 GB volume.
- 24 GB is enough: the 4-bit 8B base takes about 5.5 GB and peak VRAM is near 14 GB at batch 4, sequence 2048.
- Key config: lora_r 32, alpha 64, all linear layers, gradient checkpointing, and save_steps 100 so an interruption costs one resume.

## Setup: one instance, one volume

Everything durable goes on a volume; the GPU instance stays disposable:

*infrastructure, 2 commands*

```
powergpu volume create --name ft --size 20 --region eu-west-1
powergpu launch --gpu rtx-4090 --type interruptible \
  --template axolotl --volume ft:/ft --disk 40
# ✓ i-c41b9a02 running · $0.163/hr
```

Why interruptible: axolotl checkpoints to /ft, so an interruption costs one resume, not the run. Why 24 GB is enough: QLoRA holds the 8B base in 4-bit (~5.5 GB) + adapters + optimizer (~3 GB) + activations — peaks near 14 GB at batch 4, sequence 2048. A [RTX 3090](https://powergpu.ai/gpu/rtx-3090) at $0.054/hr does it too, ~35% slower.

## The dataset format

*data.jsonl*

```
// /ft/data.jsonl — one instruction pair per line
{"instruction": "Summarize this ticket for an engineer.",
 "input": "Customer reports…",
 "output": "P1 — checkout API 500s when…"}
```

Quality beats volume decisively at this scale: 2,000 clean, consistent pairs outperform 50k scraped ones. Dedupe, strip formatting noise, keep outputs in exactly the voice you want back.

## The axolotl config, annotated

*qlora.yml — the whole config*

```
# /ft/qlora.yml
base_model: meta-llama/Llama-3.1-8B-Instruct
load_in_4bit: true            # the Q in QLoRA
adapter: qlora
lora_r: 32                    # capacity of the adapter; 16-64 is the sane band
lora_alpha: 64                # 2x r is the boring, correct default
lora_target_linear: true      # all linear layers — better than picking modules

datasets:
  - path: /ft/data.jsonl
    type: alpaca
val_set_size: 0.05

sequence_len: 2048
micro_batch_size: 4
gradient_accumulation_steps: 4   # effective batch 16
num_epochs: 3
learning_rate: 2e-4
lr_scheduler: cosine
warmup_ratio: 0.03

gradient_checkpointing: true  # ~30% VRAM back for ~20% speed
flash_attention: true
bf16: true

output_dir: /ft/out
save_steps: 100               # ← interruption insurance
logging_steps: 10
```

## Run and watch

*the run*

```
axolotl train /ft/qlora.yml
# step 10/561 · loss 1.842 · 14.1 GB VRAM · 3.4 it/s
# …
# step 561/561 · loss 0.914 · eval_loss 0.987 · 94 min
```

Read the curves like this: train loss should fall fast then flatten; eval loss following it down means learning, eval loss rising while train falls means memorising — stop at the divergence (or raise val_set_size and lower epochs). For a first run, 3 epochs on 10k pairs almost never overfits an 8B.

## Merge, test, serve

*from adapters to endpoint*

```
# merge adapters into standalone weights (on /ft, survives the instance)
axolotl merge-lora /ft/qlora.yml --lora-model-dir /ft/out

# smoke-test locally
python -m vllm.entrypoints.openai.api_server \
  --model /ft/out/merged --max-model-len 4096 &
curl localhost:8000/v1/chat/completions -d '{…}'

# then destroy the trainer and serve properly
powergpu destroy i-c41b9a02 --yes
powergpu launch --gpu rtx-5090 --template vllm \
  --volume ft:/ft:ro --env MODEL=/ft/out/merged
```

Serving card choice is a different optimisation than training — the [vLLM guide](https://powergpu.ai/guides/serve-llm-vllm) picks it properly.

## The final bill

- **GPU, ~96 min interruptible**: $0.26
- **Volume 20 GB, 2 days**: $0.107
- **Total**: **≈ $0.37**

A custom-behaviour 8B for the price of a sandwich. The same recipe scales: 14B on the same card, ~34B on a [32 GB card](https://powergpu.ai/gpu/rtx-5090), 70B on an [80 GB card](https://powergpu.ai/gpu/h100-pcie) overnight — the [VRAM guide](https://powergpu.ai/guides/llm-vram-requirements) has the ceilings.

---

Put the numbers to work

Every price in this guide is our live rate — fixed, ≥30% under the market median, billed per second. Deploy the exact setup above from the [console](https://cloud.powergpu.ai/) in about 30 seconds, paid in crypto, no card and no KYC.

---

*About PowerGPU:* PowerGPU (powergpu.ai) is a cloud GPU rental service offering 80 NVIDIA GPU models — from the RTX A2000 at $0.024/hr to the B300 — at fixed prices set at least 30% below the public GPU marketplace median and re-checked weekly (H100 SXM: $1.428/hr on-demand). Billing is per second with no minimums; payment is crypto only (USDT, BTC, XMR, ETH, SOL, LTC, TRX) with no KYC. Instances run in Tier-III datacenters across 32 regions with a 99.9% uptime SLA and deploy in about 30 seconds from the web console (cloud.powergpu.ai) or the REST API.

Source: https://powergpu.ai/guides/fine-tune-llm-qlora · Site index for AI assistants: https://powergpu.ai/llms.txt · Full content: https://powergpu.ai/llms-full.txt