LoRA vs QLoRA: Which Should You Use?

QLoRA is LoRA with one change: the frozen base model is loaded in 4-bit instead of 16-bit. You get a large cut in base-model VRAM and slightly slower training. On small student models, plain LoRA is usually the right default; QLoRA earns its place when memory is the binding constraint.

How do LoRA and QLoRA differ?

They differ in the precision of the weights that are not being trained. Everything else — the low-rank adapters, the rank, the optimizer — is shared.

	LoRA	QLoRA
Frozen base weights	16-bit	4-bit NormalFloat (NF4)
Trainable adapters	Low-rank BA pair	Same low-rank BA pair, higher precision
Base-model VRAM	Baseline	Roughly 3x lower
Training speed	Baseline	Slightly slower
distil labs default	use_lora: true	use_qlora: false
Extra dependency	None	bitsandbytes (Linux only)
Prerequisite	—	Only applies when use_lora is true

The VRAM, speed, dependency and default rows come from the distil labs configuration reference, which documents use_qlora as loading the base model in 4-bit NF4 during finetuning and then attaching LoRA adapters in higher precision.

What does QLoRA actually change?

QLoRA backpropagates gradients through a frozen, 4-bit quantized model into low-rank adapters. The QLoRA paper introduces three pieces to make that work without losing quality: 4-bit NormalFloat, a data type the authors describe as information-theoretically optimal for normally distributed weights; double quantization, which quantises the quantisation constants themselves; and paged optimizers to absorb memory spikes.

The headline result is a memory result, not an accuracy result. The authors report finetuning a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance, and their Guanaco family reached 99.3% of the performance level of ChatGPT on the Vicuna benchmark after 24 hours of finetuning on a single GPU.

Note what is unchanged. Quantisation here is a training-time memory technique applied to weights you are not updating. It is a different thing from quantising a finished model for cheaper inference, which is the subject of distillation vs quantization.

When is QLoRA the right choice?

Choose QLoRA when the base model does not fit in memory any other way. That is the problem it was designed for, and the original LoRA work already shows the pattern it extends: freezing the base and training rank-decomposition matrices cut GPU memory for GPT-3 175B by 3 times, and 4-bit quantisation of the frozen part compounds that.

Concretely, reach for it when:

• You are fine-tuning a model at the top of the size range you can host, and a training run has already failed on memory.
• You are on a single consumer GPU and the alternative is not training at all.
• You can accept a slightly longer run in exchange for the run completing.

When should you stay on plain LoRA?

Stay on LoRA when memory is not what is limiting you, which on task-specific small models is most of the time. Every entry in the supported students catalog sits well below the 65B scale QLoRA was built to rescue, so the 4-bit path mostly buys headroom you were not using while adding a dependency and slowing each step.

Three concrete reasons to leave use_qlora off:

• Nothing is failing. Quantising the base to save memory you had spare is a pure cost.
• You want the shortest run. QLoRA is documented as slightly slower per step; see how long does fine-tuning take for the other factors that move wall-clock.
• You are not on Linux. bitsandbytes is a Linux-only requirement for this path.

How do you switch between them on distil labs?

Both are tuning flags in your config file, and QLoRA is strictly a modifier of LoRA rather than an alternative to it:

tuning:
  use_lora: true
  use_qlora: true
  lora_r: 64

Setting use_qlora: true with use_lora: false does nothing — the flag only takes effect when LoRA is on. If you hit memory limits and want a non-quantisation option first, memory_optimized_training enables activation offloading and gradient checkpointing, which the config reference flags as significantly slower rather than slightly slower. The rank you pick is orthogonal to both; see what is LoRA rank.

For the underlying implementations of both methods, the Hugging Face PEFT library is the reference: it exposes LoRA and its quantised variants through the same adapter interface, which is why switching between them changes a flag rather than a pipeline.