LoRA Rank: Short Definition

LoRA rank is a single integer in your training config. It sets the width of the two small trainable matrices LoRA inserts next to each frozen weight matrix, and so bounds how much an adapter is able to change the model’s behaviour. Nothing else about the adapter is tuned independently of it.

This is the glossary entry. For the full treatment (the parameter arithmetic, how alpha couples to rank, and how to choose a value), see what is LoRA rank.

Which config key is it?

lora_r, in the training section of the configuration file, which distil labs defaults to 64 with use_lora enabled. In the reference implementation the same quantity is the r argument of the PEFT LoraConfig.

What moves when you change it?

Three things move at once, which is why rank is rarely the first knob to reach for.

Quantity	As rank rises	As rank falls
Trainable parameters	Grow linearly in r	Shrink linearly
Adapter file size	Larger	Smaller
Capacity to learn new behaviour	More	Less
Risk of memorising the training set	Higher	Lower

Frozen base weights appear nowhere in that table. Rank changes the size of the update, never the size of the model you serve.

When is rank the actual problem?

Rarely, on a first run. Raise it when a task demands genuinely unfamiliar behaviour and held-out scores stall below your bar; lower it when the model repeats training examples but fumbles paraphrases, which is overfitting rather than a capacity shortfall.

If you’re still choosing between adapters and moving every weight, LoRA vs full fine-tuning is the comparison to read first, and LoRA vs QLoRA covers what rank means once the frozen base is quantized.