Q&A

What is fine-tuning in machine learning?

By AlgoInsight Team2026-06-18machine-learning

Overview

Large models are pretrained on vast general data. Fine-tuning refines this general capability for a specific use case - a medical diagnosis assistant, a code completion model, a customer-service bot with a particular tone. The key question is always: is fine-tuning necessary, or can prompting/RAG achieve the same result? Fine-tuning is more expensive but can embed knowledge and behaviour deeply.

graph TD
    A["Pretrained LLM<br/>(billions of params, general)"] --> B["Fine-tuning on task-specific data"]
    B --> C["Full fine-tuning: update all weights"]
    B --> D["LoRA / QLoRA: update rank-decomposed matrices only"]
    D --> E["Efficient: few trainable params<br/>Same expressiveness"]
    C & E --> F["Task-specialized model"]
    style D fill:#1e1b4b,color:#fff

When to Fine-Tune vs Prompt vs RAG

Fine-tune when: you need a consistent tone/persona, the task requires specialized reasoning not learnable by prompting, output format must be strictly structured, or you can't include all context in the prompt.

Full Fine-Tuning

Update all model weights on your dataset. Requires significant GPU memory (e.g., fine-tuning a 7B model needs ~60GB+ VRAM). Standard: prepare (instruction, response) pairs in a JSONL format, train with cross-entropy loss.

LoRA (Low-Rank Adaptation) - the standard PEFT method

Instead of updating all weight matrices W (d×k), LoRA adds a small low-rank update:

W' = W + ΔW = W + A × B
where A ∈ ℝ^(d×r) and B ∈ ℝ^(r×k), r << min(d, k)

Only A and B are trained (typically 0.1–1% of original parameters). Original weights are frozen. At inference, ΔW is merged back in - zero latency overhead.

from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3-8b")
lora_config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj","v_proj"])
peft_model = get_peft_model(model, lora_config)
peft_model.print_trainable_parameters()
# trainable params: 4,194,304 || all params: 8,030,261,248 || 0.05%

QLoRA - Quantized LoRA

Combines LoRA with 4-bit quantization of the base model, reducing VRAM from ~60GB to ~6-10GB for a 7B model - allowing fine-tuning on a single consumer GPU. The base model is stored in 4-bit; gradients flow through LoRA adapters in higher precision.

Data Preparation

Fine-tuning data format matters enormously:

{
  "instruction": "Summarize the following contract in plain English.",
  "input": "<full contract text>",
  "output": "This contract is a 2-year service agreement between..."
}

Data quality >> data quantity. Even a few hundred high-quality examples can meaningfully improve behaviour.

Overfitting in Fine-Tuning

A major risk: fine-tuning on too little data or too many epochs causes the model to memorize the training examples and lose its general capabilities (catastrophic forgetting). Mitigations: early stopping on validation loss, low learning rate (1e-4 to 1e-5), and keeping fine-tuning epochs low (1–3 for most tasks).

Evaluation

Always evaluate fine-tuned models on a held-out set with task-relevant metrics:

• Classification: accuracy, F1.
• Generation: ROUGE, BERTScore, human evaluation, LLM-as-judge.
• Compare against baseline (zero-shot prompting of the same base model).

Why This Matters in Interviews

Fine-tuning is a key LLM deployment topic. Cover: full fine-tuning vs LoRA (why LoRA - parameter efficiency, low-rank weight updates), QLoRA for single-GPU fine-tuning, data format and quality, catastrophic forgetting, and the fine-tuning vs prompting vs RAG decision. Being able to position fine-tuning as a last resort after exploring prompting/RAG (and knowing its costs) shows engineering maturity.

Common Pitfalls

• Fine-tuning when better prompting or RAG would suffice (unnecessary cost).
• Catastrophic forgetting: training too long, too hard, on too little data.
• Low-quality or inconsistent training data - garbage in, garbage out.
• Not evaluating on held-out data or comparing to the untuned baseline.
• Forgetting that LoRA adapters must be applied to the correct target modules (attention projections).

References

• Hu et al., "LoRA: Low-Rank Adaptation of LLMs" (2021): https://arxiv.org/abs/2106.09685
• Dettmers et al., "QLoRA" (2023): https://arxiv.org/abs/2305.14314
• HuggingFace PEFT: https://github.com/huggingface/peft

More answers