// Research / Custom Fine-tuning
SFT that actually ships.
Axolotl or LlamaFactory configs you can re-run, TRL SFTTrainer for low-level control, Unsloth for single-GPU long-context, FSDP for full fine-tunes at scale. LoRA / QLoRA / full FT chosen by data shape - eval-gated, reproducible.
SFT lifts a 2.5 Flash Lite from F1 0.84 → 0.91 - matching GPT-5.1 at ~1/20th the cost and 1/3rd the latency
// What we see
Most failed SFT runs failed before training started.
01
Wrong chat template, silently degraded model
Llama, Qwen, Mistral, Gemma each ship different chat templates. Train with the wrong one and the model loses structured output behavior - half the SFT runs we audit hit this bug.
02
LoRA targeting only attention, not MLPs
Old tutorial configs apply adapters only to q/k/v/o. MLPs and gate/up/down projections often matter more - per Thinking Machines, LoRA underperforms unless adapters hit all linear layers, especially on MoE models.
03
Hyperparameters by folklore
Copy-pasted learning rates, epochs by gut feel, no ablations, no eval gates. The model overfits at epoch 2 but trains to epoch 5 because the YAML said so.
// Case Study
Fine-tuned a small model to frontier quality - 50× cheaper at high volume
Customer's frontier-API entity-extraction pipeline worked but the per-token bill was eating margin at the volume they wanted to ship at. We split the task into hybrid retrieval + two fine-tuned Gemini 2.5 Flash Lite models. 98.3% F1 retention on the customer's existing eval suite, ~50× cheaper per 1000 requests, ~3× faster - without touching the prompt or the eval.
50×
cheaper per 1000 requests
3×
lower end-to-end latency
98.3%
F1 retention vs. frontier-API baseline
// What we do
Three layers, from framework choice to deploy gate.
Pick the training framework that fits the engagement. Get the templates and hyperparameters right. Wire eval gates so the run stops when it should and only the good checkpoints ship.
Framework + adapter strategy
Training framework and adapter strategy chosen per engagement, driven by data shape and memory budget.
- Axolotl, TRL, LlamaFactory, Unsloth, MS-Swift, VeRL - picked per engagement
- QLoRA-SFT 7B/13B on a single H100 in 18-36h, ~$50-100 compute
- Full SFT on 70B via FSDP / ZeRO-3, 8-32 GPUs
- All-linear-layer LoRA targeting - q/k/v/o plus gate/up/down
Template + hyperparameter discipline
Chat-template parity with the base model - ChatML, Llama, Qwen, Gemma, Phi-4 each get the right one.
- Chat-template normalization with tokenizer-level validation
- Sample packing + sequence length sized to the dataset's distribution
- LR / warmup / weight-decay schedules tuned per model family
- Gradient checkpointing + flash-attention-2 / flash-attention-3 by default
Eval-gated training + handoff
Eval suites mined from production traces and the failure modes you're trying to fix.
- Per-step eval on smoke suite, per-epoch eval on full suite
- Early stopping driven by held-out eval
- W&B / MLflow tracking with config + dataset hash for full reproducibility
- Checkpoints promoted only on eval pass
// Method fit
SFT is the right move when you have data and a base model.
skip it if
You don't have training data yet
SFT eats demonstrations. Without them you need a synthesis pipeline first - teacher distillation, self-instruct, evol-instruct, persona-based generation. We do both halves.
Synthetic Data PipelinesPrompt engineering already gets you there
If system prompt plus few-shot on a frontier model hits your bar, SFT is overkill unless inference cost or latency justifies it.
SFT alone won't reach the bar - you need preference optimization
If the gap is subjective quality - style, safety, last-mile reasoning - DPO/SimPO/KTO or GRPO is the next step after SFT.
Preference Optimization (DPO / KTO / GRPO)You're still pre-PMF and the workload keeps shifting
Ship on a frontier API, find fit, then SFT what stabilizes - usually a 7B/8B specialist dropping cost 10-20× at equal quality.
use it if
You have demonstration data (real, synthetic, or both) and an open-weight base model - Llama, Qwen, Mistral, Gemma, Phi, DeepSeek - that you want to specialize.
Your inference bill or latency is high enough that replacing a frontier API with a fine-tuned 7B/8B/13B pays back inside a month.
You want a reproducible training pipeline you can re-run as your task evolves.
// How we work
Get the data right. Get the templates right. Then train.
Almost every SFT failure we audit failed before the first gradient step. We spend the first week on data, templates, and eval - then training is the boring part.
01
Data audit + template normalization
Inspect format, length distribution, balance, and contamination against your evals. Validate the chat template tokenizes round-trip cleanly. Eval suite mined from production traces or curated golden cases.
02
Adapter strategy + framework choice + first run
QLoRA on Unsloth for single-H100 runs, Axolotl + LoRA on multi-GPU, full SFT on FSDP when adapters underfit. A 30-minute sweep on ~1k examples locks learning rate and warmup.
03
Eval-gated full run + handoff
Full training with per-step smoke eval, per-epoch full eval, early stopping on eval. W&B / MLflow run with hashed config and dataset fingerprint. Handoff: config, dataset, W&B project, retraining runbook.
// Expert insight
“SFT used to be the whole training game. Now it's the bootstrap before preference optimization. But the ceiling of what comes next is set by how well SFT was done - wrong template, wrong LoRA targets, wrong learning rate, and you're handicapping every downstream step. The teams that win at fine-tuning are the ones who sweat the boring details before they touch the optimizer.”
Karol Gawron
Head of R&D @ bards.ai
// Why bards.ai
80K+ monthly downloads of models trained exactly this way.
We've shipped 16+ open-weight models on Hugging Face with 80K+ monthly downloads - every one through the same SFT discipline we'd apply to your engagement.
16+ open-source models on Hugging Face
Polish-language SFTs, financial sentiment classifiers, fine-tuned chat models - 80K+ monthly downloads. We've shipped, served, and learned from each one.
Axolotl, TRL, Unsloth, LlamaFactory, MS-Swift
Framework picked per engagement. Configs and datasets stay with you, fully reproducible.
Full FT, LoRA, QLoRA fluency
Single-H100 QLoRA when iteration speed matters, all-linear-layer LoRA targeting per Thinking Machines' findings, full FT on FSDP / ZeRO-3 when adapters underfit.
Eval-gated, not loss-gated
Held-out evals run per-step and per-epoch. Early stopping driven by what your customers care about.
Template + tokenizer paranoia
Chat templates validated round-trip through the tokenizer before any GPU spins up.
Reproducible, license-aware
Every run hashed and tracked. Dataset provenance documented. License compatibility verified before training starts.
// FAQ
Common questions about SFT
QLoRA when memory is the bottleneck - single-H100 SFT of 7B/13B/34B models becomes feasible at the cost of slightly slower convergence. LoRA when you have multi-GPU but want fast iteration; per Thinking Machines, LoRA matches full FT on most SFT tasks if you target all linear layers (not just attention). Full FT on FSDP / ZeRO-3 when adapters underfit - common for large pretraining-like datasets, MoE models, or when you need to retrain the embedding layer.
Decode a tokenized training example back to text and compare it byte-by-byte with the base model's chat template documentation. The Llama-3 template, Qwen template, ChatML, Gemma template, and Phi-4 template are all subtly different - getting one wrong silently degrades structured output, refusal behavior, and tool-calling. Half the SFT failures we audit trace back to a template bug. We validate this in the data audit before any training spins up.
Less than people assume for narrow tasks, more than they assume for broad ones. For a focused capability (one domain, structured outputs, a specific tool-calling pattern), 5K-20K well-filtered examples is usually enough. For general instruction tuning, 50K-200K. Past 200K you're paying for marginal gains unless the task is genuinely broad. Quality of demonstrations dominates volume - well-filtered 10K beats noisy 100K every time.
Axolotl when you want config-driven, reproducible runs across multi-GPU and you'll iterate on the YAML over time. TRL's SFTTrainer when you want close-to-the-metal control or are integrating into a custom training loop. Unsloth when you're on a single H100 and need long-context or memory savings (90% VRAM reduction with their kernels). LlamaFactory for breadth of methods + LlamaBoard UI when you're triaging which approach works. MS-Swift for Chinese-language stack alignment. The configs stay with you regardless.
QLoRA-SFT on Qwen-7B with ~50K examples on 1×H100: 12-24h, ~$30-60 in compute. Axolotl LoRA-SFT on Llama-3-8B with ~100K examples on 8×H100: ~20h, ~$620. Full FT on Llama-70B with FSDP across 16 H100s: 2-4 days, ~$5-10K. The bigger expense in most engagements is the data preparation, not the GPUs.
OpenAI, Anthropic, and Google each expose SFT through their fine-tuning APIs - useful when the closed model is the right base for your task and the algorithm doesn't matter. The downside: you don't own the weights, you're locked into the provider's pricing, and you can't move to DPO/GRPO. We default to open-weights (Llama, Qwen, Mistral, Gemma, DeepSeek) for any engagement where downstream RL matters or where pricing flexibility is part of the brief.
A reproducible training pipeline: Axolotl/TRL config in your repo, dataset with provenance, W&B / MLflow project with all runs, model checkpoint, eval suite, and a runbook for retraining as the task evolves. Not a tarball thrown over the wall. The configs stay yours so your team can re-run the recipe six months from now without us.
// Related services
Adjacent problems we solve
- Learn more
Custom Fine-tuning
Synthetic Data Pipelines
Teacher distillation plus the part that matters: dedup, reward-model filtering, and contamination checks that discard the 80% of synthetic data that would hurt you.
- Learn more
Custom Fine-tuning
Preference Optimization (DPO / KTO / GRPO)
DPO, KTO, or GRPO chosen by your data shape - for the quality bar SFT alone can't reach.
- Learn more
Custom Fine-tuning
Reinforcement Learning for Agents
PPO with self-play leagues on JAX - thousands of parallel environments per GPU for games and multi-agent decision problems.
// Let's train it
Ship a fine-tune that actually does the job.
What base model, what data, what task? Those three answers are enough - we'll come back with an adapter strategy, framework choice, and a number within a business day.
Karol Gawron
Head of R&D @ bards.ai