Fine-tuning is the process of taking a pre-trained AI model and continuing to train it on a smaller, specialised dataset to adapt its behaviour for a specific task or domain. Rather than training from scratch, fine-tuning builds on the general capabilities of a foundation model.
Try Lucy OS1 →Pre-trained LLMs have broad general knowledge but may not behave optimally for specific applications — they might use the wrong tone, ignore domain-specific terminology, or fail to follow particular output formats. Fine-tuning on examples of the desired behaviour efficiently corrects these issues without requiring the massive compute of full training. Techniques include supervised fine-tuning (SFT), reinforcement learning from human feedback (RLHF), and the parameter-efficient LoRA method.
Lucy OS1 uses prompt engineering and system-level context rather than fine-tuning to shape Lucy's personality and conversational style. This approach keeps Lucy responsive to rapid iteration — personality updates don't require model retraining.
Try Lucy OS1 →Training on labelled input-output pairs that demonstrate the desired behaviour. The model learns to replicate the target response style.
A parameter-efficient fine-tuning method that modifies only a small fraction of model weights, making fine-tuning feasible with far less compute and data.
Reinforcement learning from human feedback. Human raters evaluate model outputs, and the model is trained to produce outputs raters prefer. Used by OpenAI, Anthropic, and others to align LLMs.
Prompting (system instructions) is faster and cheaper but less reliable for complex style or format requirements. Fine-tuning is more robust for consistent behaviour but requires data and compute.
When should you fine-tune vs prompt?
Prompt first. If you cannot achieve the desired behaviour through prompting and in-context examples, then consider fine-tuning. Fine-tuning is warranted for: consistent output format, domain-specific vocabulary, or tone that is hard to specify in a prompt.
How much data do you need to fine-tune an LLM?
For style and format, 100-1000 high-quality examples often suffice. For domain knowledge, you may need tens of thousands of examples. Quality matters far more than quantity.
Does fine-tuning make an AI smarter?
Fine-tuning shapes behaviour, not intelligence. A fine-tuned model follows instructions more reliably in its target domain but does not gain broader reasoning capability from fine-tuning.
Lucy OS1 puts these concepts to work in a real, streaming voice AI pipeline — Deepgram STT, GPT-4o-mini, and Cartesia TTS delivering natural voice conversation.
Start talking to Lucy →Welcome