What is fine-tuning?

You may have seen an email app offer to write replies "in your style," or a company say its assistant was "trained for lawyers." It sounds as if someone built a new artificial intelligence (AI) from scratch. Often, nobody did. They took a general model that already existed and gave it extra lessons in one narrow thing. That extra training is called fine-tuning.

Fine-tuning begins with a model that has already been through pretraining, the long first stage where it learned language from a huge collection of writing. Instead of starting over, you keep training it on new examples. Its internal numbers shift a little toward the patterns in those examples. Anthropic's documentation puts it plainly: the model starts to mimic the patterns and characteristics of the fine-tuning data. Its general skills come along with it.¹²

The new examples are a tiny set next to what the model first read. International Business Machines (IBM) puts a typical fine-tuning set at a few hundred to several thousand examples. Each one shows the behavior you want: a request, and the kind of answer you'd like, written the way you'd like it. Because the set is small, fine-tuning takes far less time and computing than training a model from scratch. That puts it within reach of much smaller companies.¹

The chat assistant you use is itself a fine-tune. A freshly pretrained model only continues text. It was fine-tuned on examples of requests and helpful replies, so that it would answer you instead of rambling on. Anthropic says this about its Claude models directly: they aren't bare language models, because they've already been fine-tuned to be helpful assistants.²³

Fine-tuning is strong at some jobs and weaker at others.

• It's good at tone, like a brand's voice or a shorter way of replying.
• It's good at format, like answering in the same layout every time.
• It's good at a specialist's vocabulary, the words a field uses every day.
• It's less reliable for teaching new facts, and facts trained in this way go out of date.

For facts, OpenAI's own guide to accuracy points elsewhere: hand the model the right documents at the moment it answers.⁴

So "we fine-tuned it on our data" can sound more impressive than it is useful. A clearer prompt may fix the problem in minutes. When the model needs facts it doesn't have, looking them up as it answers, a method called retrieval, often does better. Fine-tuning earns its cost when you need a lasting habit across thousands of requests, and the prompt alone can't hold it.⁴

The next time a product says it was trained for your industry, ask what exactly it was trained to do better. Does it write in your field's style? Or does it claim to know your field's latest facts? Those are different promises.