How are AI models created? The product decisions behind the model

Date
November 16, 2026
Hot topics 🔥
AI & TechHow-to Guides
Contributor
Dmitry Ermakov
Summarize with AI:
Laptop screen with ChatGPT on it

AI models are created in five stages: collecting data, preparing and labelling it, training, evaluating, and deploying with monitoring. This article walks through each stage, shows where product decisions shape model behaviour, and compares training from scratch with fine-tuning and prompting. It is written for product and engineering teams deciding how much model they need to build.

Key takeaways

  • AI models are created in five stages, and data preparation and labelling consistently take more effort than the training itself.
  • Defining a correct answer and the acceptable balance of false positives and false negatives are product decisions to settle before evaluation begins.
  • Start by prompting a general model, move to fine-tuning when evaluation shows a specific gap, and reserve training from scratch for novel problems.
  • Deployed models degrade through input and concept drift, so set retraining triggers before launch and add every real production failure to the evaluation set.

There is a gap between the sentence “we trained a model” and what teams picture when they hear it. The picture usually involves algorithms. The reality involves arguments about what counts as a correct answer, which are product decisions dressed as technical ones.

Understanding how AI models are created matters because it tells you which decisions are yours.

How to create an AI model: the five stages

Creating an AI model means moving through five stages in sequence, where each stage constrains the ones after it.

1. Collect data. Assemble examples of the thing you want the model to learn. For a classifier, that means labelled examples of each class. For a language model, it means text at enormous scale. The relevant question is whether your data represents the situations the model will actually meet, because anything absent from the data is something the model has no basis for handling.

2. Prepare and label. Clean, deduplicate, standardise formats, and attach labels. This stage takes the majority of the effort on almost every project. Labelling is where human judgement enters the model permanently, and the consistency of your labellers puts a hard ceiling on the accuracy you can reach.

3. Train. Show the model the data repeatedly, adjusting its parameters to reduce the difference between its predictions and the labels. This is the stage people imagine when they think about how to make an AI, and on most commercial projects it is the shortest and least uncertain part.

4. Evaluate. Measure performance on data the model has never seen. Aggregate accuracy is a weak measure on its own. What matters is performance on the cases that carry consequences, and the shape of the errors rather than their count.

5. Deploy and monitor. Put the model into a system, watch its inputs and outputs, and set the conditions under which you retrain. A model in production is a component with a shelf life.

The product decisions hidden in each stage

Four decisions determine model behaviour more than any architectural choice, and all four belong to the product team.

What counts as correct. Before anything can be labelled, someone must define the right answer, including for the ambiguous cases. That definition becomes the model’s definition. Teams that delegate it to whoever is doing the labelling have delegated their product behaviour.

Which errors are acceptable. Every model trades false positives against false negatives. A document classifier that wrongly flags a clean file wastes a reviewer’s minute. One that misses a problem file costs something else entirely. The balance point is a business decision, and it needs setting before evaluation rather than after.

Who labels, and how consistently. Two people labelling the same ambiguous examples differently introduces noise that no amount of training removes. Measuring agreement between labellers before scaling up is cheap and rarely done.

What gets measured after launch. Deciding which signals indicate the model is drifting, and who looks at them, determines whether you find out from your monitoring or from a customer.

Train, fine-tune or prompt?

Three routes exist to a working AI capability, and the decision comes down to how specific your task is and how much control you need over the result.

Train from scratchFine-tune an existing modelPrompt a general model
Data neededVery large, task-specificHundreds to thousands of good examplesFew or none
CostHigh, compute and expertiseModerateLow, per request
Control over behaviourCompleteSubstantialLimited to instructions and context
Time to first resultMonthsWeeksDays
Best fitNovel problems, no suitable base modelSpecialised domains, consistent formatsGeneral tasks, early validation

Training a language model from scratch is out of scope for almost every organisation, and the interesting decision sits between the other two. Fine-tuning earns its place when a task has a consistent structure and a vocabulary that general models handle poorly. Prompting with good context handles a surprising amount of what teams assume requires training. Hugging Face’s documentation is the practical starting point for the middle column, and Google’s machine learning crash course covers the fundamentals underneath all three.

Our own default is to start at the right of that table and move left only when evaluation shows a specific gap. Starting on the left commits budget before anyone knows what correct looks like.

What changes after deployment

A deployed model degrades, because the world it was trained on keeps moving while the model does not.

Two things cause it. Input drift, where the data arriving differs from the training data, and concept drift, where the relationship you trained on genuinely changes. Both are detectable if you are watching input distributions and output confidence rather than only error reports.

Set retraining triggers in advance: a threshold on a monitored metric, a fixed interval, or a defined change in the upstream system. Under the EU AI Act, post-market monitoring is an obligation for higher-risk applications rather than a matter of good practice.

Keep the evaluation set alive as well. The hundred or so cases you used to decide the model was ready will stop representing the work within a year, and a stale evaluation set gives you confident results about a situation that no longer exists. We add new cases to it whenever a genuine failure reaches production, which turns each incident into a permanent regression test.

Key takeaways

  • Creating an AI model runs through five stages, and data preparation consistently takes more effort than training does.
  • The definition of a correct answer is a product decision that becomes the model’s behaviour permanently, so it should never be delegated to the labelling process.
  • The acceptable balance between false positives and false negatives is a business decision that must be set before evaluation begins.
  • Fine-tuning suits specialised domains with consistent structure, while prompting with good context handles more general tasks than teams expect.
  • Deployed models degrade through input and concept drift, so retraining triggers and monitoring should be defined before launch.

Working out how much model you actually need?

We help teams answer that question with evaluation evidence rather than assumption, then build the route the evidence points to. Talk to our AI and machine learning team about what you are trying to build.

__________________________________________________________

The Groundwork Advantage

Between 60 and 95 per cent of enterprise AI initiatives produce no measurable business value. Our free AI handbook covers what has to be true inside an organisation before AI produces anything: clear business objectives, data you can trust, governance built in from the start, and infrastructure that keeps your options open.

Read the handbook | Take the five-question self-assessment
__________________________________________________________

SaveSaved
Summarize with AI:

Dmitry Ermakov

Dmitry is our our Head of Engineering. He's been with WeAreBrain since the inception of the company, bringing solid experience in software development as well as project management.
Woman holding the Working machines book

Working Machines

An executive’s guide to AI and Intelligent Automation

Working Machines eBook