Custom machine learning solutions: when they beat an API

Most companies asking about custom machine learning solutions today do not need one, and the ones who do usually need it for a different reason than they think. General-purpose models have absorbed an enormous amount of work that used to require training something yourself — classification, extraction, summarisation, translation, sentiment, image description, document parsing. If your problem is in that list, the honest answer is to call an API and spend the saved budget elsewhere.

What is left after that subtraction is narrower, less fashionable, and considerably more valuable: predicting specific numbers about your own operations from data nobody else has. That is where a custom model still wins decisively, and it is worth being precise about why.

What custom machine learning solutions actually means now

Three routes get called "custom AI" in sales conversations, and they differ by an order of magnitude in cost and commitment.

1. Calling a general model. You send a prompt, you get a result. No training, no infrastructure, no data requirement beyond what you send at request time. You pay per request and nothing when idle.
2. Adapting a general model. Retrieval over your documents, structured prompting, or fine-tuning on examples of your desired output. You now have data preparation and an evaluation problem, but no model architecture to own.
3. Training a task-specific model on your data. A gradient-boosted tree predicting next month's demand per SKU, a vision model inspecting your production line, a churn model over your usage events. This is what the phrase should mean, and it is the only one of the three that requires a labelled dataset you own.

Route three is the subject of this article. Routes one and two are usually the right answer and are covered by the general case for integrating AI and machine learning into custom software.

The four conditions that justify training your own model

A custom model earns its cost when at least two of these are true. When only one is true, it is usually a hard sell. When none is, you are building a science project.

1. The signal lives in data nobody else has

A general model knows what the internet knows. It does not know that your Tuesday deliveries to one region run late whenever a particular supplier ships, or which combination of sensor readings preceded your last three equipment failures. If the predictive signal is in your transaction history, sensor logs, production records or usage telemetry, no amount of prompting reaches it. This is the single strongest reason to build, and it is the one most often overlooked in favour of weaker ones.

2. The output is a number with a measurable target

"Predict units sold next week", "score this claim for fraud risk", "estimate remaining useful life". These have a ground truth that arrives later, which means you can measure whether the model is right and improve it. Problems without a measurable target — "make our content better" — cannot be trained against and should not be framed as machine learning at all.

3. A constraint rules out the API

Latency budgets in single-digit milliseconds, inference on a device with no connectivity, data that cannot legally leave your infrastructure, or a regulator who wants to know why a specific decision was made. A small model you control satisfies these; a hosted general model often cannot. Auditability in particular is underrated — a gradient-boosted tree can tell you which features drove a decision in a way that satisfies a compliance review.

4. The per-unit economics stop working at your volume

Per-request pricing is excellent at low volume and unattractive at very high volume for a narrow, repetitive task. If you are scoring millions of events a day with the same simple decision, a small purpose-built model is dramatically cheaper per call. But be careful with this argument: it only holds if the volume is real today, not projected. We have watched teams build for a volume that never arrived and pay for idle infrastructure for a year.

There is also a fifth case that is not machine learning at all. If your rules are known, write the rules. A deterministic decision model is faster, cheaper, testable and explainable, and frequently beats a model for exactly the problems people reach for AI on.

Where the money actually goes

The most common budgeting mistake is treating the model as the expensive part. It is usually the cheapest part. Four lines dominate the real cost:

Labelled data. Not "we have lots of data" — labelled data, where someone has recorded the correct answer for each example. If nobody has been recording outcomes, the first phase of the project is instrumentation and waiting, which nobody enjoys estimating and everybody underestimates.

Inference infrastructure that bills whether or not you use it. This is the line that surprises teams moving from API calls. On Vertex AI, you are billed per node hour for each node in an endpoint, and you pay for each model deployed to an endpoint even if no prediction is made — a node hour includes time the machine spends waiting in an active state. SageMaker's real-time endpoints are similarly persistent managed infrastructure with autoscaling. An API call costs nothing at 3am. A deployed endpoint does.

Monitoring and drift. Models degrade because the world moves. Your customer mix changes, a supplier changes, a pricing change alters behaviour. Without monitoring, a model quietly becomes wrong while continuing to return confident numbers, which is worse than having no model. Budget for the monitoring, the retraining cadence, and the person who reads the dashboards.

Someone who owns it. A model without an owner is a liability with a schedule. This is the line most likely to be missing from a proposal, including ones we have reviewed from other vendors.

Training cost, by contrast, is usually modest and forgiving. Vertex bills training per compute hour used, and does not bill for a run that fails for reasons other than your own cancellation. Most business prediction problems train in hours on hardware that costs less than the meeting where you discussed it. If someone is quoting you a large number for training specifically, ask what it is for. The related trap on the generative side is the same shape — the real cost of self-hosting a model is rarely the number in the headline.

Problems that reliably justify a custom model

Categories where the four conditions tend to line up:

• Demand forecasting at the level of your own SKUs, stores or routes, where the seasonality and the substitution behaviour are specific to your catalogue.
• Predictive maintenance on equipment you instrument, where failure signatures are particular to your machines and duty cycles.
• Quality inspection on a production line, where the defects are yours and the latency budget is set by the conveyor.
• Churn and expansion scoring over your own product usage events, which no general model has ever seen.
• Pricing and yield decisions where the elasticity is specific to your market and the decision must be explainable.
• Routing and scheduling under constraints that are unique to your operation.

Notice what these have in common: a number, a ground truth that arrives, and data generated by the business itself. Notice also what is absent — anything involving general language understanding, which routes one and two now handle better than a team of three could in a year.

How to decide in two weeks rather than two quarters

You do not need a discovery phase to answer this. You need a small, honest experiment:

1. Write down the decision the model would change. Not the prediction — the decision, and who makes it today. If nobody would act differently given a perfect prediction, stop here. This kills more projects than any technical finding, and it should.
2. Establish the dumb baseline. Last week's number. The seasonal average. The current heuristic your operations team uses. Measure its error properly. This baseline is frequently better than people expect and is the bar the model must clear to be worth anything.
3. Label a sample by hand. A few hundred examples is enough to learn whether the labels are even consistent. If two experts disagree on the correct answer, a model will not resolve it.
4. Train the simplest model that could work on that sample and compare it to the baseline. A simple model on good features beats a sophisticated one on poor features almost every time.
5. Only then talk about infrastructure, monitoring and deployment.

If steps one to four take longer than two weeks, the blocker is almost always data access rather than modelling — which is itself the finding. Fixing the data foundations and architecture is the prerequisite, and it delivers value even if the model never ships.

Questions worth asking any vendor

Including us. A vendor who cannot answer these plainly is selling you a demo:

• What is the baseline you will measure against, and what error rate would make this not worth doing?
• Who labels the data, and how long does that take?
• What does inference cost per month at our volume, including idle time?
• How will we know when the model has drifted, and who retrains it?
• What happens to this system if you and I both leave the company?

The short version

Use a general model unless the signal is in data only you have and the output is a number you can check. When both are true, a custom model is often the highest-return software you can build, because the advantage compounds and competitors cannot buy it. When they are not, custom machine learning is an expensive way to reach a result an API would have given you on the first afternoon.

If you are weighing a specific problem and want a straight answer about which side of that line it falls on, that is the conversation our AI engineering team has most often — and roughly half the time our answer is that you do not need us to train anything.