A couple of years ago, getting a language model to return clean, parseable JSON took some work. You wrote careful prompts, added retries and validated every response. That problem is mostly solved. The leading models support structured output natively: you pass a schema, and what comes back conforms to it.
So a fair question is: if you can give a frontier model your list of categories as an enum and get a valid answer every time, why train a classifier of your own?
Because structured output solves the format problem, not the judgement problem. And for a lot of everyday business classification, a small model that you own does the job better, more cheaply and more predictably than a general-purpose model you rent.
What I mean by a bespoke classifier
A classifier is a model that takes an input (a product description, an email, a support ticket, a line on an invoice) and assigns it to one label from a closed, bounded list. The list is yours. It might be twenty categories or two hundred, but it is finite and you decide what is on it.
A bespoke classifier is one trained specifically on that list, using examples from your own data. It is typically small, often a fine-tuned encoder model with a few hundred million parameters or fewer, rather than a frontier model with hundreds of billions. It does one thing.
A valid answer is not the same as the right one
When you hand a large model an enum of categories, the schema guarantees the answer will be one of your categories. It does not guarantee:
- That it’s the right one. The model has never seen how your business draws the line between “Fasteners” and “Fixings”. It is guessing from the label names and whatever you put in the prompt.
- That it’s consistent. Send the same borderline item twice, or slightly reworded, and you can get two different categories. For reporting and inventory, inconsistency is often worse than a known, stable error.
- That it will behave the same next quarter. The provider updates the model, deprecates a version, or changes pricing. Your categorisation can drift without anything in your own system changing.
A classifier trained on a few thousand of your own labelled examples learns where your boundaries are. Given the same input, it gives the same answer every time, because nothing changes underneath it until you decide to retrain it.
Two business examples
An inventory system
Say you run a distribution business with a product taxonomy of 140 categories. Supplier catalogues and purchase orders arrive with free-text line items like SS HEX BOLT M8X40 DIN933 BX/100, and every one needs a category before it can be stocked, priced and reported on.
That is a high-volume, narrow, repetitive task, which is the profile that suits a classifier. Each line is short, the categories are fixed, and you have years of already-categorised items to train on. A small model can classify thousands of lines a minute on ordinary hardware, at a negligible cost per item, and it will file the same bolt in the same place on Tuesday as it did on Monday.
A CRM
Now take inbound messages to a sales and support inbox. You want each one routed: new enquiry, existing client support, billing, partnership pitch, job application, spam. Each route triggers something different: a lead in the CRM, a ticket, a forward to accounts.
Here the cost of a wrong label is a misrouted customer, so you care about confidence as much as the label. A classifier gives you a probability for each category, not just an answer. You can set a threshold: anything above 90% routes automatically, anything below goes to a person, or to a larger model for a second opinion. That kind of graded control is harder to get from a model that only returns a single string.
Why small and yours matters
The two properties I’d point to are size and ownership.
Small means fast and cheap. A compact classifier runs on a CPU or a modest GPU, answers in milliseconds, and costs effectively nothing per call once deployed. At high volume, that difference against per-token API pricing adds up quickly.
Yours means the model is a file you hold. It runs inside your own infrastructure if you need it to, so the data being classified never leaves your network. No provider can deprecate it, re-price it, rate-limit it or quietly change its behaviour. For organisations with data residency requirements, that alone can settle the question.
This doesn’t make large models redundant. A pattern I like is to use both: a frontier model helps label the initial training set (with a person checking its work), and stays on hand as the fallback for the low-confidence cases. The classifier handles the bulk; the expensive model only sees the hard ones.
Keeping it current: retraining is cheap
The usual objection is that categories change. New product lines arrive, a team splits, a category gets retired. Doesn’t a custom model lock you in?
Not really, because retraining a small classifier is a routine job, not a project.
The cycle looks like this:
- Collect corrections. Every time a person overrides the classifier, or reviews a low-confidence case, that becomes a new labelled example. The review queue you already need doubles as your training data pipeline.
- Add or change categories. A new category needs examples, often a few dozen to start, which you can gather from recent items or label with help from a larger model.
- Retrain. Fine-tuning a small encoder on a few thousand short examples typically takes minutes on a single rented GPU, and the compute cost is usually in single-digit dollars per run.
- Evaluate before you promote. Keep a fixed test set that the model never trains on. The new version replaces the old one only if it scores at least as well, and you keep the previous version to roll back to.
Done this way, the classifier gets better the longer it runs, because your own team’s corrections keep feeding it. I’ve used the same approach on my own speech recognition fine-tunes: small, single-purpose models retrained cheaply as the data grows.
Deploying it
A trained classifier is a small artefact, and there are several sensible ways to run it:
- As an internal API. A small container inside your network that takes text and returns JSON such as
{"label": "billing", "confidence": 0.94}. Anything that can make an HTTP call can use it. - As a pipeline step. Dropped directly into an existing data pipeline or automation, so items are classified on the way through.
- As a tool for your agents. Exposed through an MCP server, so an AI agent can call your classifier the same way it calls any other tool, and get your business’s answer rather than its own guess.
- On serverless inference. For spiky or modest volumes, you pay only when it runs.
- Fully on-premises. For sensitive data, nothing leaves your infrastructure at all.
When not to bother
A bespoke classifier isn’t always the answer. If you classify a few dozen items a week, if your categories genuinely change every few days, or if each decision needs reasoning over long documents, a frontier model with a well-specified schema is simpler and probably good enough. Part of the job is telling those cases apart, and I’ll say so when a prompt and some validation will do.
But where the volume is real, the categories are yours and consistency matters, a small model trained on your own data is one of the most cost-effective pieces of AI you can own.
Training your own
If you’d like to explore training a classifier for your own workflows, whether I build it or plan it with your team, see my Custom Model Development service, which covers everything from the training data plan to deployment. Or if you’d rather talk it through first, book a call with me.