Classifier Training
I train custom text classifiers on your own categories and your own data. A small model that sorts emails, tickets, documents or records into a fixed list of labels, gives the same answer every time, returns a confidence score, and runs in milliseconds on ordinary hardware.
Why train a custom text classifier
Structured output solves the format problem, not the judgement problem.
A frontier model given your categories as an enum will always return one of them. It won't always return the right one, it may answer differently when the same borderline item comes in twice, and its behaviour can change when the provider updates the model. It has never seen where your business draws the line between two neighbouring categories.
A classifier trained on a few thousand of your own labelled examples learns those boundaries. It gives the same answer to the same input until you decide to retrain it, it costs effectively nothing per call, and it is a file you hold. I set out the full argument in The case for bespoke classifiers, even now that LLMs do JSON.
Classification tasks I train models for
High-volume, narrow, repetitive decisions against a closed list of labels.
Ticket and email routing
Triage for support queues and shared inboxes: new enquiry, existing client, billing, partnership pitch, spam. Each label triggers a different route in your CRM or helpdesk.
Intent classification
Working out what a user is asking for, so a chatbot, voice agent or workflow sends the request to the right handler before a larger model is involved.
Document classification
Sorting invoices, contracts, forms and correspondence by type, so each one goes to the right extraction step, folder or person.
Product and record categorisation
Assigning free-text line items, catalogue entries or records to your taxonomy, consistently, at thousands of items a minute.
Sentiment and content moderation
Sentiment classification tuned to your customers' language, and moderation models trained on your own policy rather than a provider's generic one.
Multi-label classification
Where one item carries several tags at once, such as topic, urgency and product line, a multi-label classifier assigns all of them in a single pass.
Which kind of classifier
The model is chosen on how much labelled data you have, how many labels there are and where it has to run.
Fine-tuned encoder models
ModernBERT, DeBERTa and similar encoders fine-tuned on your labels. A few hundred million parameters or fewer, fast on CPU, and usually the most accurate option once you have a few thousand examples.
SetFit few-shot classifiers
When labelled data is thin, SetFit trains a usable classifier from a few dozen examples per label. A good way to get a first model into production and start collecting corrections.
Small LLM classifiers with JSON output
For inputs that need more reading, such as long emails or mixed documents, a small fine-tuned LLM that returns the label, and any extracted fields, as JSON.
Replacing per-call LLM API classification
A common starting point: a workflow already sends every item to a frontier model just to pick a label.
That works, and it is a good way to start. At volume it means paying per token for a decision a small model can make in milliseconds on a CPU, depending on another provider's rate limits and model changes, and sending every item to an outside service.
The existing setup is useful here. The labels the API has already produced, checked by a person, become the training data. The frontier model stays on hand as the fallback for the cases the classifier is unsure about, so the expensive model only sees the hard ones. Cost and latency drop, and classification stops drifting when the provider changes something.
Labelling, confidence thresholds and retraining
A classifier is only as good as its labels, and it should get better the longer it runs.
Labelling strategy
Clear label definitions, a labelling guide your team can follow, and a larger model to pre-label with a person checking its work. See datasets and training.
Active learning
Instead of labelling at random, the model picks the examples it is least sure about for a person to label next, so each hour of labelling improves it as much as possible.
Confidence thresholds and human review
Every prediction comes with a probability. Above a threshold it routes automatically; below it, it goes to a person or a larger model. The threshold is set on your data, not guessed.
Evaluation
A fixed test set the model never trains on, with per-label precision and recall, so you can see which categories it handles well and which need more examples. See evals and benchmarks.
Cheap retraining
Corrections from the review queue become new training data. Retraining a small encoder takes minutes on a single GPU, and a new version is promoted only if it scores at least as well as the old one.
Deployment
An internal API returning {"label": "billing", "confidence": 0.94}, a step in an existing pipeline, a tool your agents call over MCP, serverless inference or fully on-premises.
When a classifier isn't worth it
If you classify a few dozen items a week, if the categories change every few days, or if each decision needs reasoning over long documents, a frontier model with a well-specified schema is simpler and probably good enough. I'll say so when that's the case. Where the volume is real, the categories are yours and consistency matters, a classifier trained on your own data is one of the cheapest pieces of AI you can own.
Hiring me for Classifier Training
A small model that sorts your items into your labels, returns a confidence score and gives the same answer every time, usually replacing a per-call LLM prompt.
Who it's for
- Teams routing tickets, emails or documents by hand, or through a per-call LLM prompt
- Classification tasks with a fixed label list and steady volume
- Workflows that need a confidence score to decide what goes to a person
- Data that shouldn't be sent to a third-party API
What you get
- A trained classifier (encoder, SetFit or small LLM) you own
- Label definitions and a labelling guide your team can follow
- A fixed test set with per-label precision and recall
- Confidence thresholds set on your data, with a human-review route below them
- An API or pipeline step returning label and confidence, plus a retraining loop
What I need from you
- Your label list, or the decisions people currently make
- A sample of real items, labelled or not
- Roughly how many items a day, and where predictions need to go
- Someone who can label or check examples for a few hours
How it runs
Typically 3–5 weeks, faster if labelled data already exists.
- Week 1
Labels and examples
Agree the label list and definitions; collect existing labelled items.
- Weeks 1–3
Labelling
Pre-label with a larger model, with your team checking the hard cases.
- Weeks 2–4
Train and evaluate
Train the right kind of classifier and report per-label results on held-out data.
- Weeks 3–5
Thresholds and deploy
Set confidence thresholds, wire in human review and deploy.
Training work has no public price list. Each model is scoped after a short call and quoted as a fixed price, paid against milestones.