Local AI and the right-sized model for each task

When people talk about AI, they usually mean the biggest models: the ones you reach through a website or an API (a connection to an outside service) and pay for by usage. They’re impressive. But for a good share of a company’s day-to-day work, using them is like hiring a professor to sort the post.

Most real tasks are narrower than they look. And that opens up an option many companies never consider: smaller models, installed on their own machines.

Not everything needs a huge model

Before choosing a model, it’s worth asking what actually needs doing. Classify an email as an order, a complaint or an invoice. Pull five fields out of a document. Decide whether a line on a bank statement matches a payment. Summarise a complaint in two sentences.

For tasks like these, you often don’t need an LLM at all (a large language model, the kind of AI that reads and writes text). A well-written rule or a simple statistical model can be more reliable, faster and easier to explain. And when you do need to understand language, a small model tuned to the task is often enough.

There are now open models, meaning you can download and install them without depending on a vendor, in a wide range of sizes. They can also be quantised: their numbers are stored at lower precision so they take up less memory and run on more modest hardware, in exchange for a loss of quality that is barely noticeable on many tasks. You always have to measure that on your own cases before deciding.

Agents: each step with the smallest model that handles it

An agent is a system that doesn’t just answer: it chains steps together to get a piece of work done. It reads a request, queries your systems, drafts a reply or a document and, where needed, leaves it ready for a person to approve.

The key is not to use the same model for everything. Take an order that arrives by email:

1. Classify the email: is it an order? A very small model, or even plain rules.
2. Extract the customer, product codes and quantities. A small model specialised in extraction.
3. Check against the ERP that the customer exists, the codes are valid and there’s stock. No AI here: these are queries to your own system, deterministic and exact.
4. Draft the confirmation, or the question back to the customer. A somewhat larger model, if the text calls for it.
5. Approve. Anything that doesn’t add up, or goes over certain amounts, is reviewed by a person.

Each step uses the smallest model that does it well. The result is faster, cheaper to run and easier to monitor: if something goes wrong, you know which step it was.

Privacy and predictable cost

Running models on your own servers, or in a European cloud under your control, has two very concrete advantages.

Your data stays in. Invoices, contracts and customer records are processed in-house. They don’t travel to an outside service, and you don’t depend on what that service does with them. When it comes to data protection, starting from there makes things much simpler.

The cost doesn’t depend on usage. With a service that charges per token (the unit used to measure text going in and out of a model, roughly a fragment of a word), every query costs money. If volume grows, or a process runs more often than planned, the bill grows with it. With your own model, the cost is the hardware and its upkeep: you know it in advance, and it doesn’t change because your team uses it more. No usage fees: the model is yours.

There’s a third, less visible advantage: control. An installed model doesn’t change overnight because the vendor updated it. You decide when to move to a new version, and every change is tested on your own cases first.

When a big model is worth it

This isn’t an argument against large models. There are situations where they’re the best choice:

• Open-ended, non-repetitive tasks, such as reasoning through a new problem or writing a long, nuanced text.
• Low volumes, where paying per use is cheaper than running your own hardware.
• Quick experiments, to check whether an idea works before investing in an installed solution.
• Non-sensitive data, where privacy isn’t a constraint.

Even then, you can combine the two: small models in-house for the daily work, and a large model only for the few cases that need it, without sending it personal data.

The decision isn’t “big or small”. It’s understanding the task properly and choosing what’s right for each step.

If you’d like to know which parts of your work could run on a model of your own, and which aren’t worth touching, write to us at info@liberionlabs.com. The first step is understanding how you work.