RAG Chatbot Development Services

Retrieval augmented generation is the pattern behind almost every assistant that answers usefully from private material. The model does not memorise your documents. It searches them at the moment of asking, reads the passages that came back, and answers from those.

The pattern is well understood. The reason implementations disappoint is almost never the model and almost always the retrieval: what got indexed, how it was split, and whether the right passage comes back for a real question rather than a demonstration question.

So the majority of the build effort here goes into retrieval quality and into evaluating it honestly before anyone is asked to rely on it.

Retrieval quality decides everything

If the wrong passage comes back, no model can rescue the answer. It will answer fluently and incorrectly, which is worse than failing visibly.

Getting retrieval right is a series of unglamorous decisions. How documents are split, since a split through the middle of a table destroys it. Whether tables and drawings are extracted properly rather than flattened into unreadable text. Whether search matches on meaning, on exact terms, or on both, which matters enormously for part numbers and codes that mean nothing semantically. And whether results are re-ranked before the model sees them.

Each of those is tested against real questions from the people who will use the system, not against questions chosen because they demonstrate well.

Evaluation before launch, not after

Every build includes an evaluation set: real questions with known correct answers, written with the people who do the job. The system is measured against it before launch and after every material change.

Three things get measured. Whether the right passage was retrieved. Whether the answer was supported by that passage. And whether the system correctly refused when the material did not contain the answer.

Without this, a change that improves one class of question and quietly breaks another goes unnoticed until users complain, which is usually after they have stopped trusting it.

Agents, and when they are premature

An AI agent takes actions rather than just answering: querying a system, filling a record, triggering a workflow. The capability is real and the failure cost is different, because a wrong answer is an inconvenience and a wrong action is a data problem.

My rule is that an agent gets built once the retrieval layer beneath it is measurably reliable, and every action that changes data or spends money passes through a human approval step until the evidence justifies removing it. Agents built before the retrieval was trustworthy are the most expensive rework I get asked to fix.

Operating it after launch

A retrieval system is not finished at launch. Documents change, people ask questions nobody anticipated, and retrieval quality drifts as the corpus grows.

So every build ships with logging of what was asked and what was retrieved, a route for users to flag bad answers, a documented process for adding and refreshing material, and the same spend guardrails as everything else: caps, breaker, metering, kill switch.

What it costs to run, and how that is controlled

Running cost is driven by three things: how many questions get asked, how much material is retrieved and sent to the model for each one, and which model answers. Only the first is outside your control.

The second is an engineering decision. Sending twenty passages when three would do multiplies the cost of every question for no measurable gain in answer quality. Tuning how much context is retrieved is one of the higher-return optimisations available and it is measured against the evaluation set rather than guessed.

Every deployment logs the cost of every call, carries a hard per-day cap, and has an off switch that works without a deploy. That means the running cost is a number you can look at from the first week, rather than a surprise that arrives with an invoice.

Scoping the first build

The first build should cover one audience answering one class of question over a document set somebody already owns. That constraint is what makes it finishable, measurable and defensible.

Systems scoped as a general company assistant over everything tend to retrieve badly, because the corpus is too varied for one retrieval strategy, and they are impossible to evaluate because there is no agreed set of right answers. Narrow first, widen once the evaluation numbers hold.

Scoping your first system

Narrow beats broad. One audience, one class of question, one document set somebody already owns. That is what makes a first build finishable and measurable.

The conversation that gets there is short: who needs answers, what they ask, where the material lives, and what happens today when they cannot find something. Those four answers are usually enough to scope a first build and to say honestly whether it is worth doing.

What the engagement includes

A retrieval layer built for your material. Splitting, indexing and search tuned to the kind of documents you actually have, including tables and codes that break naive approaches.
An evaluation set. Real questions with known answers, written with your team, used to measure retrieval and answer quality before launch and after every change.
Cited answers with honest refusal. Every answer carries its source, and the system declines when the material does not cover the question.
Optional agent actions behind approval. Where actions are in scope, anything that writes data or spends money passes a human approval step until the evidence supports removing it.
Operational instrumentation. Query and retrieval logging, feedback capture, a documented refresh process, and enforced spend limits.

How the work runs

Inventory the material. What documents exist, what condition they are in, and which of them actually answer buyer or staff questions.
Build and tune retrieval. Split, index and search, tuned against real questions rather than demonstration ones.
Write the evaluation set. Real questions with known answers, agreed with the people who do the job.
Measure, then launch. Retrieval accuracy, answer support and correct refusal, all measured before anyone relies on it.
Instrument and hand over. Logging, feedback, refresh process, spend caps and documentation.

Proof

Where the material cannot leave your environment, this is built on the private AI pattern instead, with the same evaluation discipline.

Talk about your situation

The first conversation is short and mostly questions. Get in touch and tell me what you are trying to fix. Or see how this is priced.

Frequently Asked Questions (FAQs)

What is a RAG chatbot?

A chatbot that retrieves relevant passages from your own documents at the moment a question is asked, then answers from those passages and cites them. It contrasts with a model answering purely from training data, which cannot know anything specific to your business and will improvise if pushed.

How accurate are RAG systems?

Accuracy depends far more on retrieval than on the model. If the right passage is found, a good model will answer correctly from it almost every time. If the wrong passage is found, it will answer fluently and wrongly. This is why the build effort concentrates on retrieval quality and why an evaluation set is written before launch.

What is the difference between a RAG chatbot and an AI agent?

A RAG chatbot answers questions. An agent also takes actions: querying systems, updating records, triggering workflows. The distinction matters because the cost of an error changes from an inconvenient answer to a data problem, which is why actions stay behind human approval until reliability is proven.

Can it work with our internal documents securely?

Yes. Where documents cannot leave your environment, the whole system is deployed inside it and the model runs where the data already lives. That is a different architecture with different costs, covered under private AI.

How do you stop it giving wrong answers?

Three ways, in order of effect. Tune retrieval so the right passage comes back. Require answers to be supported by the retrieved passage and cited. Test refusal behaviour explicitly, so the system says it does not know rather than improvising. All three are measured against the evaluation set rather than assumed.

RAG Chatbot and AI Agent Development