Retrieval-augmented generation — RAG — is the difference between an assistant that answers from your actual documentation and one that answers from a vague memory of the internet. It's also the difference between a support feature that's useful and one that invents a refund policy you don't have.
It's worth understanding before you commission one, because the word gets used to mean anything from a genuine retrieval system to a chatbot with your PDF pasted into the prompt. Of all the AI features worth adding to a website, this is the one most often sold as something it isn't.
What it actually is
Without retrieval, a model answers from what it absorbed in training. It has never seen your pricing page, your policies, or last month's product change — so on anything specific to you it either declines or improvises.
With retrieval, the sequence changes. A question arrives. Your system searches your content for the passages that bear on it. Those passages are handed to the model along with the question, with an instruction amounting to: answer from this, cite it, and if it isn't here, say so.
The model stops being the source of truth and becomes the thing that reads and phrases. That's the whole idea, and it's why the citation is not a nice-to-touch — it's the mechanism that makes the answer checkable.
When it's worth building
- Documentation people genuinely struggle to search. Substantial, technical, or spread across formats — where the answer exists but nobody finds it.
- Support answering the same questions from the same sources. If your team keeps pasting the same three paragraphs, that's a retrieval system waiting to happen.
- Catalogues with real specification depth. "Which of these is rated for outdoor use" is a question a keyword search cannot answer and a salesperson answers twenty times a week.
- Internal knowledge that walks out the door. Onboarding, policies, the institutional memory currently held by one person.
When it's the wrong tool
When your content is small. Twelve pages don't need a retrieval layer. They need good navigation, and a visitor will find the page faster than they'll negotiate with a chat window.
When your content is a mess. This is the one that catches people. Retrieval faithfully surfaces whatever it finds — so if your documentation is contradictory, outdated in places, or has three versions of the same policy, you have just built a machine for confidently reciting the wrong one at scale. Fixing the content is the project, and it's usually most of it.
When the answer must be exact and lives in a database. Order status, stock level, account balance. Query the database and show the number. Do not route a fact through a language model. That's ordinary web application work, and it will be cheaper, faster and correct every time.
What a good implementation looks like
Citations are not optional
Every answer links to the page it came from. This does two things: it lets the user verify, and it makes the system debuggable — when an answer is wrong you can see immediately whether retrieval fetched the wrong passage or the model misread the right one. Those are completely different bugs with completely different fixes.
A real "I don't know"
If retrieval finds nothing relevant, the correct output is "I can't find that — here's how to reach a person," not an improvised answer. This is a design decision, and it is the line between a useful assistant and a liability.
How the content is split matters more than the model
Retrieval works on chunks of your content, and how you cut them decides what can be found. Split mid-procedure and you retrieve half an answer. This is unglamorous work and it determines the quality of the result far more than which model sits on top — I've run the same retrieval layer behind three different ones and the chunking mattered more than the swap.
Evaluate on real questions
Collect the questions people actually ask — from support tickets, from site search logs — and check the system against them. Without that you're tuning on vibes.
The honest summary
RAG is not hard to build badly and not trivial to build well. Most of the difficulty is in your content, not the model — which is good news, because fixing your documentation is worth doing regardless of whether an assistant ever reads it. Clean, well-structured content is also what gets you cited by AI assistants in the first place, so the work pays twice.
I'm Divyansh Sood. I build retrieval and AI features into websites and products. If you're wondering whether your content is in a shape that would support this — that's the first thing I'd look at, and I'll tell you honestly if the answer is "fix the docs first." Email me.