Ask a general-purpose AI model about your refund policy, your pricing tiers, or last quarter's onboarding checklist and it has nothing to draw on. It will either say so or, worse, produce something that sounds right and isn't. The model knows a great deal about the world in general and nothing about your business specifically.
Retrieval-augmented generation, usually shortened to RAG, is the most common way to fix that. Instead of hoping the model remembers the right facts, you look the facts up first and hand them to the model along with the question.
The idea in plain terms
Think of an open-book exam. A closed-book student answers from memory and fills gaps with plausible guesses. An open-book student finds the relevant page, reads it, and answers from what it says. RAG makes the AI the open-book student.
When a question comes in, the system searches your documents for the passages most likely to contain the answer, places those passages in front of the model, and asks it to answer using them. The model's job shifts from remembering to reading and summarizing, which it does far more reliably.
How it works under the hood
First, your documents are split into pieces of manageable size and indexed so they can be searched by meaning, not just by exact keywords. A question about 'canceling a subscription' can then find a passage titled 'ending your plan' even though the words don't match.
At question time, the system retrieves the best-matching pieces, assembles them with the question, and sends that to the model with instructions to answer from the supplied material. A well-built version also returns where the answer came from, so a person can check the source in one click.
Why not just train the model on your data?
Retraining a model on your documents is expensive, slow, and awkward to keep current. Your policies change, your prices change, and a retrained model keeps confidently repeating last year's version until you retrain it again.
With RAG, updating the knowledge means updating the documents. The next question draws on the new version immediately. It also makes access control practical: the system can be set to retrieve only what the person asking is allowed to see, which is far harder to enforce once information is baked into a model's weights.
Where RAG tends to fail
The model can only answer well if retrieval finds the right passage. Poorly organized, outdated, or contradictory documents produce poor answers no matter how capable the model is. Teams are often surprised that the biggest improvement comes from cleaning up their documentation, not from changing the AI.
Retrieval can also miss. A question phrased in an unusual way, or one that needs information scattered across several documents, may pull back the wrong pieces. And when nothing relevant exists, a careless system will still produce an answer. A good one is built to say 'I don't have that' and route the question to a person.
What to ask before building one
Where does the source material live, and who keeps it current? An assistant is only as trustworthy as the documents behind it, so someone has to own them. What should happen when the answer isn't in the documents? Decide that deliberately rather than discovering it in front of a customer.
Who is allowed to see what? If some documents are internal only, the system must respect that boundary from the start. And how will you check quality? Keep a set of real questions with known good answers and rerun them whenever the documents or the system change.
RAG turns a general-purpose model into one that answers from your own material and can show its sources. Its quality depends less on the model than on how well your documents are organized, how well retrieval finds the right passage, and how the system behaves when the answer isn't there.