What Is RAG? Retrieval-Augmented Generation Explained
By Darus S Alfatah · 2025-09-05 · Edukasi · 8 menit
Author note: RAG solves two problems at once: fresh data and answers you can cite.
The Problem RAG Solves
Language models know general patterns, not your product catalogue or internal procedures. Asked about them, a model either admits ignorance or invents an answer. RAG fixes this by retrieving relevant text first and instructing the model to answer only from that text.
The benefit is verifiability: every answer can point to a source a human can check.
The Four Steps
- Split documents into chunks of a few hundred tokens.
- Convert each chunk to an embedding vector and store it.
- For a question, find the closest chunks.
- Send question plus chunks to the model, requiring citations.
Why Chunking Matters Most
If chunks cut a table in half or separate a heading from its content, retrieval returns fragments that cannot be used. Split by structure: headings, paragraphs, and rows. Keep a small overlap so sentences spanning boundaries are not lost.
Hybrid Search
Vector search handles meaning well but struggles with product codes, invoice numbers, and names. Combine it with keyword search and filter by metadata such as date or department. This single improvement often beats switching to a better embedding model.
Prompting for Faithfulness
Answer ONLY from the context below.
If the context does not contain the answer, say it is unavailable.
Cite the source number for each claim.
Without these rules, models blend retrieved facts with their own assumptions.
Memory Versus Retrieval
Retrieval answers “what does the system know”; memory answers “what already happened”. Store user preferences and past decisions separately, and let users view and delete them.
When RAG Is Wrong
Aggregation questions — totals, averages, trends — belong in SQL, not in retrieval. RAG is also a poor fit for data that changes every second without a synchronisation mechanism.
Frequently Asked Questions
Do I need a vector database? For a few hundred documents, a simple index works; scale up when needed.
How do I know it works? Track retrieval recall and answer faithfulness on a fixed question set.
How often do I update the index? Whenever documents change; store dates so recency can be ranked.
Can RAG replace fine-tuning? For changing knowledge, yes; for style and format, fine-tuning still helps.
What is the most common failure? Bad chunking, followed by missing metadata filters.
Conclusion
RAG is the cheapest way to make AI answer from your own data. Invest in clean documents and sensible chunking before spending on models.
A Minimal Implementation
Store chunks and their vectors in one table with the document id, source, and update date. On query, embed the question, fetch the top five chunks, filter by permissions, then build a prompt with numbered sources. Return the sources alongside the answer so users can verify.
Log which chunks were used for each answer. When a wrong answer appears, the log tells you immediately whether retrieval failed or the model ignored the context — two problems with very different fixes.
Review retrieval quality monthly by sampling queries and checking whether the right chunk appeared in the top results. Most RAG complaints trace back to this single metric.
Example: a document-grounded question: An employee asks about the leave policy. The system retrieves the relevant HR document section, then the model answers from that excerpt.
Frequently Asked Questions
What is the core topic of this article?
This article explains the fundamentals of AI APIs and gateways in plain language for developers, with practical examples.
Do I need a technical background?
No. The material builds up from the basics, so beginners can follow while experienced developers can jump to the relevant section.
How do I get started with an AI API after reading this?
Create a BankAI account, get an API key, then change your Base URL to BankAI’s OpenAI-compatible endpoint.