What Is RAG? Retrieval-Augmented Generation Explained

By Darus S Alfatah · 2025-09-05 · Edukasi · 8 menit

What Is RAG? Retrieval-Augmented Generation Explained

Author note: RAG solves two problems at once: fresh data and answers you can cite.

The Problem RAG Solves

Language models know general patterns, not your product catalogue or internal procedures. Asked about them, a model either admits ignorance or invents an answer. RAG fixes this by retrieving relevant text first and instructing the model to answer only from that text.

The benefit is verifiability: every answer can point to a source a human can check.

The Four Steps

  1. Split documents into chunks of a few hundred tokens.
  2. Convert each chunk to an embedding vector and store it.
  3. For a question, find the closest chunks.
  4. Send question plus chunks to the model, requiring citations.

Why Chunking Matters Most

If chunks cut a table in half or separate a heading from its content, retrieval returns fragments that cannot be used. Split by structure: headings, paragraphs, and rows. Keep a small overlap so sentences spanning boundaries are not lost.

Hybrid Search

Vector search handles meaning well but struggles with product codes, invoice numbers, and names. Combine it with keyword search and filter by metadata such as date or department. This single improvement often beats switching to a better embedding model.

Prompting for Faithfulness

Answer ONLY from the context below.
If the context does not contain the answer, say it is unavailable.
Cite the source number for each claim.

Without these rules, models blend retrieved facts with their own assumptions.

Memory Versus Retrieval

Retrieval answers “what does the system know”; memory answers “what already happened”. Store user preferences and past decisions separately, and let users view and delete them.

When RAG Is Wrong

Aggregation questions — totals, averages, trends — belong in SQL, not in retrieval. RAG is also a poor fit for data that changes every second without a synchronisation mechanism.

Frequently Asked Questions

Do I need a vector database? For a few hundred documents, a simple index works; scale up when needed.

How do I know it works? Track retrieval recall and answer faithfulness on a fixed question set.

How often do I update the index? Whenever documents change; store dates so recency can be ranked.

Can RAG replace fine-tuning? For changing knowledge, yes; for style and format, fine-tuning still helps.

What is the most common failure? Bad chunking, followed by missing metadata filters.

Conclusion

RAG is the cheapest way to make AI answer from your own data. Invest in clean documents and sensible chunking before spending on models.

A Minimal Implementation

Store chunks and their vectors in one table with the document id, source, and update date. On query, embed the question, fetch the top five chunks, filter by permissions, then build a prompt with numbered sources. Return the sources alongside the answer so users can verify.

Log which chunks were used for each answer. When a wrong answer appears, the log tells you immediately whether retrieval failed or the model ignored the context — two problems with very different fixes.

Review retrieval quality monthly by sampling queries and checking whether the right chunk appeared in the top results. Most RAG complaints trace back to this single metric.

Example: a document-grounded question: An employee asks about the leave policy. The system retrieves the relevant HR document section, then the model answers from that excerpt.

Frequently Asked Questions

What is the core topic of this article?

This article explains the fundamentals of AI APIs and gateways in plain language for developers, with practical examples.

Do I need a technical background?

No. The material builds up from the basics, so beginners can follow while experienced developers can jump to the relevant section.

How do I get started with an AI API after reading this?

Create a BankAI account, get an API key, then change your Base URL to BankAI’s OpenAI-compatible endpoint.

Sources & References

About the author

Darus S Alfatah — Founder & Writer

Darus S Alfatah is the founder of BankAI. He builds and operates the LLM API gateway that serves Indonesian developers, and writes about LLM integration, gateway architecture, and using AI practically and cost-effectively. Every guide is tested in a real environment — including code, configuration, and token calculations — before it is published, and updated when provider documentation or pricing changes.

Editorial policy

Related articles

🚀 Ready to Use the All-in-One API Key?

Sign up, top up with QRIS, and start building with ChatGPT, Claude, DeepSeek and every other model through one key.

🔑 SIGN UP NOW · Pricing