Embeddings & Vector Databases for Smarter Search
By Darus S Alfatah · 2026-11-04 · Tutorial · 10 menit
Author note: We once built heavy vector infrastructure before having users. The simple version (one table + cosine) actually gave feedback faster.
Keyword Search Has Limits
Ordinary search matches exact words. Search "how to make coffee" and a document titled "guide to brewing espresso" will not appear because no word matches. Embeddings solve this: they turn text into numeric vectors that capture meaning, so texts with similar meaning sit close together even when the words differ.
What Are Embeddings?
An embedding represents text as a series of numbers (say 1,536 dimensions). Embedding models are trained so similar meanings produce nearby vectors. Closeness is measured with cosine similarity. With this, you can search "cheap electric car" and find documents about "affordable electric vehicles" without a shared word.
# pseudo
v1 = embed("how to brew coffee")
v2 = embed("espresso guide")
similarity(v1, v2) # high, despite different words
Vector Databases: Why You Need One
Comparing vectors one by one against millions is inefficient. A vector database stores vectors and provides fast nearest-neighbour search (ANN — approximate nearest neighbour). It also handles metadata (filter by category, date, access rights) that matters in production.
Building Simple Semantic Search
- Split documents into small chunks of 200–500 words.
- Embed each chunk; store vector + text + metadata.
- On a query, embed the query and find the nearest chunks.
- Send the top chunks as context to the language model to answer.
This pattern is called RAG (Retrieval-Augmented Generation). It lets a model answer from your data without retraining.
Common Mistakes
- Chunks too large — context blurs; split with logic.
- No metadata — hard to filter relevant results.
- Forgetting to re-embed — when documents change, old vectors must be updated.
- Relying on one search — combine keyword and semantic search (hybrid).
Hybrid Search
Combine keyword scores (BM25) with vector scores. This captures short queries (product names, codes) and nuanced questions alike. Average or merge the rankings, then take the top-N for context.
Real Use Cases in Indonesia
- Internal knowledge base — employees ask about company policy.
- Customer support — answer from product documentation.
- Catalog — search products with free-text descriptions.
Production Tips
- Store source document IDs so you can cite them.
- Limit the number of chunks sent to the model to control token cost.
- Measure retrieval quality (is the right chunk found), not just the answer.
- Use a consistent embedding model for index and query.
Choosing the Right Chunk Size
Too small and context is lost; too large and search blurs. Start with 200–400 words at 10–15% overlap. Test with real questions and see whether relevant chunks surface at the top.
Evaluating Retrieval Quality
Measure precision@k: of the top k chunks, how many are relevant. If low, fix the splitting, add metadata, or change the embedding model. Do not evaluate only the final answer — retrieval errors are often invisible in the output.
Cost and Scale
| Component | Main cost |
|---|---|
| Embedding | Once per document + per query |
| Vector storage | Based on number of vectors |
| Generation | Context tokens + answer |
Limit the number of chunks sent to the model; three to five is usually enough for many cases.
When to Re-embed
When documents change, or when you switch embedding models. Never mix vectors from two different models in one index — similarity becomes meaningless.
Alternatives Without a Vector Database
For small scale (thousands of documents), in-memory cosine search or a vector extension on a relational database is enough. A dedicated vector database is only needed as data grows or when you need complex filtering at low latency.
Best Practices for Small Teams
Start simple: one script to embed documents, one table to store vectors, and one search function. Add complexity only when data demands it. Many projects fail by building heavy infrastructure before real users exist; conversely, a light prototype quickly gives feedback on the true retrieval quality.
Conclusion
Embeddings and vector databases unlock smart search and affordable RAG. The keys: the right chunks, metadata, and retrieval evaluation. Combine with a language model through one BankAI — a single API key for embedding and generation.
Example: FAQ search: Suppose you have 400 help questions. Split each into small chunks, create embeddings, store them in a vector database. When a user asks, retrieve the 5 most similar chunks and include them as context. The answer then cites your documents instead of the model inventing one.
Frequently Asked Questions
Can beginners follow this tutorial?
Yes. The steps are explained in order with code examples, suitable for beginners and experienced developers alike.
Is BankAI’s endpoint compatible with the library I already use?
Yes, BankAI is 100% OpenAI-compatible, so libraries such as the OpenAI SDK and LangChain work without code changes beyond the Base URL and API key.
Where do I get an API key to follow this tutorial?
You get the API key (and Secret Token) after registering and completing payment in the BankAI dashboard.