Prompt
Do I need a vector database for a small RAG app?
Latest observation
Oct 9, 2026
OpenAI APIWeb search: off
Not necessarily.
For a small RAG app, you only need a vector database if your retrieval needs benefit from semantic search at scale or you want the convenience of a managed/vector-native system. Many small apps work fine with simpler options.
When you don’t need a vector database
You can often skip it if:
- Your corpus is small (e.g. a few hundred to a few thousand chunks)
- You can tolerate basic similarity search without advanced filtering/ranking
- You’re okay storing embeddings in:
- a regular SQL database with vector support (e.g. Postgres + pgvector)
- an in-memory index (for prototypes)
- a local library/index (e.g. FAISS, Chroma, LanceDB)
- You want to minimize infrastructure and cost
When a vector database helps
A vector DB becomes more useful if:
- Your dataset is growing quickly
- You need metadata filtering alongside vector search
- You want persistence, scaling, or concurrency
- You need hybrid search (keyword + vector)
- You’re building something production-facing and want easier ops
Good “small app” defaults
For most small RAG apps, these are often enough:
- Postgres + pgvector if you already use Postgres
- FAISS if you want a simple local ANN index
- Chroma/LanceDB if you want a lightweight developer-friendly local store
Rule of thumb
- Prototype / small internal tool: no, you probably don’t need a dedicated vector database
- Production / growing corpus / filtering needs: yes, it may be worth it
If you want, I can recommend the best setup based on:
- your data size
- whether you need metadata filters
- your deployment environment
- your preferred stack