What should I use to store embeddings for millions of chunks?

For millions of chunk embeddings, the best storage choice depends on whether you need fast similarity search, easy scaling, or just cheap persistence. In practice, you usually want a vector database or a database with vector indexing.

Good options

1. Vector databases

Best if you need semantic search / retrieval.

• Pinecone — managed, easy to scale, good for production
• Weaviate — open source + managed, strong features
• Qdrant — open source, fast, popular for self-hosted setups
• Milvus — built for large-scale vector search, strong for very large corpora

Use these if you want:

• efficient nearest-neighbor search
• metadata filtering
• millions to billions of vectors
• less operational work than rolling your own

2. PostgreSQL + pgvector

Best if you already use Postgres and your scale is “millions” rather than “tens of billions.”

Use this if you want:

• simple architecture
• transactional metadata + vectors together
• moderate scale
• easy development

Caveat: it can work well for millions of chunks, but for very high QPS or very large indexes, dedicated vector DBs usually perform better.

3. Search engines with vector support

• Elasticsearch / OpenSearch with vector fields

Good if you already use them for keyword search and want hybrid search (keyword + vector).

4. Object storage + separate vector index

If you mainly need to store embeddings cheaply, not search them directly:

• store embeddings in Parquet / Arrow / NumPy / S3 / GCS
• build an ANN index separately using FAISS or a vector DB

This is useful for offline pipelines, batch retrieval, or experiments.

Simple recommendation

• If you need production semantic search: use Qdrant or Pinecone
• If you want simplest stack and already have Postgres: use pgvector
• If you need very large-scale search: consider Milvus
• If you only need cheap storage: use S3/object storage + FAISS or another index

Rule of thumb

For millions of chunks, don’t store raw embeddings in a plain relational table unless:

• you only need occasional lookup
• you’re okay with slower search
• scale is modest

If you want, I can recommend a specific setup based on:

• expected number of embeddings
• vector dimension
• read/write rate
• cloud/on-prem preference
• whether you need metadata filtering or hybrid search