best way to index text embeddings with metadata filters

The best way depends on scale, filter selectivity, and whether you need fast updates. In practice, the most robust pattern is:

Recommended approach

Use a vector index for embeddings plus a separate metadata index/filter layer.

Common architecture

• Embeddings: store in a vector database or ANN index
• Metadata: store as structured fields (SQL, document store, or the vector DB’s native metadata)
• Query flow:
  1. Apply metadata filters
  2. Search vectors only in the filtered subset
  3. Return top-k nearest neighbors

Best techniques by situation

1) Native vector DB with metadata filtering

Best if you want the simplest operational setup.

Examples:

• Pinecone
• Weaviate
• Qdrant
• Milvus
• Elasticsearch/OpenSearch vector search

Use this when:

• You want one system
• Metadata filters are moderate to complex
• You need decent performance without building custom infrastructure

Important:

• Ensure the engine supports pre-filtering or efficient filtered ANN search
• Some systems do better than others when filters are highly selective

2) Hybrid: SQL/NoSQL metadata filter + vector search

Best if metadata filtering is critical and you need strong control.

Pattern:

• Store metadata in Postgres / MySQL / MongoDB
• Store vectors in a vector index
• First fetch candidate IDs via metadata query
• Then do vector search on only those candidates, or search globally then rerank/filter

Use this when:

• Metadata is highly selective
• You already have a strong relational store
• You need auditing, joins, or complex predicates

3) Partitioning/sharding by metadata

Best when a few metadata fields are dominant, like:

• tenant_id
• language
• region
• document type

Pattern:

• Keep separate indexes per partition
• Search only relevant partitions

Use this when:

• Large scale
• Strong tenancy isolation
• Filters are usually on the same fields

Practical guidance

If filters are selective

Prefer pre-filtered search:

• Filter first
• Then vector similarity

This avoids wasting work on irrelevant vectors.

If filters are broad

Use vector search first, then apply metadata filter or rerank:

• Search top 100–1000 nearest vectors
• Filter them by metadata
• If needed, fetch more candidates

If you have many tenants or categories

Use partitioning or a routing key:

• tenant_id as partition
• language as partition
• category as partition

This can dramatically improve latency.

What to avoid

• Storing metadata only in application memory for large datasets
• Doing full vector search over everything and filtering afterward when filters are highly selective
• Over-partitioning into too many tiny indexes
• Using complex joins at query time if latency matters

Rule of thumb

• Small to medium scale: vector DB with native metadata filters
• Large scale, strict filter performance: partition by key metadata + vector search per partition
• Complex business queries: SQL metadata store + vector index

If you want, I can recommend the best setup for your exact case (e.g. Postgres + pgvector, Pinecone, Qdrant, Elasticsearch) based on:

• dataset size
• number of metadata fields
• filter patterns
• update frequency
• latency requirements