Are there any similarity search engines that handle incremental indexing for recommendation systems?

Yes. Several vector/similarity search engines support incremental indexing or near-real-time updates, which makes them suitable for recommendation systems where items, embeddings, and user vectors change continuously.

Common options

• FAISS

  • Great for high-performance similarity search.
  • Supports adding vectors incrementally in many index types.
  • Some indexes are more update-friendly than others; fully dynamic deletion/update is limited depending on the index structure.

• Milvus

  • Built for large-scale vector search.
  • Supports inserting new vectors continuously and managing collections over time.
  • Good fit if you need distributed storage and scaling.

• Qdrant

  • Designed for fast vector search with payload filtering.
  • Supports upserts, deletes, and incremental updates well.
  • Often used in recommendation and retrieval applications.

• Weaviate

  • Supports incremental ingestion of vectors and hybrid search.
  • Good if you want schema + metadata + vector search together.

• Pinecone

  • Managed vector database with real-time upserts.
  • Easier operationally, and incremental indexing is a core feature.

• Elasticsearch / OpenSearch

  • Can do vector similarity search with ANN support.
  • Useful if you already use them for text search and want hybrid search.
  • Incremental indexing is naturally supported, though pure vector performance may lag specialized engines.

What to look for in recommendation systems

For recommender use cases, important features are:

• Incremental upserts for new items and updated embeddings
• Deletes for removed items
• Metadata filtering for category/user constraints
• Low-latency search
• Support for hybrid retrieval if combining text + vector similarity
• Scalability if you have many items or frequent updates

Practical note

If your embeddings change often, choose a system that:

• supports upsert/update by ID
• can handle background reindexing
• has clear consistency guarantees for reads after writes

If you want, I can also give you:

1. a comparison table of these engines for recommendation systems, or
2. a recommendation based on your scale and update frequency.