Prompt
I'm building a product search app with vector embeddings and metadata filters
Latest observation
Great use case. A product search app with vector embeddings + metadata filters usually works best as a hybrid retrieval system:
- Embeddings handle semantic search: “running shoes for winter” can match products described as “trail sneakers with waterproof lining”
- Metadata filters handle hard constraints: brand, price, color, category, size, availability, rating, etc.
Typical architecture
1) Ingest products
For each product, store:
- Text fields for embedding:
- title
- description
- bullet points / features
- maybe reviews or tags
- Metadata fields for filtering:
- product_id
- brand
- category
- price
- color
- size
- gender / age group
- in_stock
- rating
- shipping region
- created_at / updated_at
2) Generate embeddings
Create an embedding from a concatenation like:
“Nike Air Zoom Pegasus 40. Lightweight running shoe. Breathable mesh upper. Cushioned midsole. Best for road running.”
Often you’ll get better results if you embed:
- a curated product summary
- or multiple chunks/representations if the product page is long
3) Store in a vector database
Use a vector DB or search engine that supports:
- nearest-neighbor search
- metadata filtering
- ideally hybrid keyword + vector search
Examples:
- Pinecone
- Weaviate
- Milvus
- Qdrant
- Elasticsearch/OpenSearch with vectors
- pgvector in Postgres for smaller scale
4) Query flow
A user query like:
“waterproof trail shoes under $120 in black”
Can be processed as:
- vector query: “waterproof trail shoes”
- filters:
- price < 120
- color = black
Then return top-k nearest products satisfying the filters.
Recommended retrieval strategy
Option A: Filter first, then vector search
Best when filters are strict and reduce the search space a lot.
Example:
- category = shoes
- in_stock = true
- price < 120
Then run vector similarity over that subset.
Option B: Vector first, then filter
Works if your vector DB supports post-filtering, but you may lose relevant results if too many top matches fail filters.
Option C: Hybrid search
Best overall when you need both semantic relevance and keyword precision:
- combine BM25/text search + vector similarity
- apply metadata filters at query time
Data model example
{
"id": "prod_123",
"title": "Waterproof Trail Running Shoe",
"description": "Lightweight trail shoe with rugged grip and waterproof membrane.",
"embedding": [0.012, -0.98, ...],
"metadata": {
"brand": "Acme",
"category": "shoes",
"price": 109.99,
"color": "black",
"size": [8, 8.5, 9, 10, 11],
"in_stock": true,
"rating": 4.6
}
}
Query example
User:
“black waterproof running shoes under $120”
Parsed query:
- semantic text: “waterproof running shoes”
- filters:
- color = black
- price < 120
Practical tips
1) Normalize metadata
Make sure metadata is clean and consistent:
- “Black” vs “black”
- numeric price format
- standardized categories
2) Use facets
If users browse, metadata filters should support facets:
- category counts
- brand counts
- price ranges
- sizes available
3) Store structured filterable fields separately
Don’t rely on embeddings for things like:
- price
- inventory
- size availability
- region
Those should always be hard filters.
4) Consider query understanding
You can improve search by extracting:
- intent
- attributes
- constraints
Example:
“cheap red Nike sneakers for women”
Could map to:
- semantic: Nike sneakers for women
- filters:
- brand = Nike
- color = red
- price <= some threshold for “cheap”
- gender = women
5) Re-ranking
A good pattern is:
- retrieve top 50–200 candidates
- apply filters
- re-rank with a cross-encoder or LLM-based scorer
This often improves relevance a lot.
If you’re choosing a stack
Simple and fast to build
- Postgres + pgvector
- structured columns for filters
- full-text search if needed
More scalable search stack
- OpenSearch/Elasticsearch for hybrid search
- vector + BM25 + filters
Dedicated vector DB
- Qdrant or Pinecone if you want clean vector-first architecture with filtering
Example user experience
Search bar:
- “women’s walking shoes” Filters:
- brand
- price slider
- size
- color
- rating
- in stock only
Backend:
- convert query to embedding
- apply metadata filters
- return ranked results
- optionally show “why matched” explanations
One key design choice
Decide whether you want:
- search by natural language only
- or search + faceted filtering
- or catalog exploration + recommendation
That determines how much weight to put on embeddings vs filters.
If you want, I can help you with any of these next:
- a schema design
- a query pipeline
- a sample implementation in Python/TypeScript
- choosing between Pinecone, Qdrant, pgvector, or Elasticsearch