How do I set up a deal search app for tracking price drops and comparing products across multiple merchants?

To set up a deal search app for tracking price drops and comparing products across multiple merchants, think of it as 5 parts: product data collection, normalization, matching, price tracking, and user alerts/search.

1) Define the core features

Start with a clear MVP:

• Search products across merchants
• Compare prices for the same or similar product
• Track price history
• Alert users when a product drops below a target price
• Filter/sort by brand, category, merchant, shipping, rating, availability

Optional later:

• Coupons/promo codes
• Geo-based availability
• Wishlists
• Personalized recommendations

---

2) Get product data from merchants

You need a reliable source of merchant catalog data.

Best options

1. Merchant APIs

   • Most accurate and compliant
   • Often include price, availability, images, shipping, and identifiers like UPC/EAN/SKU
   • Example sources: retailer APIs, affiliate APIs, product feeds

2. Affiliate product feeds

   • Common for e-commerce comparison apps
   • Easier to ingest at scale
   • Usually updated periodically

3. Web scraping only if allowed

   • Use only where permitted by terms of service and robots rules
   • More fragile and maintenance-heavy

What to collect

For each offer/product:

• Merchant name
• Product title
• Brand
• Model number / SKU / UPC / EAN
• Category
• Price
• Currency
• Shipping cost
• Stock status
• Product URL
• Image URL
• Ratings/reviews if available
• Timestamp of crawl/update

---

3) Build a normalized product catalog

Different merchants name the same product differently, so normalize data into a unified schema.

Example schema

• product_id — canonical product
• offer_id — merchant-specific listing
• merchant_id
• title
• brand
• model
• gtin/upc/ean
• category
• attributes — color, size, memory, etc.
• price
• shipping
• availability
• url
• last_seen

Why this matters

You’ll often have:

• “Apple iPhone 15 Pro Max 256GB”
• “iPhone 15 Pro Max - 256 GB - Natural Titanium”

These should map to the same canonical product if identifiers match or if matching confidence is high.

---

4) Match identical or similar products

This is the heart of product comparison.

Matching strategy

Use a layered approach:

A. Exact identifier matching

Match by:

• UPC
• EAN
• GTIN
• MPN/model number

This is the most reliable.

B. Attribute-based matching

If identifiers are missing:

• Brand + model + key specs
• Category-specific attributes
• Example: laptop CPU, RAM, storage, screen size

C. Fuzzy text matching

Use:

• Token similarity
• TF-IDF / cosine similarity
• Embeddings for semantic matching
• Rules to remove marketing words like “best”, “new”, “sale”

D. Human review or confidence thresholds

For uncertain matches:

• Flag for manual review
• Or only show “similar products” instead of “same product”

Important

Avoid false matches. A bad match hurts trust more than showing fewer results.

---

5) Track prices over time

Store each price change as a time series.

Price history table

• offer_id
• price
• shipping
• total_price
• timestamp
• merchant_id

Use cases

• Show “lowest price in 30 days”
• Detect price drops
• Trend charts
• Alert on threshold breach

Price-drop logic

Trigger if:

• Current price < user’s target price
• Current price dropped by X% from recent average
• Current price is lowest in N days

---

6) Build the search experience

A good deal app needs fast and relevant search.

Search capabilities

• Full-text search on title/brand/category
• Filters:
  • price range
  • merchant
  • category
  • shipping
  • condition
  • rating
• Sort:
  • lowest total price
  • biggest discount
  • best rating
  • newest deals

Search stack options

• Elasticsearch / OpenSearch
• Meilisearch
• Typesense
• Or PostgreSQL full-text search for smaller MVPs

---

7) Add alerts and watchlists

Let users save products and price targets.

Alert types

• Price below target
• Price drop by percentage
• Back in stock
• New merchant offers lower price

Delivery methods

• Email
• Push notifications
• SMS
• In-app alerts

---

8) Design the backend architecture

A practical architecture:

Ingestion pipeline

• Merchant API/feed fetchers
• Scheduler/cron jobs
• Raw data storage
• Normalization service

Processing pipeline

• Deduplication
• Matching engine
• Pricing history updater
• Alert evaluator

Serving layer

• Search API
• Product comparison API
• User watchlist API
• Notification service

Storage

• PostgreSQL for core data
• Redis for caching and queues
• Object storage for raw feeds/images if needed
• Search index for fast product lookup

---

9) Handle data quality issues

This is critical in deal apps.

Common issues

• Outdated prices
• Missing shipping costs
• Duplicate listings
• Variant confusion
• Currency differences
• Temporarily unavailable products

Solutions

• Timestamp every price
• Prefer the most recent data
• Show “last updated”
• Re-crawl stale offers
• Normalize currency
• Separate product vs offer data

---

10) Consider legal and compliance concerns

• Respect merchant terms and API usage limits
• If scraping, ensure compliance with applicable laws and site policies
• Disclose affiliate relationships if you use affiliate links
• Handle user data carefully for alerts and wishlists

---

11) Recommended MVP stack

A simple, effective stack:

• Frontend: Next.js / React
• Backend: Node.js or Python FastAPI
• Database: PostgreSQL
• Search: Meilisearch or OpenSearch
• Jobs/Queue: Celery, BullMQ, or Sidekiq
• Cache: Redis
• Hosting: AWS / GCP / Azure / Render / Fly.io

---

12) Suggested build order

1. Ingest data from 1–2 merchants
2. Normalize catalog
3. Implement product matching
4. Build search and filters
5. Add price history
6. Add alerts/watchlists
7. Expand to more merchants
8. Improve matching and ranking

---

13) Key metrics to monitor

• Match precision/recall
• Data freshness
• Search latency
• Alert delivery success
• User conversion from search to click
• Price-drop alert engagement

---

If you want, I can also provide:

1. a database schema for this app,
2. a system architecture diagram in text, or
3. a step-by-step MVP implementation plan.