What's the best way to scrape hotel prices?

The best way is usually not to scrape the public hotel website directly if you can avoid it. Hotel prices are often dynamic, personalized, and protected by anti-bot measures, and scraping them can violate terms of service.

Better options

1. Use an official API

   • Best for reliability and legality.
   • Examples: Expedia, Booking.com partners, Amadeus, Sabre, Google Hotels partners, etc.
   • You’ll get structured data, availability, and price updates more cleanly.

2. Use affiliate/partner feeds

   • Many travel platforms offer price feeds or inventory access for partners.
   • Good if you need many hotels and regular updates.

3. Work with a data provider

   • If your use case is market intelligence, competitors, or price monitoring, a licensed third-party hotel data provider is often the safest path.

If you must scrape

If you have permission and the site allows it, the practical approach is:

• Check robots.txt and terms of service
• Use browser automation only when necessary
  • Playwright is usually better than Selenium for modern sites.
• Prefer network/API interception
  • Often the hotel price is loaded by an XHR/JSON request.
  • Capturing those responses is faster and more stable than parsing the rendered page.
• Handle dynamic pricing carefully
  • Prices can vary by dates, locale, currency, device, and session.
• Rate limit and rotate responsibly
  • Use caching, backoff, and low request rates.
• Expect anti-bot defenses
  • CAPTCHA, fingerprinting, and geo restrictions are common.
• Store raw responses
  • Makes debugging price changes much easier.

Recommended architecture

• Start with a search query endpoint or page
• Identify the API calls in browser dev tools
• Call the endpoint directly if permitted
• Normalize output into:
  • hotel ID
  • check-in/check-out
  • occupancy
  • currency
  • total price
  • taxes/fees
  • timestamp

Important caution

For hotel pricing, “scraping” can easily cross legal/contractual lines. If this is for a business or product, the safest route is usually licensed API access.

If you want, I can help you with:

• finding the underlying API calls in a hotel site,
• designing a price-monitoring pipeline,
• or comparing APIs for hotel pricing data.