site change detection

If you mean detecting changes on a website/page over time, common approaches are:

1) Simple HTTP polling

• Fetch the page periodically
• Compare current content to the previous version
• Trigger an alert if different

Best for:

• Small pages
• Public pages
• Low-frequency checks

2) Diff the HTML/text

• Save the page HTML or extracted text
• Compare with a diff tool
• Filter out noisy changes like timestamps, ads, or random IDs

Best for:

• Monitoring specific sections of a page
• Reducing false positives

3) DOM/selector-based monitoring

• Extract only the element(s) you care about using CSS selectors/XPath
• Compare those values over time

Best for:

• Price changes
• Inventory status
• Job postings
• Specific content blocks

4) Headless browser monitoring

• Use Playwright, Puppeteer, or Selenium
• Render JavaScript-heavy pages
• Capture rendered DOM or screenshot

Best for:

• Single-page apps
• Dynamic content loaded after page load

5) Screenshot comparison

• Take periodic screenshots
• Compare images pixel-by-pixel or with perceptual diff

Best for:

• Visual layout changes
• Detecting UI changes

6) Third-party services

Examples:

• Visualping
• Distill.io
• ChangeTower
• Wachete

Best for:

• No-code monitoring
• Quick setup

---

Basic implementation idea

1. Download the page
2. Normalize content
   • remove whitespace
   • strip scripts/styles
   • ignore known dynamic parts
3. Hash the result
4. Store last hash
5. Compare on next run
6. Alert if changed

---

Example in Python

import requests
from bs4 import BeautifulSoup
import hashlib

url = "https://example.com"

html = requests.get(url, timeout=10).text
soup = BeautifulSoup(html, "html.parser")

# Remove noisy elements
for tag in soup(["script", "style", "noscript"]):
    tag.decompose()

text = " ".join(soup.get_text(" ", strip=True).split())
current_hash = hashlib.sha256(text.encode()).hexdigest()

print(current_hash)

---

Important considerations

• Robots.txt / Terms of Service: make sure monitoring is allowed
• Rate limiting: don’t poll too frequently
• Authentication: some pages require login/session handling
• False positives: ads, timestamps, and personalized content can change constantly

If you want, I can help you with:

• a Python script
• a JavaScript/Node.js version
• browser-based monitoring
• or a tool recommendation for your specific site.