What should I use to find parameter URL crawl waste?

To find parameter URL crawl waste, use a combination of Google Search Console and server/log analysis.

Best tools

1. Google Search Console

   • Check Pages / Indexing reports for lots of parameterized URLs being crawled or indexed.
   • Use URL Inspection on sample parameter URLs to see how Google treats them.
   • Look for patterns like ?sort=, ?filter=, ?session=, ?page=.

2. Server logs or log analysis tools

   • Best way to see what search engines are actually crawling.
   • Look for repeated crawling of URLs with parameters that don’t add value.
   • Tools: Screaming Frog Log File Analyser, Splunk, ELK/Elastic, GoAccess, or your CDN logs.

3. Crawling tools

   • Use Screaming Frog, Sitebulb, or similar to discover parameter URL patterns and duplicates.
   • Compare canonical tags, noindex, redirects, and internal links.

4. Analytics

   • Check whether parameter URLs get traffic or create duplicate content issues.
   • Useful for spotting wasteful URLs that users don’t really need.

What to look for

• URLs with tracking parameters (utm_*, gclid)
• Faceted navigation parameters (sort, filter, color)
• Session IDs
• Pagination parameters
• Duplicate content with different parameter values

Practical workflow

• Start with Search Console to identify crawling/indexing issues.
• Confirm with log files to see crawler behavior.
• Use a crawler like Screaming Frog to map parameter URL patterns.
• Then decide whether to:
  • canonicalize
  • noindex
  • block in robots.txt
  • remove internal links
  • redirect
  • or leave them crawlable if they’re valuable

If you want, I can also give you a step-by-step process to measure parameter crawl waste specifically in Google Search Console and logs.