Diffbot misses too many fields on my pages

If Diffbot is missing too many fields, the usual causes are:

1. The page structure is too custom or dynamic

   • Content rendered heavily by JavaScript
   • Fields buried in nonstandard HTML
   • Data loaded after page load via API calls

2. The page doesn’t look like a supported article/product/page type

   • Diffbot works best when pages fit common patterns
   • Highly bespoke templates can reduce extraction quality

3. Important data is not visible in the rendered DOM

   • Hidden in scripts, JSON blobs, or network responses
   • Inside tabs, accordions, or lazy-loaded sections not captured

4. The page is incomplete without interaction

   • Requires scrolling, clicking, consent acceptance, or login
   • Diffbot may not see all content unless configured appropriately

What you can do:

• Inspect the rendered HTML and compare it to what Diffbot sees.
• Use custom extraction rules if the pages are consistent.
• Preprocess pages or feed Diffbot cleaner, fully rendered HTML.
• Check whether the missing fields are available in page metadata like JSON-LD, Open Graph, or embedded state.
• Test a few representative URLs to see if the miss is systematic or page-specific.

If you want, I can help you troubleshoot more precisely if you share:

• an example URL
• which fields are missing
• whether the page is JavaScript-heavy
• and whether you’re using Diffbot Article, Product, or custom extraction.