Prompt
How do I troubleshoot pages that are crawled but not indexed?
Latest observation
Oct 5, 2026
OpenAI APIWeb search: off
Pages that are crawled but not indexed usually mean Google found the URL, fetched it, but decided not to keep it in the index. The fix depends on why.
1) Confirm the exact reason in Google Search Console
In URL Inspection, check:
- Crawled - currently not indexed
- Discovered - currently not indexed
- Duplicate, Google chose different canonical
- Alternate page with proper canonical
- Excluded by ‘noindex’
- Blocked by robots.txt
This reason tells you where to look next.
2) Make sure the page is indexable
Check the basics:
- The page returns 200 OK
- It does not have a
noindexmeta tag or header - It is not blocked by
robots.txt - The canonical tag points to the correct URL
- It is not redirecting unexpectedly
- It is accessible without login/paywall issues
3) Check for duplication and canonical issues
Google may skip pages that look like duplicates. Look for:
- Similar or near-identical content on multiple URLs
- Parameters, filters, faceted navigation creating many versions
- Canonicals pointing to another page
- Canonical chains or conflicts
Fix by:
- Consolidating duplicate URLs
- Using the correct canonical
- Strengthening the unique value of the page
4) Improve page quality and uniqueness
A common reason is that the page is technically crawlable but not valuable enough to index. Improve:
- Unique, substantial main content
- Clear purpose and search intent match
- Strong title and heading
- Unique text, images, data, or tools
- Internal links from important pages
Thin, template-heavy, or near-empty pages are often excluded.
5) Strengthen internal linking
Google indexes pages it sees as important. Do this:
- Link to the page from relevant high-authority pages
- Add it to category pages, hubs, or breadcrumbs
- Avoid orphan pages
- Use descriptive anchor text
6) Check crawl budget and site quality at scale
If many pages are affected:
- Remove low-value pages from indexation strategy
- Noindex or consolidate thin pages
- Improve site architecture
- Reduce parameter-generated duplicates
- Fix soft 404s and broken internal links
7) Re-test after fixes
After changes:
- Use URL Inspection → Test Live URL
- Request indexing
- Monitor coverage over the next few days/weeks
8) Common patterns by reason
Crawled - currently not indexed
Often:
- Low-quality or thin content
- Duplicate-like content
- Weak internal linking
Discovered - currently not indexed
Often:
- Crawl budget constraints
- Too many low-value URLs
- Poor site architecture
Duplicate, Google chose different canonical
Often:
- Canonical mismatch
- Duplicate content
- URL variations
Blocked by robots.txt / noindex
Often:
- Accidental technical settings
9) What not to do
- Don’t keep asking for indexing without fixing the underlying issue
- Don’t mass-publish thin pages
- Don’t canonical everything to the homepage
- Don’t use noindex on pages you actually want indexed
If you want, I can give you a step-by-step troubleshooting checklist or help diagnose a specific URL/status from Search Console.