Why technically valid Astro.js pages can still remain unindexed An Astro.js site can be technically crawlable and still have a meaningful share of URLs outside Google's index. The first job is to separate discovery, crawl and indexing states instead of treating every unindexed URL as the same failure.
Start with a URL inventory that joins the site route, sitemap presence, internal-link evidence and Search Console state. This creates a comparable dataset before any code or content changes are made.
Group affected URLs by Search Console indexing state. Keep indexed URLs as the control group. Record route template, sitemap presence, internal-link depth and rendered-content characteristics. Understand the Search Console states before changing anything These states describe different points in the discovery-to-indexing path. A URL that is discovered but not crawled needs a different investigation from a URL Google has already crawled and chosen not to index.
Discovered – currently not indexed: focus on discovery, crawl prioritization and how strongly the site signals page importance. Crawled – currently not indexed: compare content, rendering, duplication, canonical interpretation and page-level value against indexed peers. URL unknown to Google: verify that the URL is actually discoverable through the sitemap or internal links before looking for deeper indexing causes. What to investigate after robots, canonicals and schema are already correct If robots directives, canonical tags, schema, HTTP 200 responses and basic indexability have already been verified, repeating that checklist is unlikely to explain the difference between indexed and unindexed groups.
The higher-value next step is differential diagnosis: identify what is systematically different about the pages Google indexes versus the pages it does not.
Compare route templates and rendered main content. Compare internal-link count, link context and click depth. Compare sitemap inclusion and last-modified patterns. Compare content uniqueness and overlap across parameterized or templated pages. Do not restart from the generic SEO checklist The observed demand already reports the basic indexability layer as clean. The higher-value next step is to compare indexed and unindexed URL cohorts and isolate the layer where their signals diverge.
schema canonical robots HTTP 200 basic indexability
Indexing diagnosis flow Follow the failing layer in order. Do not change multiple layers at once and then guess which change mattered.
01 Discovery
Does Google know the URL?
If no, fix sitemap/internal discovery. 02 Crawl
Has Google fetched the URL?
If no, investigate crawl prioritization. 03 Canonical
Did Google select the expected canonical?
If no, investigate duplication/canonicalization. 04 Render
Is the rendered main content meaningfully distinct?
If no, investigate template/content output. 05 Internal support
Is the URL linked and treated as important?
If no, improve architecture and contextual links. 06 Verify
Did the same cohort change state after the fix?
Re-inspect the same URLs and record the result. Compare indexed and unindexed Astro.js routes For Astro.js, route generation can make many pages look structurally valid while still producing weak or repetitive page-level signals. Build two cohorts—indexed and unindexed—and compare them by route family instead of inspecting URLs one by one.
Route or content collection Template/layout Rendered title and primary heading Main-content length and uniqueness Internal links pointing to the URL Canonical target Sitemap membership Rendered HTML and page-template differences Static generation does not guarantee that every route carries the same quality of indexable signals. Inspect the final rendered HTML, not only the source component logic.
Look for page groups where the main content, headings, metadata or internal links collapse into near-identical output even though each URL returns 200.
Compare server-rendered HTML between indexed and unindexed examples. Check whether templated pages expose meaningful unique text above shared components. Confirm that navigation and contextual links point to important routes using stable hrefs. Internal linking, sitemap discovery and page importance A sitemap can expose a URL, but internal links still help communicate how the page fits into the site. The useful question is whether important pages are consistently discoverable from relevant indexed pages and whether the site architecture treats them as important.
Measure internal-link depth from strong indexed pages. Identify orphaned or weakly linked route groups. Check whether sitemap membership matches the URLs the site actually wants indexed. Use Search Console inspection data to build a diagnosis Once the affected URL cohorts are defined, combine Search Console inspection data with the same route-level evidence instead of treating the API as a separate reporting exercise.
Use the URL Inspection API at scale For a multi-URL diagnosis, the URL Inspection API is most useful when its output is joined back to your route inventory. The goal is not to request indexing blindly; it is to create a repeatable readback layer for inspection state, canonical interpretation and crawl/index information.
Sample representative URLs from each route group. Store inspection results with the route inventory. Compare Google-selected signals across indexed and unindexed cohorts. Re-run the same inspection set after targeted fixes. Build an indexing diagnosis matrix A diagnosis matrix turns a vague indexing complaint into a set of comparable evidence. It also prevents the team from changing multiple layers at once and losing the ability to tell which change mattered.
Illustrative cohort comparison — not a YEEDOOR client result.
URL Search Console state Route/template Sitemap Internal links Rendered uniqueness Google canonical Action URL A Indexed Template A Yes Strong High Self Control URL B Crawled – not indexed Template A Yes Weak Low Self Improve differentiation and linking URL C Discovered – not indexed Template B Yes Very weak Unknown Unknown Improve discovery/priority signals
URL URL A
Search Console state Indexed
Route/template Template A
Sitemap Yes
Internal links Strong
Rendered uniqueness High
Google canonical Self
Action Control
URL URL B
Search Console state Crawled – not indexed
Route/template Template A
Sitemap Yes
Internal links Weak
Rendered uniqueness Low
Google canonical Self
Action Improve differentiation and linking
URL URL C
Search Console state Discovered – not indexed
Route/template Template B
Sitemap Yes
Internal links Very weak
Rendered uniqueness Unknown
Google canonical Unknown
Action Improve discovery/priority signals
YEEDOOR FRAMEWORK YEEDOOR Indexing Diagnosis Matrix Compare indexed and unindexed URL cohorts using the same observable dimensions before changing the site.
Google knows the URL? If no → discovery problem. Google crawled the URL? If no → crawl prioritization/discovery problem. Google selected the expected canonical? If no → canonicalization/duplication investigation. Rendered content is meaningfully distinct and internally supported? If no → template/content/internal-link investigation. After a targeted fix → re-inspect the same cohort and verify index-state change. Fix, re-index and verify the same URL cohort Requesting re-indexing should come after a specific cause has been addressed. Keep the same URL cohort, record the change date, and compare inspection state plus Search Console coverage over time.
The success condition is not that a request was submitted. It is that the important URLs move into a stable indexed state and remain discoverable through the site architecture.
Change one diagnosable layer at a time where practical. Record affected URL cohorts and implementation dates. Re-read the same URLs after the change instead of relying only on aggregate coverage counts. Practical tool YEEDOOR Indexing Diagnosis Worksheet Use the same evidence columns for indexed and unindexed URL cohorts so changes can be compared before and after a targeted fix.
URL Search Console state Route/template Sitemap Internal links Rendered uniqueness Google canonical Last crawl Change date Action Verification result
Download CSV worksheet → Frequently asked questions Why can Astro.js pages be crawled but still not indexed? A crawl confirms that Google fetched the URL; it does not guarantee indexing. Compare the unindexed page against indexed peers for rendered uniqueness, internal-link strength, route/template patterns and Google's canonical interpretation.
What is the difference between Crawled – currently not indexed and Discovered – currently not indexed? Discovered means Google knows about the URL but has not necessarily crawled it yet; Crawled means Google fetched the URL but did not add it to the index at that time. They should be investigated as different stages of the pipeline.
Can Astro.js pages return HTTP 200 and still remain outside Google's index? Yes. HTTP 200 is necessary for a normal indexable page, but indexing also depends on how Google interprets the page, its canonical signals, content, duplication and its place in the site architecture.
How can the Search Console URL Inspection API help diagnose indexing issues at scale? Use it as a readback layer for representative URLs, then join the inspection results to your route inventory so indexed and unindexed cohorts can be compared systematically before and after fixes.
ABOUT THE AUTHOR SEO Strategist & Organic Growth Advisor at YEEDOOR
Ethan focuses on SEO diagnostics, technical SEO, search intent, content strategy, ecommerce SEO and organic growth. He works with YEEDOOR’s internal eight-person SEO team, supported by the company’s broader 17-year cross-border ecommerce operating background.
Get a free first-pass diagnosis before you change the site. Send us the URL and the Search Console state you're seeing. We'll help narrow the problem to discovery, crawl, canonicalization, rendering, internal support or verification.
Get a Free Diagnosis →