Technical SEO Audit Playbook: From Crawl Budget to JavaScript Rendering
Posted: September 16, 2025 to Announcements.
Technical SEO Audit Playbook: Crawl Budget, Log Files, Sitemaps, Robots.txt, Canonicals, Pagination & JavaScript Rendering
Introduction
Technical SEO starts with making it easy for search engines to discover, render, and index the right content. This playbook walks through a practical audit flow that prioritizes crawl efficiency, index hygiene, and rendering integrity—backed by examples you can map to your own site.
Crawl Budget
Crawl budget is the number of URLs a bot will request within a timeframe. Waste it on duplicates, parameters, or soft 404s and important pages get crawled less.
- Identify crawl traps: infinite calendars, session IDs, and faceted combinations. Contain with disallow rules, noindex, and canonicalization.
- Prioritize freshness: ensure frequently updated and revenue-driving pages are internally linked within three clicks from the homepage.
Log Files
Server logs reveal what bots actually crawl, not what you hope they crawl. Segment by user agent, status code, and directory to spot inefficiencies and gaps.
Example: An eCommerce site found 38% of Googlebot hits on filtered URLs with zero impressions. After parameter handling and canonical fixes, crawl allocation shifted to top categories within two weeks.
Sitemaps
XML sitemaps should be a precise index of canonical, 200-status URLs. Split large sets by type and keep lastmod accurate to encourage recrawls.
- Exclude noindex, 3xx, 4xx, 5xx, and canonicalized duplicates.
- Host and reference via robots.txt and Search Console for monitoring.
Robots.txt
Use robots.txt to manage crawl paths, not indexation outcomes. Avoid blocking resources (CSS, JS) needed for rendering, and document rules with comments.
- Add a crawl-delay only if server strain is real; prefer server hardening.
- Handle parameters with Search Console and link architecture first.
Canonicals
Canonicals consolidate signals across variants (UTM, sort, pagination). Keep self-referencing canonicals on canonical pages and avoid conflicts with hreflang.
Example: Product pages with both hreflang and inconsistent canonicals lost signals; aligning them recovered rankings in localized markets.
Pagination
Since rel=prev/next deprecation, emphasize strong internal linking and unique page-level signals. Use self-referencing canonicals on each page, logical titles (Page 2, 3), and noindex only for non-browseable parameter views. Cap faceted depth and provide “view all” when performance allows.
JavaScript Rendering
Google’s two-wave indexing can delay JS-dependent content. Test raw HTML vs. rendered DOM; ensure critical content and links exist server-side or via hydration that renders instantly. Pre-render or adopt SSR/ISR for key templates.
- Audit with Fetch as Google/URL Inspection and compare innerText to source HTML.
- Example: Reviews loaded after 6s via XHR never indexed; embedding markup server-side restored rich results.