The Crawled Pages report shows you every page Site Audit crawled and the status of each one, so you can diagnose issues page by page instead of site-wide.
Crawled Pages
Click Manage columns to choose which columns appear in the table:
- ILR (Internal Link Rank)
- Page URL
- Blocked AI Search Bots
- Title
- Description
- Status Code
- Issues
- Crawl Depth
- Pageviews
- Load Time
- Markup
- Structured data
- Canonicalization
- Sitemap
- Incoming Int. Links
- Outgoing Int. Links
- Outgoing Ext. Links
- AMP link
- Hreflang
- JS and CSS files
- JS and CSS size
- Reaudit
Use the search bar to filter by URL, Blocked AI search bots to filter by which AI bots your robots.txt blocks, or Advanced filters to filter by issue status, crawl depth, HTTP status code, and more. You can combine multiple filters at once.
For example, add 2 filters to find pages that have issues and an Internal Link Rank above 70. Then sort by pageviews and fix the most important pages first.
Click the external link icon beside a URL to open that page in a new tab without leaving the report.
To re-crawl a single page, click the circular icon in the Reaudit column. This sends SiteAuditBot to that page only — an efficient way to track maintenance progress without spending crawl budget on a full audit.
To see your account's crawl budget, go to your Subscription Info Summary and look for Pages to crawl.
Understanding page attributes
|
Attribute |
What it tells you |
|
ILR |
Internal Link Rank. How accessible a page is, based on its incoming internal links and crawl depth. Higher ILR means more internal links and lower crawl depth. |
|
Blocked AI Search Bots |
Which AI search bots your robots.txt blocks from crawling the page |
|
Status Code |
The server's response to a request for the page. Pay attention to 4xx and 5xx codes — these mean the page is unavailable. |
|
Crawl Depth |
How many clicks it takes to reach the page from your homepage by the shortest path. The homepage is 0; a page linked from it is 1. |
|
Pageviews |
Unique pageviews from Google Analytics. Requires a connected Google Analytics account. |
|
Load Time |
How long the page's HTML takes to load. Large or numerous JavaScript (JS) and CSS files are a common cause of slow loading. |
|
Markup |
Which structured formats the page uses, such as Schema.org, Open Graph, and Twitter Cards. Markup helps search engines and social networks interpret your content. |
|
Structured data |
How many structured data items the page contains, by type |
|
Canonicalization |
Which URL the page points to as the preferred version. Canonical tags stop duplicate content from splitting your rankings. |
|
Sitemap |
Whether the page appears in your sitemap. Pages in your sitemap are easier for crawlers to find. |
|
Incoming Int. Links |
How many internal links point to the page |
|
Outgoing Int. Links |
How many internal links the page points to |
|
Outgoing Ext. Links |
How many external links the page points to |
|
AMP link |
Whether the page has an Accelerated Mobile Page version — simplified HTML that loads faster on mobile |
|
Hreflang |
The language and regional targeting of the page. Check these if you run localized versions of your site. |
|
JS and CSS files |
How many JavaScript and CSS files the page loads |
|
JS and CSS size |
The combined size of the page's JavaScript and CSS files |
For more detail on these attributes, see the Statistics report.
Best practice: Keep your most important content at a crawl depth of 3 or less, and make sure it appears in your sitemap.
Single Page view
Click any URL in the table to open the Single Page view for that page. The page URL, title, and HTTP status code appear at the top, followed by 2 sections:
- Page attributes: Internal Link Rank, Incoming Internal Links, Outgoing Internal Links, Outgoing External Links, Canonicalization, Markup, and Structured Data Items
- Issues Found: The total number of issues on the page, followed by each issue with the number of affected items and a How to fix link
Hover over the info icon beside an attribute for an explanation. Click the chevron beside an issue to expand the affected items, or click How to fix for guidance on resolving it.
Site Structure
Click Site Structure to see your site's subdomains and subfolders, with the number of URLs and issues in each. Expand a directory to see the subfolders inside it and identify which parts of your site need the most work.
Note: Site Audit builds this structure from the pages it crawled. If you limited your page count or set a crawl scope, the result may differ from your actual site structure. Adjust your campaign settings for a fuller picture.
Powrót do Zestawy narzędzi Semrush kategoria