Crawler

Update to the robots.txt parser

Mon, 11 Apr, 2022 at 10:56 AM

Why are broken links highlighted in relation to the "longdesc" tag?

Mon, 14 Feb, 2022 at 10:05 AM

What does "Pending site approval" mean?

Mon, 9 May, 2022 at 2:15 PM

Can Siteimprove scan a site built on our website framework?

Mon, 30 May, 2022 at 12:22 PM

Adding and replacing credentials for non-public sites

Fri, 7 Mar, 2025 at 7:12 PM

The Siteimprove Crawler – Bot Information

Wed, 4 Dec, 2024 at 6:24 PM

Crawl Errors: What They Mean and How to Fix Them

Mon, 29 Sep, 2025 at 2:56 PM