Crawl Budget
Crawl budget is how much crawling a search engine will do on your site in a given period. It becomes a real constraint on large sites, where wasted crawling on low-value URLs means important pages get visited less often.
Most small sites never hit the limit and shouldn't spend time on it. It matters past roughly ten thousand URLs, or where faceted navigation and parameters generate near-infinite variations of the same content.
The fix is reducing waste rather than requesting more crawling: block parameter combinations, avoid indexing thin generated pages, fix redirect chains, and keep the sitemap to canonical URLs only.
Why it matters
If you're planning to publish hundreds of programmatic pages, this is the constraint that decides whether they get indexed — and index bloat can drag down the pages you care about.
Related terms
More in SEO & AI Search
Need this built rather than explained?
We publish every price we charge, and you get a quote in writing before anything starts.