Pause scheduled web crawl for specified crawl configuration
Pause scheduled web crawl for specified crawl configuration
Your scheduled web crawl runs can overload origin servers or waste compute when unnecessary. Pause the targeted crawl automatically each morning to prevent load spikes before peak traffic.
Overview
Uncontrolled scheduled crawls can consume origin capacity and muddy performance metrics. This workflow pauses a targeted crawl automatically each morning so your SEO and SRE teams avoid load spikes and keep monitoring reliable before peak traffic.