AI crawler
Applebot-Extended
Apple's training crawler.
- Vendor
- Apple
- robots.txt token
Applebot-Extended- Purpose
- Training
- Respects robots.txt
- Not a crawler. It is a control token that Apple reads from robots.txt
- Source
- Vendor documentation
What it does
Apple says Applebot-Extended lets publishers opt out of their content being used to train Apple's generative AI models.
Apple states that "Applebot-Extended does not crawl webpages". It only governs how data already crawled by Applebot is used.
What blocking it changes
Applebot-Extended collects data for model training. It is a different user agent from any AI search crawler, so a robots.txt rule for Applebot-Extended does not apply to them.
Apple says pages that disallow Applebot-Extended "can still be included in search results", and that its rules are not considered in ranking for Search.
robots.txt
Block it
User-agent: Applebot-Extended
Disallow: /Allow it
User-agent: Applebot-Extended
Allow: /Check your own site
The free Chompr.io extension shows each AI search crawler's status for the page you are on.