2.1. Crawl Time vs. Inference Time

Diese Seite fasst den entsprechenden Abschnitt von RFC 9969 zusammen und bewahrt wichtige Begriffe, Daten und RFC-Verweise.

A central issue was the "crawl time vs. inference time" problem. Preference statements are apparent when content is crawled, either by location, such as robots.txt, or embedded in the content as metadata.

However, the target of those directives is often separated from the crawler. Crawl data may be used for more than AI model training, and some preferences may be intended to apply later, at inference time.