2.1. Crawl Time vs. Inference Time

このページは RFC 9969 の該当節を要約し, 重要な用語, 日付, RFC 参照を保持します.

A central issue was the "crawl time vs. inference time" problem. Preference statements are apparent when content is crawled, either by location, such as robots.txt, or embedded in the content as metadata.

However, the target of those directives is often separated from the crawler. Crawl data may be used for more than AI model training, and some preferences may be intended to apply later, at inference time.