IETF's AI Preferences group is still drafting, not shipping

The IETF has a working group standardizing how a site tells AI systems what they may do with its content, and it has been at it for a year and a half without publishing a single RFC. The AI Preferences (AIPREF) working group was chartered on 7 January 2025. As of September 2026 it has produced two Internet-Drafts. Neither is a finished standard.

What the charter actually asks for#

AIPREF’s charter describes the goal as standardizing “building blocks that allow for the expression of preferences about how content is collected and processed for Artificial Intelligence (AI) model development, deployment, and use.” It names Well-Known URIs under RFC 8615, specifically the Robots Exclusion Protocol of RFC 9309, and HTTP response header fields as candidate mechanisms for attaching those preferences to content. The charter frames these as delivery options the working group can build on, not as a problem statement about robots.txt itself, and the datatracker record does not contain language calling existing AI-vendor signals confusing or non-standard. That framing shows up in commentary around the group, not in the charter text.

The practical background is not controversial even without a quotable charter line: several AI companies already publish their own robots.txt user-agent tokens, GPTBot and Google-Extended among them, and site owners have had no single vocabulary for saying yes to search use while saying no to model training. AIPREF’s job is to give that distinction a standard shape. This is a separate effort from llms.txt, the informal file-at-the-root convention some blog posts have wrongly attributed to a W3C standards track; AIPREF never touches llms.txt and has no relationship to it.

The two drafts, and what they cover#

draft-ietf-aipref-vocab defines the preference vocabulary. As of version 08, dated 14 September 2026, it specifies three usage categories: train-ai, for using content to modify a generative AI model’s learned parameters; ai-use, for using content as input to a generative AI model; and search, for applications whose primary purpose is directing users to the original source. Version 07, in August, had only train-ai and search. A category takes allow or disallow, and one with no stated preference is unknown; the draft takes no position on a default.

draft-ietf-aipref-attach, version 05, also dated 18 August 2026, defines how a preference gets attached to content: a new Content-Usage directive inside robots.txt, and a matching Content-Usage HTTP response header for sites that would rather signal at the response level than the file level. Both mechanisms are meant to coexist with RFC 9309 rather than supersede it.

The vocabulary draft opens with a note that its contents do not reflect working group consensus in whole or in part. The vocabulary draft expires on 18 March 2027 and the attachment draft on 20 February 2027 if not revised or advanced. That is normal for an active draft, not a sign of stalling, but it does mean the vocabulary and the syntax can still change before either document reaches working group last call.

Why we care#

Nothing here changes what belongs in a robots.txt file today. The working group has not published an RFC, the vocabulary covers just three usage categories, and the attachment syntax is still draft text that IESG review could still alter. What’s worth tracking is the direction: if AIPREF finishes, sites get one IETF-sanctioned way to say “index and cite me, don’t train on me” instead of the patchwork of vendor-specific tokens crawlers currently interpret however they choose. Technical SEOs should watch draft-ietf-aipref-attach advance past working group last call before treating a Content-Usage directive as something to deploy, not before.