Anonymous Bot Authentication: Authorization and Rate Limiting for Web Agents

Automated agents ("bots") represent a large fraction of the traffic to many Web sites. In some cases, this traffic is desired, in others undesired, and in yet others, desired as long as it remains within certain rate limits. This memo describes Anonymous Bot Authentication (ABA), a system that allows Web site operators to distinguish wanted from unwanted traffic, while not tying a given request to a specific sender.