pipette
ENEnglish

ScrapeBench: Evaluating Legal Compliance of AI Agents in Website Scraping

Joseph Marvin Imperial, Daniel Slate, Noam Kolt

Preprint

En palabras de los autores

AI agents deployed on the open web are becoming more capable of performing actions that may breach contractual provisions and violate computer security laws. In this paper, we introduce ScrapeBench, the first agentic benchmark for empirically evaluating the legal compliance of AI agents with website scraping policies, namely terms of service (ToS) and robots.txt directives. Across widely used closed-weight and open-weight AI agents, we find that most agents exhibit low compliance rates (< 5%) when instructed to scrape websites in our sample of 1,626 popular websites that explicitly prohibit scraping. In addition, we observe that even where AI agents actively check the applicable ToS and robots.txt, they continue to exhibit low compliance rates. Conversely, in our sample of 269 websites that do not prohibit scraping, we observe that agents sometimes refuse to scrape despite it being lawful, such as Claude Opus 4.6 exhibiting an over-refusal rate of 98.5%. Our human baseline experiments (n = 180 participants) suggest that humans exhibit compliance rates comparable to AI agents but over-refuse at much higher rates.

Resultado principalEl resumen no menciona limitaciones.

En palabras simples Resumen con IA

Los autores crearon ScrapeBench, la primera prueba para verificar si los agentes de IA siguen las reglas de extracción de datos de sitios web. Incluso cuando los agentes revisaron las reglas, siguieron mostrando tasas de cumplimiento bajas. En 269 sitios que permiten la extracción de datos, los agentes a veces se negaron de todos modos, con Claude Opus 4.6 rechazando el 98.5% de las veces.

Escrito por Kimi K2.5 usando solo el abstract de arriba. Jev verificó que cada oración esté respaldada por las oraciones del abstract que cita y eliminó las que no. Pasá el cursor por una oración para ver de dónde sale. Se eliminaron 2 oraciones por no estar respaldadas por el abstract.

Apareció: martes, 10 de noviembre. arXiv. Preprint, todavía sin revisión por pares.

Comentario de los autores: Accepted to NeurIPS 2026 (Evaluations and Datasets Track)