Darius Rostam (Bucerius Law School) has posted Web Scraping in Culture and Law on SSRN. Here is the abstract:
This article examines the practice of web scraping through its under-illuminated cultural dimension. It situates scraping within internet culture, identifying semiotic markers of both openness and enclosure. Web scraping occupies an ambivalent position between these cultures, laden with symbols invoking both trespass and free access. The article analyzes how these cultural formations have historically shaped legal outcomes by identifying recurring argumentative narratives around privacy and authorization that reflect the cultural logics embedded in the law of web scraping. It demonstrates that cultural assumptions that supported a permissive legal treatment in the past do not hold for contemporary practices of AI training data collection. Yet the law has not itself enforced any particular culture – it merely demands that data hosts who wish to prevent scraping coherently signal their intent to enclose, while allowing both cultures to coexist. This antihegemonic stance proves increasingly untenable, complicated by the fact that both data hosts as well as AI companies operate as hybrids of openness and enclosure. The article advocates instead for explicit legal enforcement of either openness or enclosure.