Web-scraping AI bots cause disruption for scientific databases and journals
June 3, 2025

(Nature) – Automated programs gathering training data for artificial-intelligence tools are overwhelming academic websites.
In February, the online image repository DiscoverLife, which contains nearly three million photographs of different species, started to receive millions of hits to its website every day — a much higher volume than normal. At times, this spike in traffic was so high that it slowed the site down to the point that it became unusable. The culprit? Bots.
These automated programs, which attempt to ‘scrape’ large amounts of content from websites, are increasingly becoming a headache for scholarly publishers and researchers who run sites hosting journal papers, databases and other resources. (Read More)