PROJECTS » Web Crawling
The L3S currently runs several web crawls for research purposes. It is possible that our crawlers index your website, too.
Currently, we are running crawlers on different machines in the subnet 18.104.22.168/24.
We always try to minimize additional workload to your servers by our crawlers. However, if you notice that our crawlers behave poorly, please send me an email. You can help us with tracking down the problem. In the worst case, we can simply exclude your website from being crawled.