Abstract:
In this ongoing work, we introduce UCYMicra; a crawling system
that utilizes concepts similar to those found in Mobile and
distributed Crawling introduced in [3, 4]. UCYMicra extends
these concepts and introduces new ones in order to build a more
efficient model for distributed Web crawling, capable of keeping
up with Web document changes in real time. UCYMicra proposes
a complete distributed crawling strategy by utilizing Mobile
Agents technology. The goals are (a) to minimize network
utilization (b) to keep up with document changes by performing
on-site monitoring, (c) to avoid unnecessary overloading of the
Web servers by employing time realization, and (d) to be
upgradeable at run time.
Keywords: distributed web crawling, mobile agents
@inproceedings{papapetrou:www03poster,
author = {Odysseas Papapetrou and Stavros Papastavrou and George Samaras},
title = {Distributed Indexing of the Web using Migrating Crawlers.},
booktitle = {WWW (Alternate Track Papers {\&} Posters)},
year = {2003},
abstract-url={http://www2.cs.ucy.ac.cy/~cspapap/abstracts/www03.html},
url={http://www2.cs.ucy.ac.cy/~cspapap/publications/www03poster.pdf}
}