-
Statistical Analysis : Robots can be deployed to
discover and count the number of web servers. Other statistics could include
the average number of documents per web server, the proportion of certain
file types, the average size of a web page, the degree of interconnectedness
etc.
-
Maintenance : A Robot that verifies references can
assist an author in locating dead links and as such can assist in the maintenance
of the hypertext structure.
-
Mirroring : Mirroring is a popular technique for
maintaining FTP archives. A mirror copies an entire directory tree recursively
by FTP and then regularly retrieves those documents that have changed. This
allows load sharing, redundancy to cope up with host failures, and faster
and cheaper local access and off-line access. On the web, mirroring can
be implemented with a robot.
-
Resource Discovery : Perhaps the most exciting application
of robots is their use in resource discovery. Where humans cannot cope with
the amount of information, it is logical to let the computer do the work.
Robots summarise large parts of the web, and provide access to a database
with these results through a search engine.
|
