W3C libwww Robot

Webbot - the Libwww Robot

Basics | Get it!FeaturesOptions | Release Notes | How to

The webbot is a very fast Web walker with support for regular expressions, SQL logging facilities, and many other features. The webbot comes with the libwww codebase. It can be used to check links, find bad HTML, map out a web site, download images, etc.


The Basics

Author(s)
Henrik Frystyk Nielsen, John Punin, Bob Racko
Status
An example application for libwww. The robot is the primary tool used for the HTTP performance measurements.
Plans
New releases follow the releases of libwww with no exception.
Platforms
It runs on the same platforms as libwww
Getting the Source Code
The webbot comes as an integral part of the libwww distribution package.
Forums
Discussions on libwww and its example applications take place on <www-lib@w3.org> (archives). See the documentation for how to subscribe.

Features

How to Run It

Be careful - this is a robot and hence can be used to traverse many links - it should be used with care and is not designed to be let loose on the Internet at large. Its primary design goal was to be able to test HTTP/1.1 pipelining features.

The robot has a large set of command line options that can be used in a large set of different combinations. You can try and see this simple script in order to see an example of how it can be run.


Henrik Frystyk Nielsen,
@(#) $Id: Overview.html,v 1.23 1999/05/04 13:18:52 frystyk Exp $