W3C AnnoteaDocumentation

Annotation Server HOWTO

Annotea project | server status | mailing list archive

This document is a general description of how to set up and use a W3C-Perllib Annotations server. Please see the debian-specific or windows-specific guide if you are installing on one of those systems. The W3C annotation server is maintained on a debian potato distribution. The debian-specific is maintained in parallel with this server and is therefor the recommended system.

The W3C PerlLib Annotation Server consists of a set of perl libraries, CGI scripts and configuration files. These libraries have been tested with MySQL and Apache. Following are the instructions for installing PerlLib, MySQL, Apache, and configuring the libraries to work with them.

Overall Steps

  1. Start with a Debian Woody distribution
  2. Install Perllib
  3. Install MySQL
  4. Set up RDF datastore with MySQL
  5. Install a Web Server
  6. Configure the Web Server
    1. Annotations Script
    2. Access Control
    3. Access Script
    4. Algae Script
  7. Debugging Tools
    1. apache authentication database
    2. Permissions
    3. Using Perldb
  8. Try It!
  9. Common Problems

1. Start with a Debian Woody distribution

These instructions assume the installation of a Debian Woody distribution. Installing a Debian Linux system is outside the scope of this document, however, this section can describe some of the problems you may encounter with an alternate distribution.

The most critical system dependency is matching the apache dbm auth module with the W3C/Annotations/CGI/access script. Apache web servers on different platforms use different Berkely DB forms (dbm, gdbm, ndbm, sdbm). See the section on configuring the access script for more information.

2. Install Perllib

Perllib installation for general Un*x is explained in the distribution guide and for Debian Linux system in the debian-specific guide. These instructions assume a working knowledge of perl on a Un*x server. In short, you need to get the libraries from CVS like so:

cd /usr/local
cvs -d :pserver:anonymous@dev.w3.org:/sources/public login
cvs -d :pserver:anonymous@dev.w3.org:/sources/public -z3 checkout -r V1_1 perl/modules

When prompted for a password for login, enter "anonymous".

PerlLib assumes a configuration directory relative to the installation directory.

mkdir /usr/local/perl/modules/Conf

3. Install MySQL

The libraries have been written to work with MySQL. We have not tried to use them with other SQL databases like PostGres. MySQL is available from mysql.com. Select a MySQL 3.x package and follow the included instructions to build and install the SQL server.

4. Set up RDF datastore with MySQL

Perl code implements an RDF datastore that is installed on top of MySQL. The Annotations server is an additional narrow shim on top of the general RDF database. The annotate CGI script's SQL access parameters are configured with the Conf/rdf.prop properties file:

cat > /usr/local/perl/modules/Conf/rdf.prop
user: root
password: 
database: rdf
queryHost: localhost:3306

5. Get a Web Server

Annotations have been set up to work with a standard CGI interface. The scripts have been tested under Apache. Apache is available from apache.org. Follow the instructions to build and install the Apache server. Note - Annotations requires mod_cgi but does not require suEXEC. The default installation and configuration works well. Note, annotea has not yet been tested with Apache 2.

6. Configure the Web Server

The Apache instructions also cover configuration of your web server.

6.1. Annotations Script

You need to configure Apache to run CGI scripts from the <perl libraries>/CGI directory. The following should work, with some path adjustment to reflect the actual location of your perllib installation:

ScriptAlias /annotations  /usr/local/perl/modules/W3C/Annotations/CGI/annotate
<Location "/annotate">
   AllowOverride None
   Options ExecCGI
   Allow from all
  AuthType Basic
  AuthName Annotations
  AuthDBMUserFile users
  AuthDBMGroupFile groups
    <Limit PUT POST DELETE>
        Require group registered
    </Limit>
</Location>

6.2. Access Control

You may wish to provide some authentication and trackability. To use Apache's basic authentication modules, uncomment the authentication-related directives in the Location "/access" directive (above).

You may wish to customize your own security policy, in which case you may wish to check out these notes on implementing authentication in a CGI script. There is a script to manipulate user access in W3C/Annotations/access. This is traditionally accessedvia the /access URL.

LoadModule dbm_auth_module /usr/lib/apache/1.3/mod_auth_dbm.so

ErrorDocument 401 /authRequired.html

ScriptAlias /access  /usr/local/perl/modules/W3C/Annotations/CGI/access
<Location "/access">
   AllowOverride None
   Options ExecCGI
   Allow from all
</Location>

The access script works with db, gdbm, ndbm, and sdbm formats. Most distributions of apache that come with linux distributions are linked to either db or ndbm libraries. You can use ddb(described below) to see exactly which format your apache is using. Assuming it is ndbm, add the following line to ...Conf/annotate.prop and Annotations.access.prop:

auth.database.parms: -type=>'ndbm', -file=> /etc/apache/users

This file should correspond to the one indicated in <Location "/annotate">'s AuthDBMUserFile directive.

The authRequired.html page should have a pointer to /access if you wish to advertise a facility for unknown users to sign up to use the service. For instance, annotest has uses the ErrorDocument directive:

ErrorDocument 401 /authRequired.html

to point to an HTML document with a link to /access.

<p>You may request an account by completing and submitting the
  <a href="/access">access request form</a>.</p>

6.3. Access Script

The access script is generally installed as a 404 handler and allows users to create authentication identities and store them in a Berkely DB file accessable to apache. The web server uses this database to provide access control and the annotations script uses it to find preferences for those authenticated users. The access script uses a properties file much like the RDF database properties file.

cat > /usr/local/perl/modules/Conf/Annotations.access.prop.prop
email.from=webmaster+iggy@w3.org
html.navbar="<a href="http://www.w3.org/"><img src="http://www.w3.org/Icons/WWW/w3c_home" alt="W3C" class="W3CIcon" border="0" /></a><a href="/"><img src="/Icons/pen3.png" alt="Annotea" width="150" height="110" border="0" /></a>"

If the apache is not running on a debian system, the authentication database may be need to be a different form. See the authentication database format section for instructions for determining the format of your database. Once the format is determined, add a line to the Annotations.access properties file to tell access what format to use.

auth.database.parms: -type => 'ndbm', -file => '/etc/apache/conf/users'

The type may be one of dbm,gdbm,ndbm,sdbm and the file will be the location indicated by the AuthDBMUserFile option in the httpd.conf.

6.4. Algae Script

The algae CGI script is usefull for exploring the results of queries.

ScriptAlias /algae  /usr/local/perl/modules/W3C/Rdf/CGI/algae
<Location "/algae">
  AllowOverride None
  Options ExecCGI
  Allow from all
  AuthType Basic
  AuthName W3C_RDF
  AuthDBMUserFile users
  AuthDBMGroupFile groups
  <Limit GET PUT POST DELETE>
    Require group registered
  </Limit>
</Location>

7. Debugging Tools

apache authentication database

The AuthDBMUserFile option in the httpd.conf may be an absolute path (from the root) or from the apache configuration directory. apache can tell you the configuration directory. Find the apache server:

ps ax| grep httpd
  376 ?        S      0:00 /usr/local/apache/bin/httpd
...

Find the conf directory:

/usr/local/apache/bin/httpd -V 
Server version: Apache/1.3.27 (Unix)
...
 -D HTTPD_ROOT="/usr/local/apache"
...
 -D SERVER_CONFIG_FILE="conf/httpd.conf"
...

This apache is look in /usr/local/apache/conf for the users file references in the above httpd.conf. The next issue is the format for that database. Makefile.PL attempts to find a user/password database and check the format, however, you may need to intercede if it fails.

The apache bin directory has a program called htpasswd which can be used to create and maintain password databases. To see the apache database format, create a database in apache 1.x:

/usr/local/apache/bin/dbmmanage /tmp/users add username password
User username added with password encrypted to password using crypt

or in apache 2.x:

htpasswd -c /tmp/users username

and use the perllib ddb program to convert it back to ascii. Try the different forms of database until it is able to interpret the database:

perl/modules$ cd W3C/Database/bin/
perl/modules/W3C/Database/bin$ ./ddb ifd=/tmp/users idelim=: odelim=:
couldn't open "/tmp/users" : 
W3C::Util::Exception thrown at ../../../W3C/Database/DbInterface.pm:79
...
# not a dbm file, try ndbm
perl/modules/W3C/Database/bin$ ./ddb ifn=/tmp/users idelim=: odelim=:
couldn't open "/tmp/users" : 
W3C::Util::Exception thrown at ../../../W3C/Database/DbInterface.pm:79
...
# nor ndbm, try gdbm
perl/modules/W3C/Database/bin$ ./ddg ifn=/tmp/users idelim=: odelim=:
username:password
# looks like gdbm, no need to try sdbm

annotate and access would need to be told about the above gdbm database. Put

auth.database.parms: -type => 'ndbm', -file => '/etc/apache/conf/users'

into the annotate.prop properties file.

Permissions

You will probably need to fix permissions if you are running annotate, bookmark, or access as a user. You should see messages like:

Exception: unable to open "/tmp/annotate.log": Permission denied

reminding you set read or write permissions for these files.

Using Perldb

Here are some handy idoms for checking/manipulating the users database with the perl debugger:

$ cd /etc/apache/conf/
$  perl -de 0
DB<1> use POSIX;

DB<2> use NDBM_File;

DB<3> tie(%users, 'NDBM_File', 'users', O_RDONLY, 0) || print "failure: $!"

DB<4> p join ("\n", keys %users)
joe@localhost
fred@expample.com
DB<5> p $users{'fred@expample.com'}
xxxxxxxx:2:0:3:yyyyyyyy:fred@expample.com:Smith:Fred:978120258:978120140
DB<6> untie(%users)

DB<7> tie(%users, 'NDBM_File', 'users', O_RDRW, 0) || print "failure: $!"

DB<8> %db{'joe@localhost'} = 'zzzzzzzz:2:0:3:aaaaaaaa:joe@localhost:Smith:John:978120258:978120140'

DB<9> untie(%users)

Property file example: *Note* this "documentation" will move shortly

The newUser script relies on a property file in Conf/newUser.prop for configuration:

email.from=webmaster@annotest.w3.org
html.navbar="<a href="http://www.w3.org/"><img src="http://www.w3.org/Icons/WWW/w3c_home" alt="W3C" class="W3CIcon" border="0" /></a><a href="/"><img src="/Icons/pen3.png" alt="Annotea" width="150" height="110" border="0" /></a>"

8. Try It!

You can use Amaya to test the annotations server. The server also has an HTML interface so you can create and query annotations with any browser. Direct the browser to http://<server>/annotatations, assuming that is the script name you selected when you configured the httpd.

Common Problems

OK, so that didn't work. You probably noticed this when you see something other than what you want showed up in your browser. This is useful information. So is the content of the error.log, which is in /var/log/apache/error.log in the standard setup.

I got a page of perl back.
Perhaps you don't have CGI enabled.
LoadModule cgi_module /usr/lib/apache/1.3/mod_cgi.so
Authentition Required.
You need to tell apache to read the correct authentication database and tell access what form of database to write. You may need to use dbmmanage and ddb to determine what form of database your particular installation of apache is expecting.
Auth Required page not found.
You need to create /var/www/authRequired. You will probably want to put in some HTML with a link to /access if you want people to be able to make their own accounts.
Internal Server Error

Look in your error log. It probably says something like

Premature end of script headers: /usr/local/perl/modules/W3C/Annotations/CGI/access

This happens when the script has a problem and responds with an error message rather than the HTML headers and body that apache is expecting from it. Run the program in question

cd /usr/local/perl/modules/W3C/Annotations/CGI/
./access DEBUG

You will usually see something like

unable to open "/tmp/access.log": No such file or directory
W3C::Util::Exception thrown at ../../../W3C/Util/W3CDebugCGI.pm:488
W3C::Util::W3CDebugCGI::LoadSelf(...) at ../../../W3C/Util/W3CDebugCGI.pm:139
W3C::Util::W3CDebugCGI::loadOrSave(...) at ../../../W3C/Util/W3CDebugCGI.pm:107
W3C::Util::W3CDebugCGI::new(...) at ./access:70
(eval) at ./access:68
W3C::Annotations::CGI::Access::main(...) at ./access:64

which will give you a hint as to how to proceed.


Eric Prud'hommeaux,
@(#) $Id: Annotations-HOWTO.html,v 1.47 2004/11/12 02:07:07 eric Exp $ Valid XHTML 1.0!