W3 SERVER SOFTWARE A W3 server, like the ftp daemon , is a program which responds to an incoming tcp connection and provides a service to the caller. There are many varieties of W3 server software to serve different forms of data. Basic W3 servers CERN server The basic W3 daemon program serves files already in hypertext or plain text. This daemon then is used as a basis for many other types of server and gateways . Platforms: unix, VMS. NCSA server A server for files, written in C, public domain. Runs on top of a gopher-style database just like "gopherd". Platforms: unix. GN A single server providing both HTTP and Gopher access to the same data. In C, General Public License. Designed to help serevrs transition from gopher to WWW. Platforms: unix. Perl server from Marc VanHeyningen at Indiana University. Wriiten in perl . Platforms: unix Plexus Tony Sander's server originally based on Marc VH's, but incorporating lots more stuff, including an Archie gateway, etc etc. Platforms: unix. MacHTTP Server for the Macintosh. REXX for VM A server consisting of a amall C program which passes control to a server written in REXX. Whatever server you are running, you will probably be interested in: Tools for information providers Syle Guide for Online Hypertext Writing a new server This daemon is often used as a basis for a more specific server for a given application. A server which allows a world of data to be seen as part of the W3 universe is known as a gateway. (Most servers could therefore be regarded as gateways, but the term T. Berners-Lee 1 implies some conversion or mapping between dissimilar worlds) . For short tutorials with examples, see: Writing a server in C Writing a server as a script It is a good idea to pick the basic daemon or one of the servers in the list as a starting point when making a new server. Other servers and Gateways These are servers which provide data extracted from other systems. they are built using code from the basic daemon, or scripts. See List of Gateways available . Tim BL About documents generated from hypertext Paper manuals generated from hypertext are made for convenience, for example for reading when one has no computer to turn to. We have tried to make the hypertext into fairly conventional paper documents, but they may seem a little strange in some ways. All the links have been removed. Therefore, it is worth looking at the table of contents to see what there is in the manual. Something which is not explained in place may be explained in detail elsewhere. We have tried to keep related matter together, but sometimes necessarily you might have to check the table of contents to find it. Please remember that these are for the most part "living documents". That is, they are constantly changing to reflect current knowledge. If you see a statement such as "Product xxx does not support this feature", remember that it was the case when the document was generated, and may not be the same now. So if in doubt, check the online version. Of course, the living document may be out of date too, in which case it is helpful to mail its author. Tim BL WWW SERVER USER GUIDE The basic WWW server allows files and directories in a file system to be server to the world as menu trees, multimedia, and/or hypertext. The http daemon, httpd , is a general server program which runs a w3 protocol, " HTTP ". This is a TCP/IP based protocol running by T. Berners-Lee 2 convention on port 80. In this guide Distribution How to get the code. Compilation The daemon is compiled in the same way as the library and line mode browser -- see WWW distributed code . Installation How to install a server under unix internet daemon Options Command line options at run time Rule File The format of a rule file. By default, /etc/httpd.conf Etiquette Conventions you should follow to make life smoother Debugging If it doesn't seem to work Known bugs and improvements desired Change History change list of improvements made and bug fixes. Related documents HTML specification A description of the hypertext markup language used for representing menus, etc HTTP specification A desription of the protocol used by the server. Status of basic WWW server A basic fast information server for files. Author TBL Status: Version 2 available by anonymous FTP, with no index search but file access, name mapping and security filter, ability to act as gateway for anything in the WWW library's repertoire, including WAIS. Plans: A version which will allow general unix users to set up an index search daemon. As index search tools are not generally available, we may use the NeXT digital Librarian or WAIS as an basis. T. Berners-Lee 3 Platforms Unix, VMS, VM/CMS (VM/XA). Next Milestone: Run shell scripts to implement virtual documents and searches. More information: User guide , Bug list , Internals , Change history . Wider scope: W3 servers , Other WWW software Features include Installation under inetd or run stand-alone Can be run stand-alone by normal user Automatically generates hypertext view of directory tree Uses "README" files to document directory listings Handles multimple formats of same file, selects format apropriate for client capabilities Document name to filename mapping for longer-lived document names Can act as gateway for WAIS, news, etc if needed Provides access authorization WorldWideWeb CERN-distributed code See the CERN copyright . This is the README file which you get when you unwrap one of our tar files. These files contain information about hypertext, hypertext systems, and the WorldWideWeb project. If you have taken this with a .tar file, you will have only a subset of the files. THIS FILE IS A VERY ABRIDGED VERSION OF THE INFORMATION AVAILABLE ON THE WEB. IF IN DOUBT, READ THE WEB DIRECTLY. If you have not got ANY browser installed yet, do this by telnet to info.cern.ch (no username or password). Files from info.cern.ch are also mirrored on ftp.ripe.net. ARCHIVE DIRECTORY STRUCTURE Under /pub/www , besides this README file, you'll find bin , src and doc directories. The main archives are as follows: bin/xxx/bbbb Executable binaries of program bbbb for system xxx. Check what's there before you bother compiling. (Note HP700/8800 series is T. Berners-Lee 4 "snake") bin/next/WorldWideWeb_v.vv.tar.Z The Hypertext Browser/editor for the NeXT -- binary. src/WWWLibrary_v.vv.tar.Z The W3 Library. All source, and Makefiles for selected systems. src/WWWLineMode_v.vv.tar.Z The Line mode browser - all source, and Makefiles for selected systems. Requires the Library . src/WWWDaemon_v.vv.tar.Z The HTTP daemon, and WWW-WAIS gateway programs. Source. Requires the Library. src/WWWMailRobot_v.vv.tar.Z The Mail Robot. doc/WWWBook.tar.Z A snapshot of our internal documentation - we prefer you to access this on line -- see warnings below. BASIC WWW SOFTWARE INSTALLATION FROM SOURCE This applies to the line mode client and the server. Below, $prod means LineMode or Daemon depending on which you are building. Generated Directory structure The tar files are all designed to be unwrapped in the same (this) directory. They create different parts of a common directory tree under that directory. There may be some duplication. They also generate a few files in this directory: README.*, Copyright.*, and some installation instructions (.txt). The directory structure is, for product $prod and machine $WWW_MACH WWW/$prod/Implementation Source files for a given product WWW/$prod/Implementation/CommonMakefile The machine-independent parts of the Makefile for this product WWW/$prod/$WWW_MACH/ Area for compiling for a given system WWW/All/$WWW_MACH/Makefile.include The machine-dependent parts of the makefile T. Berners-Lee 5 for any product WWW/All/Implementation/Makefile.product A makefile which includes both parts above and so can be used from any product, any machine. Compilation on already supported platforms You must get the WWWLibrary tar file as well as the products you want and unwrap them all from the same directory. You must define the environmant variable WWW_MACH to be the architecure of your machine (sun4, decstation, rs6000, sgi, snake, etc) In directory WWW, type BUILD. Compilation on new platforms If your machine is not on the list: Make up a new subdirectory of that name under WWW/$prod and WWW/All, copying the contents of a basically similar architecture's directory. Check the WWW/All/$WWW_MACH/Makefile.include for suitable directory and flag definitions. Check the file tcp.h for the system-specific include file coordinates, etc. Send any changes you have to make back to www-request@info.cern.ch for inclusion into future releases. Once you have this set up, type BUILD. NEXTSTEP BROWSER/EDITOR The browser for the NeXT is those files contained in the application directory WWW/Next/Implementation/WorldWideWeb.app and is compiled. When you install the app, you may want to configure the default page, WorldWideWeb.app/default.html. These must point to some useful information! You should keep it up to date with pointers to info on your site and elsewhere. If you use the CERN home page note there is a link at the bottom to the master copy on our server. You should set up the address of your local news server with dwrite WorldWideWeb NewsHost news replacing the last word with the actual address of your news host. See Installation instructions . T. Berners-Lee 6 LINE MODE BROWSER Binaries of this for some systems are available in /pub/www/bin/ . The binaries can be picked up, set executable, and run immediately. If there is no binary, see "Installation from source" above. (See Installation notes ). Do the same thing (in the same directory) to the WWWLibrary_v.cc.tar.Z file to get the common library. You will have an ASCII printable manual in the file WWW/LineMode/Defaults/line-mode-guide.txt which you can print out at this stage. This is a frozen copy of some of the online documentation. Whe you install the browser, you may configure a default page. This is /usr/local/lib/WWW/default.html for the line mode browser. This must point to some useful information! You should keep it up to date with pointers to info on your site and elsewhere. If you use the CERN home page note there is a link at the bottom to the master copy on our server. Some basic documentation on the browser is delivered with the home page in the directory WWW/LineMode/Defaults. A separate tar file of that directory (WWWLineModeDefaults.tar.Z) is available if you just want to update that. The rest of the documentation is in hypertext, and so wil be readable most easily with a browser. We suggest that after installing the browser, you browse through the basic documentation so that you are aware of the options and customisation possibilities for example. SERVER The server can be run very simply under the internet daemon, to export a file directory tree as a browsable hypertext tree. Binaries are avilable for some platofrms, otherwise follow instructions above for compiling and then go on to " Installing the basic W3 server ". XMOSAIC XMosaic is an X11/Motif W3 browser. The sources and binaries are distributed separately from FTP.NCSA.UIUC.EDU , in /Web/xmosaic . Binaries are available for some platforms. If you have to build from source, check the README in the distribution. The binaries can be picked up, uncompressed, set "executable" and run immediately. T. Berners-Lee 7 VIOLA BROWSER FOR X11 Viola is an X11 application for reading global hypertext. If a binary is available from your machine, in /pub/www/bin/.../viola*, then take that and also the Viola "apps" tar file which contains the scripts you will need. To generate this from source, you will need both the W3 library and the Viola source files. There is an Imakefile with the viola source directory. You will need to generate the XPA and XPM libraries and the W3 library befere you make viola itself. DOCUMENTATION In the /pub/www/doc directory are a number articles, preprints and guides on the web. See the online WWW bibliography for a list of these and other articles, books, etc. and also the list of WWW Manuals available in text and postscript form. GENERAL Your comments will of course be most appreciated, on code, or information on the web which is out of date or misleading. If you write your own hypertext and make it available by anonymous ftp or using a server, tell us and we'll put some pointers to it in ours. Thus spreads the web... Tim Berners-Lee WorldWideWeb project CERN, 1211 Geneva 23, Switzerland Tel: +41 22 767 3755; Fax: +41 22 767 7155; email: timbl@info.cern.ch Installing the basic WWW server IIf using unix, for the simplest method see Installation under the Internet Daemon. There are special instructions if you are installing under VMS . The usual way to install a daemon is to either run it from the bootstrap command file (for example /etc/rc) so that it runs continuously, or to set up the internet daemon (inetd) to run it when a call comes in. See a csh script which does everything below for unix BSD systems but which you should modify with care for your own system. Note: With version 2.0 on, a rule file is no longer essential if T. Berners-Lee 8 you want to just export a directory tree. The installation normally requires superuser status, but it is poosible to run httpd from a terminal session as a normal user. ACCESS AUTHORIZATION See quick guide on how to set up access authorization (for versions 2.12 and newer). LOG FILE If a log file is required, make sure that the user name under which the daemon is run has the right to write the file Tim BL PRIVILIGED PORTS The TCP/IP port numbers below 1024 are special in that normal users are not allowed to run servers on them. This is a security feaure, in that if you connect to a service on one of these ports you are fairly sure that you have the real thing, and not a fake which some hacker has put up for you. The normal port number for W3 servers is port 80, which is such a port. (This number is assigned by the Internet Assigned Numbers Authority, IANA). When you run a server as a test from a non-priviliged account, you will normally test it on other ports, such as 2784 or 5000 typically. Under unix The inet daemon (running as root) can listen for incomming conections on port 80 and pass them down to a process with a safer uid for the server itself. Of course, you have to be root to set up the inet daemon. Under VMS Under UCX, The process running as a server needs BYPASS privilege to listen to ports below 1024. This might mean you have to install the server. With other TCP/IP packages, privilege of some sort is similarly required. _________________________________________________________________ Tim BL UNDER VMS T. Berners-Lee 9 The daemon runs just as under unix, for which the rest of the document ation was written. These instructions are my ideas about how to run i t under VMS but it is a long time since I did anything like this, so p lease tell me what is wrong. We don't have effort available to distrib ute HTTPD.EXE, sorry. Compilation of the daemon for VMS requires taking the library and Daem on source files from the unix release, copying them all onto the VMS s ystem, compiling them all. The object files from the Library should g o into a libwww.olb file. The object files from the Daemon should be linked together and with the libwww.olb library. When compiling the sources, you must use a compiler flag to specify wh ether you have Multinet, UCX or Wollongong TCP/IP. (cf rebuilding the line mode browser ). The flags should be one of: /DEF=MULTINET /DEF=WIN_TCP /DEF=UCX Running The daemon works with document names which look like unix-style filena mes. At the point of reading a file, these are converted into unix sty le filenames. Testing it Suppose you have compiled and linked httpd successfully. You write a " welcome.html" file as an introduction to your server for those from ou tside, and you put it in some suitable directory which you wish to exp ort, say sys$disk[my.public]welcome.html You run it as an ordinary user on a port over 1024 from a terminal win dow. httpd == $sys$disk:[my.directory]httpd.exe httpd -p 8000 -v "/sys$disk/my/public" Note that the directory to be exported is given in unix style. Don't p anic. Watch the trace (enabled by the -v option) . The server should e nd up waiting for a message. From another terminal window, you test the server, giving the internet node name of your machine in place if mynode.dom.ain and the same por t number. We assume you have the lin mode browser installed. You co uld test it with a GUI browser, but the trace might be more difficult to find. www -v "http://mynode.dom.ain:8000/welcome.html" T. Berners-Lee 10 You should now get your welcome page displayed on the terminal. theer will be a lot of trace as well which may make it almost unreadable, bu t if it works of course you run both server and/or client next time wi thout the -v. Installing properly. Check whwther your TCP/IP brand contains an inetd daemon. If it does, that is great, you just run it under the inetd daemon following the ma nufacturer's instructions. Set the daemon up to run on any TCP connec tion to port 80. (The service name for port 80 is http).In this case the only command line parameter which you will need to pass to httpd i s the directory name. Omit the port number to tell httpd that it is r unning under the inet daemon. If you find that this daemon is too slow (very possible under VMS), then switch to using the method below. If you don't have an Inet daemon, then you have to run the daemon as a detached process. To do this you have to add something to one of the many VMS boot startup files like SYSTARTUP.COMor some such. You need to be the system manager to do this, and if you are, you probably kno w where you personally like to put these things. The command line sho uld be as in the example when you tested it, except the port should be 80 (not 8000), and there should be no trace requested. In practice it seems that under VMS you always have to start a DCL env ironment to run a server, because if you just detach HTTPD you can't p ass it any parameters. So you use the usual trick of running loginout .exe to create a DCL environment: $RUN/DETACH/IN=SYS$EXE:HTTPD.COM/OUT=SYS$TEMP:HTTPD.LO G - SYS$SYSTEM:LOGINOUT.EXE where HTTPD.COM is $ httpd == $sys$disk:[my.directory]httpd.exe $ httpd -p 80 "/sys$disk/my/public" Check that out and tell me if it doesn't work... Tim BL INSTALLING A DAEMON UNDER INETD This is how to to set up the internet daemon (inetd) to run your HTTPD server whenever a request comes in. (These steps are the same for any daemon under unix: you will probably find a similar thing has been done for the FTP daemon, ftpd, for example.) Step1 Copy the daemon program or shell script ( httpd in this example) into a suitable directory such as /usr/etc. Protect it from anyone writing to it except root. T. Berners-Lee 11 Step2 Put "http" in the /etc/services file, or use the name of a specific service of your own if you want to use have a special port number. (Exceptions: on a NeXT, see using the NetInfomanager . On any machine running NIS (yellow pages), see specicial instructions ). For example, http 80/tcp # WorldWideWeb server Step3 Put a line in the internet daemon configuration file, /etc/inetd.conf. For example, http stream tcp nowait nobody /usr/etc/httpd httpd /Public (That was all one line.) Here "http" is used as a link between the services file and inetd.conf: it could have been any identifier. "nobody" is the user name under which you want the daemon to run, which determines what privileges it has for example to read data. "/usr/etc/httpd" is the actual file name of the server. The rest of the line is the arguments passed to httpd: arg0 is the program name, "httpd", by convention. Here the argument "/Public" is the directory tree to be exported. This is in fact the default if no directory is given. See command line syntax for more details. Note: The inted.conf format varies from system to system. If in doubt, copy the format of other lines in your existing inted.conf. For example, under ultrix there is no user name field -- everything runs as root. Note: there seem to be, on the NeXT at least, a limit of 4 arguments passed across by inetd! Step 4 When you have updated inted.conf, find out which process is running inetd, and send it a "HUP" signal. On BSD unix (For system V, use ps-el for ps aux) this looks like: > ps aux | grep inetd | grep -v grep root 85 0.0 0.9 1.24M 304K ? S 0:01 /u sr/etc/inetd > kill -HUP 85 > Test it T. Berners-Lee 12 Test the server with the line mode browser by giving its address explicitly: www http://myhost.dom.ain/welcome.html This assumes that you have a file "welcome.html" in your exported directory. If it doesn't work, you have probably missed something. See notes on debugging . Tim BL USING NIS (YELLOW PAGES) If your machine is running Sun's "Network Information Service", originally know as 'yellow pages", read this. You must: First make an addition to the /etc/services file just as for a normal unix system. Then, change directory to /var/yp and type "make". This will load the /etc/services file info the yellow pages information system. Some peopl ehave found that they needed to reboot he system afterward for the change to take effect. Tim BL ADDING A SERVICE ON THE NEXT The NeXT uses the the "netinfo" database instead of the /etc/services file. This is managed with the /NextAdmin/NetInforManager application. Here's how to add the service "www": Start the NetInfomanager by double-clicking on its icon. If you are operating in a cluster, open either your local domain (/hostname) or if you have authority, the whole cluster domain (/). If you're not in a cluster, just use the domain you are presented with. Select "services" from the browser tree. Select "ftp" from the list of services Select "dupliacte" from the edit menu. Select "copy of ftp" and double-click on its icon to get theproperty editor. T. Berners-Lee 13 Click on "name" and then on the value "copy of ftp". Change this to "www" by typing "www" in the window at the botton, and hitting return. Click on "port", and then on the value "21". Change it to "80". Use "Directory:Save" menu (Command/s) to save the result. You will have to give a root password or netinfo manager password. Tim BL The Rule File The rule file (configuration file) defines how the WWW software will translate a request into a document name. For a server, it allows one to provide an extra level of name mapping above that given by links in the file system. It allows, for example, out of date names to mapped onto their more recent counterparts. For the client, it allows access to certain servers to be remapped for example caching servers, or to local copies of the same information. The rule file also allows access to be restricted. This is essential, to prevent, for example, unauthorized access to your password file. By default, the rule file /etc/httpd.conf is loaded, unless specified otherwise with the -R or -r options . See also: example rule files , Old format for software before 2.0 , Setting up gateways , Firewall gateways . MAPPING AND FILTERING Each line consists of an operation code and one or two parameters, referred to as the template and the result. Anything on a line after and including a hash sign (#) is ignored, as are empty lines. The server uses the top rule first, then EACH SUCCESSIVE RULE unless told otherwise by PASS or FAIL. The operation codes are as follows map template result If the address matches the template, use the result string from now on for future rules. pass template If the address maches the template, use it as it is, porocessing no further rules. pass template result If the string matches the template, use the result string as it is, processing no futher rules. T. Berners-Lee 14 fail template If the address matches the template, prohibit access, processing no futher rules. The template string may contain at most one wildcard asterisk ("*"). The result string may have one wildcard only if the template has one. When matching, Rules are scanned from the top of the file to the bottom. If a request matches a "map" template exactly, the result string is used instead of the original string and applied to successive rules. If the request maches a "map" template with wildcard, then the text of the request which matches the wildcard is inserted in place of the wildcard in the result string to form the translated request. If the result string has no wildcard, it is used as it is. When a map substitution takes place, the rule scan continues with the next rule using the new string in place of the request. This is not the case if a pass ro fail is matched: they terminate the rule scan. Access Authorization From the version 2.12 on daemon supports access authorization which introduces two new rules: protect and defprot. They have the following syntax: defprot