SEARCH
NEW RPMS
DIRECTORIES
ABOUT
FAQ
VARIOUS
BLOG

BotDetect - Real-Time Bot Detection API
 
 

MAN page from RedHat Other wwwoffle-2.1-1.i386.rpm

wwwoffle.conf

Section: File Formats (5)
Updated: February 18, 1998
Index 

NAME

wwwoffle.conf - The configuration file for the proxy server for the World Wide Web Offline Explorer. 

DESCRIPTION

Thewwwoffle.conffile contains the configuration for the wwwoffled proxy HTTP server part of theWorld Wide Web Offline Explorerprogram.

The file is split into sections, each of which has a format similar to a Cfunction (see the example below). The section has a name, followed on the nextline by an open brace '{', followed on the next lines by a list of names andvalues separated by an equals sign '=', the section ends with a close brace

TheStartUpsection can contain the following:

http-port = <port>
The port number to use on the local host as the HTTP proxy (default=8080).
wwwoffle-port = <port>
The port number to use on the local host as the wwwoffle control port(default=8081).
spool-dir = <dirname>
The name of the spool directory to use for the cache. A subdirectory is createdin this for each new web server that is contacted (default=/var/spool/wwwoffle).
run-uid = <username> | <uid> | none |
The username of numeric uid to run the wwwoffle server with. To use thisoption, the program must be started by root.
run-gid = <groupname> | <gid> | none |
The groupname of numeric gid to run the wwwoffle server with. To use thisoption, the program must be started by root.
use-syslog = yes | no
The syslog facility can be used to log the important error messages.(default=yes).
password = <word> | none |
The authorisation password that is required to use the wwwoffle program toconfigure the server or use the interactive control page (default=none). Ifthis is not present, or set to an empty string or 'none' then there is nopassword required. If there is a password set, then the -c option to wwwofflemust be used, and the file wwwoffle.conf should be made readable only byauthorised users.
max-servers = <integer>
max-fetch-servers = <integer>The maximum number of servers processes that are started (default=8). Themaximum number of server processes that are forked to fetch pages that wererequested in offline mode (default = 4). Themax-fetch-serversvalue must be less thanmax-serversor you will not be able to use wwwoffle interactively online while fetching.

TheOptionssection contains other options that configure the server.

log-level = debug | info | important | warning | fatal
The error messages that have a priority the same as that specified or greaterare recorded on the output, either syslog or stderr (see wwwoffled(1)).
fetch-images = yes | no
When pages are requested while offline and downloaded later, there is a choiceof whether to fetch the images from these pages as well (default=no).
fetch-frames = yes | no
When pages are requested while offline and downloaded later, there is a choiceof whether to fetch the frames from these pages as well (default=no).
index-latest-days = <age>
The maximum age in days of pages to show in the index of the latest pages(default=7).
add-info-refresh = yes | no
The option to add into the bottom of all of the spooled pages the date that thepage was cached and a refresh button (default=no). This uses the first entry inthe LocalHost section as the server name.
request-changed = <time>
While online pages will only be fetched if the cached version is older than thisspecified time in seconds (default=600). A negative value will force the cacheto always be used in preference.
pragma-no-cache = yes | no
Whether to request a new copy of a page if the request has 'Pragma: no-cache'(default=yes). This option should be set to 'no' if when browsing offline allpages are re-requested by a 'broken' browser.
offline-requests = yes | no
Whether to record requests that are made while offline or to return an error(default=yes).
monitor-interval = <age>
The interval in days between monitoring of the specified URLs (default=7 days).When this option is set to 0 it means to check each time wwwoffle is online, 1means once per day, etc.

TheLocalHostsection contains a list of possible names or IP addresses that the host runningwwwoffled may be known as.

hostname
The server may be known ashostnameso does not need to contact itself to get pages. All of the entries here arealso used as if they were in the LocalNet and AllowConnect sections. None ofthe entries here are fetched via a proxy.

TheLocalNetsection contains a list of host names or IP addresses that are not to be cachedbecause they are on the local network.

hostname
A server that matcheshostnameis on the local network and not to be cached. The match is done from the rightso that domain names can be used to match all hosts in the domain, IP addressesmatch from the left. All entries here are assumed to be reachable even whenoffline. All of the entries in the LocalHost section are also not cached as ifthey were here also. None of the entries here are fetched via a proxy.

TheAllowedConnectsection contains a list of host names or IP addresses that are allowed toconnect to the server.

hostname
A server that matcheshostnameis allowed to connect to the server. The match is done from the right so thatdomain names can be used to match all hosts in the domain, IP addresses matchfrom the left. All of the entries in the LocalHost section are also allowed toconnect.

TheDontCachesection contains a way of recognising URLs not to be cached. They will stillbe cached however if it is fetched non-interactively.

default = ...
By default don't cache files that match the right hand side. Note thatdefaultcan occur more than once with different right hand sides.
HOST-SPECIFICATION = ...
For URLs that matchHOST-SPECIFICATIONdon't cache files that match the right hand side.See the HOST SPECIFICATION section for details of theHOST-SPECIFICATIONoption.
=
FILE-SPECIFICATIONWhen getting files from the specified left hand side servers don't cache anythat matchFILE-SPECIFICATIONSee the FILE SPECIFICATION section for details of theFILE-SPECIFICATIONoption.

TheDontGetsection contains a way of recognising URLs not to be got. This can be used toreject junk adverts for example.

See the DontCache section for a description of the options in this section.

TheDontGetRecursivesection contains a way of recognising URLs not to be got when gettingrecursively.

See the DontCache section for a description of the options in this section.

TheCensorHeadersection contains a list of the header lines that are to be removed from therequest sent from the browser to the server.

header
The lines in the request that start withheader, followed by a ':' are removed before being passed to the server.

TheFTPOptionssection contains the information that is required to be able to do anonymous ftp.

anon-username = <string>
Specifies the username to use to fetch files using ftp (default is "anonymous","ftp" is another option).
anon-password = <string>
Specifies the password to use to fetch files using ftp (default is determinedfrom the user running wwwoffled and the hostname, this may not work reliablyespecially if you are behind a firewall).
auth-hostname = <host[:port]>
auth-username = <string>auth-password = <string>Specifies a triplet of hostname, username and password that allow non-anonymousaccess to a specific server. (These options must come in groups of three.)

TheMIMETypessection is a list of the mime type to associate with files that are not fetchedusing HTTP. This is required by browsers, most browsers come with a list thatcan be used here.

default = <mime-type>/<subtype>
The default MIME type to use for files that do not match any of the other rules.
.<file-ext> = <mime-type>/<subtype>
The MIME type to use for files that match the file extension.

TheProxysection contains a list of the hosts that are to be served via specified proxyservers. If no proxy is required then use 'none' or leave the proxy name blank.

default = <hostname:[port]> | none |
Specifies the default proxy that all requests are to use.
HOST-SPECIFICATION = ...
For URLs that matchHOST-SPECIFICATIONuse the specified proxy.See the HOST SPECIFICATION section for details of theHOST-SPECIFICATIONoption.
auth-hostname = <host[:port]>
auth-username = <string>auth-password = <string>Specifies a proxy server host that requires proxy authentication by username andpassword to use it. (These options must come in groups of three.)

None of the entries in the LocalHost or LocalNet section are fetched using aproxy.

TheMirrorsection allows alternate servers to be specified that are to be used inpreference to the one specified in a URL. This also allows multiple names forthe same server to be linked together so that a single spool directory is used.

protocol1/hostname1 = protocol2/hostname2
Whenprotocol1is used to accesshostname1the request is modified into a request forhostname2usingprotocol2, the two are also considered identical for the purposes of indexing, purgingand recursive fetching.
hostname1 = hostname2
Whenhostname1is accessed the request is modified into a request forhostname2for any of the protcols, the two are also considered identical for the purposesof indexing, purging and recursive fetching.

A symbolic link in the cache between protocol1/hostname1andprotocol2/hostname2will have the same effect as specifying them in the config file. The links areonly checked when wwwoffle starts or when 'wwwoffle -config' is run.

ThePurgesection controls how the cache is purged. The method to determine which pagesto purge, the default age, the host specific maximum age of the pages in days,and a maximum allowed cache size. An age of zero means to always delete when apurge is done, a negative age means never purge. The maximum cache sizeexcludes the files that are from hosts that are marked never to be purged.

use-mtime = yes | no
The decision of which pages to purge can be made on last access time (atime) orlast modification time (mtime) (default=no).
max-size = <size>
The maximum size of the cache in MB, excluding the hosts that are never to bepurged, if this is zero then it does not apply (default=0).
default = <age>
The age to purge hosts that are not otherwise specified here (default=28).
HOST-SPECIFICATION = ...
The age to purge hosts with URLs that matchHOST-SPECIFICATIONSee the HOST SPECIFICATION section for details of theHOST-SPECIFICATIONoption.

 

HOST SPECIFICATION

When specifying a host and protocol in many of the sections aHOST-SPECIFICATIONcan be used, this is a way of recognising a URL.

For the purposes of this explanation a URL is considered to be made up of threeparts.

proto
The protocol that is used (e.g. http, ftp)
host
The server hostname (e.g. www.gedanken.demon.co.uk) or a domainname (e.g. demon.co.uk) or an IP address (e.g. 1.2.3.4) or asubnet (e.g. 1.2.3).
port
The port number on the host (e.g. default of 80 for HTTP).

For example the wwwoffle homepage: http://www.gedanken.demon.co.uk/wwwoffle/The protocol is 'http', the host is 'www.gedanken.demon.co.uk', the port is thedefault (in this case 80).

AHOST-SPECIFICATIONcan be any one of the following

/
Any protocol, Any host, Any port
[/]<host>
Any protocol, Named host, Default port
[/]<host>:<port>
Any protocol, Named host, Specified port
[/]<host>:
Any protocol, Named host, Any port
<proto>/
Named protocol, Any host, Any port
<proto>/<host>
Named protocol, Named host, Default port
<proto>/<host>:<port>
Named protocol, Named host, Specified port
<proto>/<host>:
Named protocol, Named host, Any port

Where [] indicates an optional feature, and <> indicate a user supplied name ornumber.

Several of the sections allow aFILE-SPECIFICATIONto be entered, this is a way of recognising the path part of a URL.

AFILE-SPECIFICATIONcan be either of the following:

/<path>
Any file with a path that start with the specified one.
.<file-ext>
Any file that ends with the specified file extension.

Where <> indicate a user supplied name or number.(The '/' or '.' at the start is not optional.)

 

EXAMPLE


 StartUp
 {
  http-port     = 8080
  wwwoffle-port = 8081
  spool-dir     = /var/spool/wwwoffle
  use-syslog    = yes
  password      =
 }


 Options
 {
  fetch-images      = yes
  index-latest-days = 14
  add-info-refresh  = no
  request-changed   = 3600
 }


 LocalHost
 {
  localhost
  127.0.0.1
  wwwoffle.foo.com
 }


 LocalNet
 {
  foo.com
 }


 AllowedConnect
 {
  foo.com
 }


 Proxy
 {
  http/foo.com = www-cache.foo.com:8080
 }


 Purge
 {
  default  = 28
  max-size = 10
  bar.com  = 7
 } 

FILES

/etc/wwwoffle.conf The wwwoffled(8) configuration file.

/var/spool/wwwoffle The wwwoffle spool directory. 

SEE ALSO

wwwoffle(1), wwwoffled(8). 

AUTHOR

Andrew M. Bishop 1996,1997,1998 (ambAATTgedanken.demon.co.uk)


 

Index

NAME
DESCRIPTION
HOST SPECIFICATION
EXAMPLE
FILES
SEE ALSO
AUTHOR

This document was created byman2html,using the manual pages.
 
ICM Bot detect detector