WWWOFFLE Homepage

World Wide Web Offline Explorer

The wwwoffled program is a simple proxy server with special features for use with intermittent internet links. This means that it is possible to browse web pages and read them without having to remain connected.

Basic Features

• Caching of HTTP, FTP and finger protocols.
• Allows the 'GET', 'HEAD', 'POST' and 'PUT' HTTP methods.
• Interactive or command line control of online/offline/autodial status.
• Highly configurable.
• Low maintenance, start/stop and online/offline status can be automated.

While Online

• Caching of pages that are viewed for later review.
• Conditional fetching to only get pages that have changed.
  • Based on expiration date, time since last fetched or once per session.
• Non cached support for SSL (Secure Socket Layer e.g. https).
• Caching for https connections. (compile time option).
• Can be used with one or more external proxies based on web page.
• Control which pages cannot be accessed.
  • Allow replacement of blocked pages.
• Control which pages are not to be stored in the cache.
• Create backups of cached pages when server cannot be contacted.
• Requests compressed pages from web servers (compile time option).
• Requests chunked transfer-encoding from web servers.

While Offline

• Can be configured to use dial-on-demand for pages that are not cached.
• Selection of pages to download next time online
  • Using normal browser to follow links.
  • Command line interface to select pages for downloading.
• Control which pages can be requested when offline.
• Provides non-cached access to intranet servers.

Automated Download

• Downloading of specified pages non-interactively.
• Options to automatically fetch objects in requested pages
  • Understands various types of pages
    • HTML 4.0, Java classes, VRML (partial), XML (partial).
  • Options to fetch different classes of objects
    • Images, Stylesheets, Frames, Scripts, Java or other objects.
  • Option to not fetch webbug images (images of 1 pixel square).
• Automatically follows links for pages that have been moved.
• Can monitor pages at regular intervals to fetch those that have changed.
• Recursive fetching
  • To specified depth.
  • On any host or limited to same server or same directory.
  • Chosen from command line or from browser.
  • Control over which links can be fetched recursively.

Convenience

• Optional information footer on HTML pages showing date cached and options.
• Options to modify HTML pages
  • Remove scripts.
  • Remove Java applets.
  • Remove stylesheets.
  • Remove shockwave flash animations.
  • Indicate cached and uncached links.
  • Remove the blink tag.
  • Remove the marquee tag.
  • Remove refresh tags.
  • Remove links to pages that are in the DontGet list.
  • Remove inline frames (iframes) that are in the DontGet list.
  • Replace images that are in the DontGet list.
  • Replace webbug images (images of 1 pixel square).
  • Demoronise HTML character sets.
  • Fix mixed Cyrillic character sets.
  • Stop animated GIFs.
  • Remove Cookies in meta tags.
• Provides information about cached pages
  • Headers, raw and modified.
  • Contents, images, links etc.
  • Source code unmodified by WWWOFFLE.
• Automatic proxy configuration with Proxy Auto-Config file.
• Searchable cache with the addition of the ht://Dig, mnoGoSearch (UdmSearch), Namazu or Hyper Estraier programs.
• Built in simple web-server for local pages.
  • HTTP and HTTPS access (compile time option).
  • Allows CGI scripts
• Timeouts to stop proxy lockups
  • DNS name lookups.
  • Remote server connection.
  • Data transfer.
• Continue or stop downloads interrupted by client.
  • Based on file size of fraction downloaded.
• Purging of pages from cache
  • Based on URL matching.
  • To keep the cache size below a specified limit.
  • To keep the free disk space above a specified limit.
  • Interactive or command line control.
  • Compression of cached pages based on age.
• Provides compressed pages to web browser (compile time option).
• Use chunked transfer-encoding to web browser.

Indexes

• Multiple indexes of pages stored in cache
  • Servers for each protocol (http, ftp ...).
  • Pages on each server.
  • Pages waiting to be fetched.
  • Pages fetched last time online.
  • Pages requested last time offline.
  • Pages monitored on a regular basis.
• Configurable indexes
  • Sorted by name, date, server domain name, type of file.
  • Options to delete, refresh or monitor pages.
  • Selection of complete list of pages or hide un-interesting pages.

Security

• Works with pages that require basic username/password authentication.
• Automates proxy authentication for external proxies that require it.
• Control over access to the proxy
  • Defaults to local host access only.
  • Host access configured by hostname or IP address.
  • Optional proxy authentication for user level access control.
• Optional password control for proxy management functions.
• HTTPS access to all proxy management web pages (compile time option).
• Can censor incoming and outgoing HTTP headers to maintain user privacy.

Configuration

• All options controlled using a configuration file.
• Interactive web page to allow editing of the configuration file.
• User customisable error and information pages.

The README file from the source code distribution goes into some more detail.

Documentation

There is a FAQ that contains much more detailed information about using WWWOFFLE.

The original WWWOFFLE homepage had a lot more information spread across various pages, most of which is now obsolete. The information that may still be relevant is listed below.

• Server Information
• Hints And Tips

Version Status

The latest major version number is version 2.9 and has been for several years. The absolute latest version is version 2.9j.

Recent Changes

The most recent changes in version 2.9j are listed below.

Bug Fixes

Fix to remove compilation warnings for [gs]etres[ug]id() functions.
Don't delete newlines from within quoted strings when modifying HTML.
Accept 'Transfer-Encoding: Chunked' instead of '... chunked'.
Increase the socket backlog to queue requests better.
Documentation changes for mailing lists, web-page and e-mail addresses.
Accept data from servers that use gzip encoding but don't send the tail.
Improve the error messages for gnutls problems. [*]
Some gnutls fixes and improvements. [*]
Fix an error with chunked encoding and compression from slow servers.
Fix HTML (and CSS) parsing problems related to media types.

Improvements:

Increase the root certificate and fake certificates to 2048 bits. [*]
Change the certificate hash function to SHA256 (SHA1 is deprecated). [*]
Use the SNI TLS extension for incoming and outgoing https connections. [*]

*NOTE* *These only apply if WWWOFFLE is compiled with SSL/https support*

The full version history is in the NEWS file.

Full History

Version 2.9j of WWWOFFLE released on Sat Mar 23 2016
Version 2.9i of WWWOFFLE released on Mon Oct 15 2012
Version 2.9h of WWWOFFLE released on Mon Oct 3 2011
Version 2.9g of WWWOFFLE released on Sun Mar 27 2011
Version 2.9f of WWWOFFLE released on Sun Jan 31 2010
Version 2.9e of WWWOFFLE released on Sun Jan 25 2009
Version 2.9d of WWWOFFLE released on Wed Jan 23 2008
Version 2.9c of WWWOFFLE released on Sun Jul 15 2007
Version 2.9b of WWWOFFLE released on Fri Feb 16 2007
Version 2.9 of WWWOFFLE released on Sun Apr 2 2006

Version 2.8 of WWWOFFLE released on Mon Oct 6 2003
Version 2.7 of WWWOFFLE released on Sat Feb 9 2002
Version 2.6 of WWWOFFLE released on Sat Nov 18 2000
Version 2.5 of WWWOFFLE released on Sun Sep 19 1999
Version 2.4 of WWWOFFLE released on Sat Dec 12 1998
Version 2.3 of WWWOFFLE released on Sat Aug 22 1998
Version 2.2 of WWWOFFLE released on Tue Jun 16 1998
Version 2.1 of WWWOFFLE released on Thu Mar 5 1998
Version 2.0 of WWWOFFLE released on Sun Dec 21 1997

Version 1.3 of WWWOFFLE released on Wed Sep 24 1997
Version 1.2 of WWWOFFLE released on Wed May 29 1997
Version 1.1 of WWWOFFLE released on Wed Mar 26 1997
Version 1.0 of WWWOFFLE released on Sat Jan 25 1997
Beta version 0.9 of WWWOFFLE released on Sat Jan 11 1997

Download

The source code for the versions listed above and a few more are in the download directory.

Subversion

The complete source code history can be browsed in the Subversion viewer which also has a list of the latest changes.

The source code can be downloaded from the Subversion repository with a command like the following:

svn co http://gedanken.org.uk/svn/wwwoffle/trunk wwwoffle

Release Policy

Before the end of 2012 there was a WWWOFFLE announcement mailing list which was used to announce new versions. The rate of new releases for this software is very low and the complexity of maintaining the mailing list outweighs the benefit. Starting in 2013 the mailing list has been disabled; no new subscriptions are accepted and no further e-mails will be sent to ex-subscribers.

Starting in 2013 there will be two types of software releases (if needed):

Formal releases which are the traditional method used for this software with version numbers like 2.9j. These will continue to be released when there are major changes or important bug fixes and each release includes full updates for release documentation. For these releases announcements will be made on this web page and with a new file in the download directory which includes an RSS feed for notification of new files.

Informal, intermediate, releases which have not previously been used for this software and will have version numbers like 2.9j-svnxyz. These will be used for small bug fixes or minor improvements to functionality. For these releases there will only be the software and directly related documentation changes, not a complete release documentation update. These informal versions will only be available from the download directory and included in the RSS feed found there.

Starting in 2014 there is direct access to the Subversion repository which should remove the need for any intermediate release versions.

Mailing List

The WWWOFFLE users mailing list that was used for discussion of WWWOFFLE features and bugs has also been discontinued at the end of 2012 for the same reasons.

Feedback

I am happy to answer questions that you have about the program. Please just drop me an email to the address at the bottom of the page; don't forget to put WWWOFFLE in the subject so that it ends up in the right mailbox.