History
-----------------
21 Apr 1999: v2.0

	Uncompatible changes!!!!! Changed structure of table "url".
	Check 'create/alter19-20.txt' script to fix structure.

	* Added configure script to easy installation on different platforms.

	* Added robots.txt and <META Name="robots"...> support.

	* Added 'MaxHops' config file option
	Now one can define maximum way in 'mouse clicks' from start URLs.

	* Added META keywords and description processing.
	Now UdmSearch will find document first by default
	if there given words are found in keywords or description.
	UdmSearch stores keywords and description in database.
	One can easy add it into search result.

	* Added TitleWeight, BodyWeight, DescWeight, KeywordWeight 
	and UrlWeight config file options to define the 'weight' of
	each word in the different parts of the document. Weight is
	a number in the range 0..127. All weights are bit OR'ed if word
	appears in different parts at the same time.

	* Now one could easy configure indexer to run in 'ftpsearch'
	mode (search through URL rather then content of the document)
	and in 'checker' mode (search incorrect references on the site).
	Configuration samples are included.

	* Added 'FollowOutside' config file option to allow indexer
	follow outside servers given in indexer.conf. 
	Should be used carefully with MaxHops limit.

	* Added 'Index' config file option to allow/disallow
	indexer store found words in database

	* Added 'Follow' config file option to allow/disallow
	indexer store new found urls in database

	* Added 'CheckOnly regexp' config file option. Indexer
	will use HEAD instead of GET http method for URLs that 
	matches regexp. It means that the file will be checked only
	rather then downloaded. Usefull for zip,exe,arj and other
	binary files in 'ftpsearch' and 'checker' modes.

	* Added 'MaxNetErrors' config file option
	If there too many network errors on some server (server is down, 
	host unreachable etc) indexer will try to do not more then 'number' 
	attempts to connect to this server.

	* Added some nice command line arguments to indexer
	- Now one could easy reindex only subsection of database with given 
	filter(s): tag, URLs those match given patern (sql LIKE wildcards)
	or URLs with given HTTP status code.
	- One could easy reindex URLs even if not expired yet.
	- Now indexer can show some statistics of whole/part database.
	- One could easy delete URLs with given filter(s) or clear whole
	database.

	* Some fixes and nice changes in search.cgi and search.php3

29 Dec 1998: v1.9
	* Added basic http authorization (base64-encodes login:passwd)
	* Fixed some problems with FTP via proxy
	* Added automatic reconnect to mysql server

17 Dec 1998: v1.8
	* Added proxy support. 
	* Added FTP support when using proxy
	* Added User-Agent in HTTP header

3 Dec 1998: v1.7
	* Added PHP3 client
	Thanks for Mustapha MOUGHIT <emoughit@cie.fr> for porting
	it from php2.
	* Added If-Modified-Since in HTTP header
	* Best documents were not found first in search.cgi. Fixed.
	* search.cgi now reports count of found documents

19 Nov 1998: v1.6 
	* Added C cgi program to use it instead php/fi embedded html.
	It does not support advanced search with boolean query language yet.

18 Nov 1998: v1.5
	* Initial public release.
