ALIWEB

 ALIWEB (Archie Like Indexing for the WEB) can be considered the first Web search engine, as its predecessors were either built with different purposes (the Wanderer) or were literally just indexers (Archie, Gopher, Veronica and Jughead). First announced in November 1993 by developer Martijn Koster, and presented in May 1994 at the First International Conference on the World Wide Web at CERN in Geneva, ALIWEB preceded WebCrawler by several months. The earliest versions of archie simply contacted a list of FTP archives on a regular basis (contacting each roughly once a month, so as not to waste too much resources on the remote servers) and requested a listing.

These listings were stored in local files to be searched using the UNIX grep command. Later, more efficient front- and back-ends were developed, and the system spread from a local tool, to a network-wide resource, to a popular service available from multiple sites around the Internet. Such archie servers could be accessed in multiple ways: using a local client (such as archie or xarchie); telneting to a server directly; sending queries by electronic mail; and later via World Wide Web interfaces.

ALIWEB allowed users to submit the locations of index files on their sites which enabled the search engine to include webpages and add user-written page descriptions and keywords. This empowered webmasters to define the terms that would lead users to their pages, and also avoided setting bots (e.g. the Wanderer) which used up bandwidth.