Internet Tools for the Advanced Searcher.

The non-advertisement version of this page for downloading for printing as a handout is here: Internet Tools for the Advanced Searcher

Contents:
Search engines
Virtual libraries
Intelligent Agents
Weblogs
More effective ways of searching
A list of useful search engines

Search Engines

What types of search engines are there?

The number of different types of search engines break down into several major types:

How do search engines actually work?

Depends on the type of search engine. Some will emply robot or spider programs that wander around the web, and when they find a new page or site will copy the data back to their home base and will include the information when they next update their index. Other search engines, such as the Directory based services rely in web page authors visiting the engines and registering directly.
Those search engines that employ a ranking service will then also take into account a variety of things about the web page that they have returned to the user at the completition of a search. Some of the things that will be considered are:

Searching using Google

This cheat sheet is taken in part from the one provided by Google at http://www.google.com/help/cheatsheet.html

Search Example What it means  
     
Vacation Russia Find all pages containing Vacation and Russia, though not necessarily next to each other  
Russia OR England Find all pages containing Russia or England  
"holidays in Europe" Find all pages that contain that exact search string (up to 10 words total)  
virus -computer Find pages that contain virus, but exclude any that mention computer  
~guide Finds the word guide and any similar words as a synonym search  
red*blue both words next to each other  
Advanced Search What it means Example
site:www.philb Limits the search to one particular site site:www.philb.com google
site:.uk Limits the search to sites that end in www.something.uk  
site:.ac.uk Limits the search to sites that end in www.something.ac.uk  
[#]…[#] Search within a range of numbers DVD player $100...150
date: Search only a range of months Olympics date: 3
safesearch: Exclude adult-content safesearch: sex education
link: Linked pages link:www.philb.com
related: Related pages related:www.philb.com

More details

Basic search functionality - type in a number of keywords - generally up to about 4 or 5 to focus your search, though the total number of search words is limited to 10
Use '-' to exclude terms from your search, by placing it immediately before the word you wish to exclude, such as Everton -Liverpool
Phrase search by using "..." to search for an exact phrase - up to 10 words in length, such as "Everton Football Club"
Exclude phrases by using the minus sign immediately before the phrase, such as -"Liverpool Football Club"

Advanced search functionality - options to run AND OR NOT and phrase searches.
Language options, which are reasonably full.
File formats - varied, and better than other engines. These include Adobe, PowerPoint, Word, Spreadsheet
Date - limited and not impressive in comparison to Alltheweb
Occurrences - specific places on the page. Most of these are automatically taken into consideration by relevance ranking, so are of limited use.
Domain - specific site, domain, country or combination of the last two.
Similar pages - can be helpful to broaden out a search
Links - also broadens out a search

Multimedia - fine with images, but very limited with other multimedia. A better choice would be Yahoo video search at: http://video.search.yahoo.com/

Groups - the best (only!) resource for searching for newsgroup information

Directory - Yahoo! lookalike approach

News - over 4,000 resources constantly checked and updated

Google Toolbar - very effective, and it provides extra functionality such as page rank and page information.

Google Alerts - Google can automate up to 1,000 searches for you per day and email the results to you.

Google Suggest - Google will suggest terms for you as you type, and show you how many results you will get.

Soople is an easy to use advanced interface that works with Google. It clearly explains how to search and use all of the advanced search features that Google doesn’t clearly explain. Simply choose your options and run the search. You will then be taken to the Google results page and can continue as normal.

Search new additions to Google at Fresh Goo(gle) Or try Goofresh
Google sinkers - more accurate search results
Google proximity search function

Other Interesting Search Engines.

The BBC have their own search engine which is based on Google, but with a few BBC twists. I doubt that I'll use it myself, but at least take a quick look at it.
Northern Light is now dead.
Teoma at http://www.teoma.com/ and Wisenut at http://www.wisenut.com/ are two new search engines that have received good press recently and are certainly worth while investigating. Gigablastat http://www.gigablast.com/ is catching some peoples notice, but it's still small. Another search engine making a few waves is Turbo10 at http://turbo10.com/, but I'm not personally impressed with it. Another excellent example is Exalead at http://beta.exalead.com/search
Alternatively you could try the Microsoft offering at http://beta.search.msn.com/
If those engines are not enough for you, there are another 150 or so that you can try from another page on my site.

Searching using an index or directory based engine

These are much simpler to use, since they are based on a hiarchical approach, going from broad subject headings to narrower ones. Simply drill down through the headings until you get to the section which interests you and view the websites listed. Alternatively, you can make use of the search facility that they provide. With Yahoo! for example this search facility will find not only subject headings but also individual sites. The major disadvantages of this type of engine are that they only index a very small percentage of the published websites, and they may not be arranged in a sensible way with regards the hierarchy.

Some examples:
Yahoo! Directory
http://www.dmoz.org/

Searching using a multi/meta search engine

The only multi/meta search engines that I ever use are Ixquick at www.ixquick.com, for its slight emphasis on UK based sites, ez2 at www.ez2www.comfor its emphasis on the big search engines, and Kartoo at www.kartoo.combecause its rather different!
The advantage of using a multi search engine is that you will obtain a much more comprehensive overview of available pages, much more quickly than you'll ever get if you search one engine then another and so on. The major disadvantage is that you can really only use a low common denominator when it comes to searching; advanced syntax will not work, because many engines will not understand them. It's best to stick to phrase, + and - searching.
Alternatively, try Clusty at http://clusty.com/ or iZito at http://www.izito.com/index_search.htm

Natural language engines

Some search engines are now approaching the problem of finding information by using a different approach. Instead of using Boolean operators to find the required information, the user can simply type in their question using normal language. Some examples of the questions which work well are:

and so on. They don't work very well with more esoteric questions; I have found that it is best to use them for basic factual information. An example for you to try for yourself are:
Ask Jeeves

Searching for country based information

If you need to find a search engine that will concentrate on a particular country or region I've got a page on my site that covers exactly that!

Portal Services

A good introduction to portals can be found at Portals Community at http://www.portalscommunity.com/

Using Virtual Libraries: gateways to get what you need quicker.

What is a Virtual Library (VL)?

As the Internet has grown, so has the information to be found upon it. However, this leads to two major problems - how to find it, and how to assess the information when you finally get there. A VL is the answer to both of these questions. They are designed to offer quick and easy ways of finding quality information that can assist researchers in their work.

A VL is an online catalogue or directory of top quality information resources which can be found on the Internet. Quite often, a VL will allow users to read descriptions of those resources which they can assess, and then to go directly to those resources in order to use them. A VL will point to these resources and the user can go to them, confident in the knowledge that they have been selected and assessed by an information professional, making it the electronic networked equivalent of an academic research library.

A VL will provide you with access to a wide variety of different resources. SOSIG (the VL for the Social Sciences) for example describes and links to resources such as:

Each VL will point to resources which are appropriate for the subject it covers. SOSIG is again a good example; it will point you towards sources which cover:

Why and when should a VL be used?

Its worth considering using a VL in a number of different situations. If you have a very clear idea of what exactly you are looking for, you can use the appropriate VL to get you to the information you need very quickly, with the minimum fuss. Alternatively, if you are unsure of exactly what you want, but you know it is in a particular subject area it is worth spending time with the appropriate VL in order to clarify your ideas and focus yourself a little more clearly. As VL's are produced by hand, rather than electronically in the way that a search engine such as AltaVista is you can rest assured that you are going to be going direct to top quality information resources in which you can put some level of trust, rather than in aimless wandering around the Web.

However, if you want a comprehensive view of a subject area and are trying to find everything in a subject area, a VL will be a useful starting point, but it won't tell you everything. You may also need to do a search on one of the search engines for that overall picture. You may also find it worthwhile taking a moment to see when the VL of your choice was last updated - since this is done by hand you may find that the sources are not as current as those you will find using a search engine. On the other hand, since search engines are also working under a backlog it might be six of one and half a dozen of another.

Where can I find a list of VL's?

There are literally hundreds of VL's scattered around the globe, covering general subject areas, very specific subject areas, with high coverage, low coverage and so on. There is no standardisation or strict definition as to what a VL is or is not. However, there are some good starting points.

Intelligent Agents

What are these things?

Intelligent agents can be defined as pieces of software that must conform to a certain number of points:

Early implimentations of intelligent agents

The very first intelligent agents were, as you might guess, very basic indeed, and hardly deserved the term 'intelligent'. The one which most people recognise as the forerunner of what we have today is a program called Eliza at http://www-ai.ijs.si/eliza/eliza.html 'She' was designed to be a psychotherapist, and did little except echo back comments that you made to her. Although you can ask Eliza questions, she generally throws the question back to you, so the conversation is rather one-sided! However, since she was written in only just over 200 lines of code she is still quite impressive, and its possible to sit and chat to her for several minutes before you fall asleep from boredom!
Another version is ALICE - An Artificial Linguistic Computer Entity at http://www.alicebot.org/ which I personally didn't find very effective, but which is still worth taking a look at.
Alternatively, have a chat with Brain!
All three of these are known as Chatterbots for obvious reasons.
If you don't like any of these, you can visit the Botspot Chatterbot page at http://www.botspot.com/search/s-chat.htm to find some others.

Intelligent agents which learn from your preferences

There are a variety of agents which will learn from your likes and dislikes and will then attempt to make suggestions based upon your preferences. There are several nice examples of these on the Internet at the moment, and in particular I liked:
Alexandria Digital Literature Library at http://www.alexlit.com/This agent asks you to rate a number of books that you have read, and once you have input data rating a minimum of 40 titles it will be able to suggest other titles that it thinks you would probably enjoy reading. I tried the system out, and it seemed very top heavy with science fiction titles, but there is an option of choosing your own favourite authors and rating those as well. I was quite impressed with the results that were returned to me.
If you don't like this version, you may wish to explore the The Readers Robot which can be found at http://www.tnrdlib.bc.ca/rr.html
The Amazon bookshop at http://www.amazon.com/ has a facility which will update you every time books are added to its catalogue that match your own particular interests. It is pushing a point rather a lot to call it an 'intelligent agent', since it is very basic, but having said that, it is a useful feature of their website.

Portals

A 'portal' is the latest jargon term and is used to describe search engines which offer more than just the ability to search their indexes for appropriate web pages for you. The ones that I have found to be among the best are:
My Yahoo at http://my.yahoo.com/
My Excite at http://www.excite.com/

Intelligent search engines.

Search engines are becoming more intelligent, and a case can be made that they are moving from very basic and primative systems through to quite advanced pieces of software. One important aspect here is that they should be able to take a query, understand what is being asked and provide appropriate information.

Copernic Agent http://www.copernic.com/
Lexibot. This has now been superseded by DQM2 (Deep Query Manager) at http://brightplanet.com/products/dqm.asp
Letizia: An Agent That Assists Web Browsing. http://lieber.www.media.mit.edu/people/lieber/Lieberary/Letizia/Letizia-Intro.html
WebFerret at http://www.ferretsoft.com/ This isn't exactly an intelligent agent, but it is a useful tool which acts like a multi-search agent and interrogates a variety of search engines for you and displays the results neatly and compactly. There is a commercial version available, but you can also download a freebie.
The Autonomy suite of programs at http://www.autonomy.com/ There are a variety of different products, such as: The Daily briefing, which monitors a wide variety of different sites to seek out news in accordance with the users interests and displays these in an html format. Live Alert, which is a real time updating service, User profiling to find out exactly what will be of interest to a particular person and then produces content accordingly and Communities, which is software designed to put people with similiar interests in touch with each other - perfect for a large company intranet.
FirstStop Websearch http://www.firststopwebsearch.com/ is desktop software that searches multiple search engines and websites simultaneously for a more comprehensive Internet experience. This award winning, customizable multi-search engine has been described as the "FASTEST no-nonsense meta search for the net". (That's their quote, not mine by the way). I've played around with it a little bit and it seems to do exactly what it says on the tin. There are various commercial versions that are reasonably priced, and it's possible to get a free version under certain circumstances.

Updating bots

TrackEngine Useful little pull down link that you keep on your toolbar.
http://www.boutell.com/morning Morning Paper automatically visits your favorite web sites every so often to find out what's new, and presents a summary of what's new on each page as part of a "newspaper" which it displays in your web browser.
Tracerlock used to be a free service that will save your favorite search engine queries and web sites, check them periodically, and send you email whenever there are new or updated web pages. It's just moved to a commercial product however.
Mind-it. A commercial product. http://www.pumatech.com/mind-it_service/service.html#mindit
Change Detection at http://www.changedetection.com/monitor.html is a simple and free service, also worth looking at.
Infominder at http://www.infominder.com/ has received good press.
Watch that page at http://www.watchthatpage.com/ is a free service, with daily reports. (This wasn't working as of 7/703, but I'll keep it in until the next update, just in case it's a glitch)
Website watcher at http://aignes.net/ is shareware software that resides on your own computer. (Now a commercial product)

Daily newspapers

CRAYON, or CReAte Your Own Newspaper allows you to do exactly that. Quick and effective, well worth using. At http://crayon.net/

Resource sites

Botspot at http://botspot.com/ is THE single source for information on intelligent agents.
The UMBC Agent Web http://www.cs.umbc.edu/agents
An agent newsletter can be found at http://www.cs.umbc.edu/agents/agentnews/

Weblogs

What are weblogs?

A weblog is a website or page that is the product of (generally) an individual or of non-commercial origin that uses a date limited or diary format, and which is updated either daily or at least regularly with new information about a subject, range of subjects, or personal details.
This information may have been written by the author of the log, obtained from other sources on the web, contributed by others, or a combination of those. They are consequently usually topical and timely, and can be viewed as a developing commentary on a situation, event or subject.
Weblogs are also referred to as logs, Blogs, Web logs and so on. There appears to be no single standard way or referring to them.

There are a variety of different types of weblog, all doing different things. The single most popular weblog is Slashdot which is the work of programmer and graphic artist Rob Malden and some of his colleagues. Slashdot is an extended weblog, in that it carries discussion threads which are contributed to by various individuals, and on many subject areas, such as games, hardware, programming and so on. To this extent, it may appear to be more akin to a portal, rather than a diary.
At the other end of the spectrum is the Weblog of Jenny Levine The Shifted Librarian which is a personal weblog of an information professional.

Despite their differences, they have several key elements in common:

Brief history of weblogs

It could be argued that the first webpages, the creation of Tim Berners-Lee, were themselves a weblog while he was documenting the origins and growth of the environment he was creating.
However, it has only been towards the end of the last decade, 1997-98 that people started to create weblogs. The name ‘weblog’ was coined by Jorn Barger in December 1997. In 1998 the first list of weblogs was created at http://www.camworld.com/ Another listing, http://www.jjg.net/portal/tpoowl.htmllisted those weblogs that existed in the early days. This listing has not been updated since 12thOctober 2000, so it is of little use now as anything other than historical value. Peter Merholz established the pronounciation ‘wee-blog’, which then was shortened to blog, and the author or editor in turn became a ‘blogger’.
Weblogs shortly then began to expand as more people created them. Brigitte Eaton produced an early listing of every weblog that she was aware of at http://portal.eatonweb.com/ and the listing currently stands at 11,654.

A small sampling of some other blogs that you might like to take a look at:

Explodedlibrary.info at http://blogs.salon.com/0001429/
Gary Price – ResourceShelf at http://www.resourceshelf.com/
Phil Bradley’s blog at http://www.philb.com/blog/blogger.html
David Little’s weblog at http://www.david-little.org.uk/weblog/
Old coats and fairy rings at http://ecphrasis.port5.com/
L.A.C.K. Librarians are corrupting kids at http://www.lemurlove.com/lack/

How do you find weblogs that you want to read?

Using a general search engine
The easiest approach is simply to go to Google or some other search engine and run a search for weblog <insert subject area of interest>. For example, a search at Google for weblog librarian results in over 50,000 results, so you might want to add in a few more terms to narrow that down a little further. Google has a directory category that covers weblogs at: http://directory.google.com/Top/Computers/Internet/On_the_Web/Weblogs/Personal/?tc=1/Yahoo also has an offering at: http://dir.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/Weblogs/ (This is in my opinion a better collection than the Google offering)

Using a blog specific search engine.

Daypop at http://www.daypop.com/
Daypop searches 35,000 news sites, weblogs and RSS feeds for current events and breaking news. It crawls ‘the living web’ daily, and search options exist to allow searchers to search News, Weblogs, both or RSS headlines. All the usual search options are available – “phrase searching with quotes” +including –excluding. You can also search for a specific link to a URL with link:www.mysite.com or in the Advanced search function, limit to a country or language. There are also options to check the top 40 links, top news stories, top posts, word bursts, news bursts, and top weblogs.

Blogdex at http://blogdex.media.mit.edu/search.asp
This is very basic in terms of search techniques and syntax.

BlogStreet at http://www.blogstreet.com/
This search engine lists and indexes the way that blogs link to each other in neighbourhoods, so it can be quite useful in terms of finding other blogs that you might like to read based on those that you already do. It also allows you to search the top 10,000 blogs, view the top 100, and see the most important blogs. It indexes 138,000+ blogs.

Detod at http://blawgs.detod.com/
Specialised engine for searching legal blogs. As well as a search facility it also lists top stories (current to a few minutes).

BlogSphere news aggregator at http://www.alpern.org/weblog/php/blogsearch/writeup.html

Weblog directories

Another way of finding the right blog(s) for you. These directories work in the same way that Yahoo does – listing types, rather than being a search engine.

Library weblogs http://www.libdex.com/weblogs.html
This site is primarily designed to list weblogs by, for or about librarians.

Blogs4God http://blogs4god.com/linker/index.php
A semi definitive listing of Christian orientated weblogs.

Globe of blogs http://www.globeofblogs.com/
This lists blogs by location and topic. Quite small, with only 5,000+ listed.

Diarist http://www.diarist.net/
This tends to be more for personal blogs.

Diaries and Journals http://www.worldimage.com/diaries/
Very small collection of personal blogs.

Weblogs http://www.weblogs.com/
Huge collection, though not well arranged or organised.

LiveJournal http://www.livejournal.com/
Arranged by region, community or interest. Also has an option to start your own.

Weblogs specifically for librarians.

Acme Book News http://www.acmebook.com/
BookNews http://futureofthebook.com/
Engineering libraries – news for http://www.englib.info/
Eprintblog (academic bias) http://eprintblog.crimsonblog.com/
Liblog – library and technology oriented blog http://www.rcpl.info/services/liblog.html
Library notes – news and events of interest http://www.librarynotes.net/
Lisfeeds news and events of interest http://www.lisfeeds.com/
Manitoba Library Assocation weblog http://mla.blogspot.com/
Research Buzz – search engines and databases http://www.researchbuzz.com/news/index.shtml
Scholarly Electronic Publishing weblog http://info.lib.uh.edu/sepb/sepw.htm

News and Weblog aggregators

Amphetadesk http://www.disobey.com/amphetadesk/
This is the tool that I prefer to use. It’s fast, quick, effective and free. It’s a download and sits on your desktop.

Active Refresh http://www.activerefresh.com/
Website content monitor, checks news sources and aggregates RSS feeds. Commercial desktop product.

Abilon http://www.activerefresh.com/abilon.php
From the same company as Active Refresh, this is a free RSS aggregator.

Feedreader http://www.feedreader.com/module.php?mod=image&tid=2
Download desktop aggregator. I think it’s free, but it’s difficult to tell from it’s home page!

Fyuze http://www.fyuze.com/zero/
Online news aggregator. Free, but you need to register.

MyRSS http://myrss.com/
This one is slightly different because you can create an RSS channel out of any news type page. Easy way to keep an eye on what the competition is doing.

MyRSS by popular channel chttp://myrss.com/catalog/popular.html
Aggregates content from popular channels.

NewsIsFree http://www.newsisfree.com/
Aggregates content from 5600+ new sources. Online rather than download. Free and commercial versions of the product.

Newsgator http://www.newsgator.com/
An aggregator that works with MS Outlook. I don’t use Outlook, so can comment no further.

NewsMonster http://www.newsmonster.org/
Free download. Works with websites and news sites and weblogs

Wildgrape News desk http://www.wildgrape.net/
Aggregates rss feeds. Free, but requests donations.

Syndirella http://yole.ru/projects/syndirella/
Free download. Another news aggregator.

Creating your own weblog

Blog Easy http://www.blogeasy.com/
Simple, easy and free way to quickly create your own blog.

Blogger http://www.blogger.com/
This is a free tool, but there is a commercial version as well. Blogger can host your site, or it can be configured to update on your own server.

EasyBlog http://www.elka.cz/easyblog/howto.htm
Download the software onto your computer and update your blog onto your own server.

Electric Diary http://www.electricdiary.com/main.aspx
Emphasis is on writers and creating communities, but everyone is welcome to create a free blog.

LiveJournal http://www.livejournal.com/
Community feel, easy and free to set up a blog. Can update it either on their site or download a small utility to do it on your own computer.

Moveable Type http://www.movabletype.org/
Well regarded software package that you download and create your blog from your own computer. Free, but donations welcome.

Radio UserLand http://radio.userland.com/
Commercial product and very well regarded by the blogging community.

Xanga http://www.xanga.com/
Cheap (free!) and cheerful. Easy to set up and run. Allows for comments to people’s weblogs (if they wish). The whole design is for a ‘community’ feel.

Further sources of information

Blogging @ your library – feature article http://www.sls.lib.il.us/reference/por/features/2003/blogging.html

Stephen Cohen’s presentation on an introduction to RSS and blogging http://www.librarystuff.net/presentations/neasist04282003_files/frame.htm

Guardian article on working with newsreaders. http://www.guardian.co.uk/online/story/0,3605,781838,00.html

Guardian article on weblogs. Nice, concise, to the point and with a good listing of weblogs. http://www.guardian.co.uk/weblog/special/0,10627,744914,00.html

A short history of weblogs http://www.rebeccablood.net/essays/weblog_history.html

Matt Maldre’s comparison of blog systems if you want to create your own. http://www.spudart.org/etc/blogresearch/

RSS info. Good collection of aggregators, some of which I’ve not mentioned. Worth a look.http://blogspace.com/rss/readers

RSS technical specifications that you probably don’t want to know about! http://web.resource.org/rss/1.0/spec

Danny Sullivan (Searchenginewatch) article on weblogs http://www.searchenginewatch.com/sereport/article.php/2175281

More effective ways of searching.

Tips and tricks on quicker, better searching.
Ok, so you've now toddled around the Internet finding things and its well.... slow, really. This section gives you some hints and tips on how to make the whole thing go a lot faster. Some of them are obvious, and you're probably already using them. Hopefully however a few of them will be new to you. Try them out and see if they work.

Get a better modem.
If you're using a slow modem, you're going to get slow downloads. Think of your modem as a doorway; if its a large doorway, you can get people (or data) going backwards and forwards with no problem. If its a small doorway then people have to start forming orderly queues, and the whole thing slows down. Your modem is a little bit like that; the faster it is, the more data can go backwards and forwards. A 28,800 modem is ok, but thats about all. Better to go for a 56K if you can afford it. Its cheaper in the long run if you can! Even better is an ADSL connection.

Understand your browser inside out.

Since Netscape Navigator is pretty much dead on its feet these days poor thing, I'm pretty much ignoring it from now on. All the following will work with most recent versions of MSIE (MicroSoft Internet Explorer) version 6, may work with previous versions and will possibly work with Netscape, but I'm not making any promises.
There's no particular order to these hints and tips, so you'll just have to work your way through them!

Alternatively, try Firefox at http://www.firefox.com. This is an excellent alternative to Internet Explorer and you can read more about it in the article I wrote about Firefox.

Bookmarks

Use your bookmarks or Favorites to add pages to a listing, allowing you to easily get back to them when needed. Save them to your own free web space so that you can access them from anywhere using Backflip at http://www.backflip.com/login.ihtml

Home page

Choose a new home page to start from. You don't need to always go straight to the default if you don't want to. For example, I spend a lot of time using Google as my search engine, and I got fed up of typing in the address and going there every single time I started up MSIE. If you want to always start off at one particular site, choose Options/General Preferences/Appearance and edit the option Startup/Browser starts with by typing in the address of the page you want to start at. Alternatively, why not make your own page? Create it locally on your hard disk with links to the places you're likely to want to go to and start with that instead.

Caching

Your browser works by using what's called a 'cache'. This is essentially a storage area used by MSIE to store the pages which it has downloaded onto your hard disk for you to view. Thats why you'll often find that when you go backwards and forwards using the browser pages you've already visited come up quicker, because you're viewing them from disk, not going back out onto the Web to collect them again. There are two sorts of cache - one for hard disk, and one for memory.

Both of these can be changed to provide you with more memory and space. If you're running a pc with 8mb or above, you could change your memory cache to 1,000K by going to Options/Network Preferences and choose Cache to make the change.
To change the amount of space that MSIE will use to store files on the hard disk follow the same procedure as above and increase the amount of hard disk space to any figure that you feel appropriate, even up to something like 20Mb if you have that available.

There is also an option in that dialog box to allow you to clear the cache, and its worth while doing this now and again, since otherwise MSIE has to keep moving files in and out of memory, deleting old ones, adding new ones and so on. This slows down the whole procedure, while if MSIE has a clear cache to use it won't encounter these problems.

Offline readers

Use one of the offline readers such as WebWhacker at http://www.bluesquirrel.com/ to hoover up a page or site and read it later, off line at your leisure and at no online connect charges. (I'm all in favour of giving BT as little money as possible!)

Visiting URLs

If you don't know a specific URL, try guessing it. It often works! Not an infallible method certainly, but its worth trying. Become familiar with the major domain identifiers such as .com, .co.uk, .gov and country codes such as .uk, .de, .jp and so on. If you're looking for a UK organisation, try http://name_of_organisation.co.uk and if that doesn't work, just try it as a .com address instead.

If the page you go to doesn't come up and you get an error message, shorten the URL one stage at a time and work your way back up the chain. Eventually you'll arrive at a page of some sort, which might give you an index to what else is on the site, and you might find the page you're actually after listed at that point.

When and where it's best to search

Remember the time.
This depends very much on where the site is that you're interested in going to. If, like me, you spend a lot of time using American sites, you'll find its best to search them either first thing in the morning (up to about 12 o'clock) or later in the evening. This leaves the middle of our day (morning their time) for the Americans to log on and get their news. A good source of information on this is the Internet Traffic report at http://www.internettrafficreport.com/

Remember geography
While the Internet doesn't really care about local/global issues, you may want to, as I've already indicated. Remember what the different time zones are; if you can get the same file or information from a site in the US and one in Australia for example, my advice is to go to whichever country is currently 'asleep', since you'll get a faster response rate.

Bookmark your searches
If you run the same search on the same search engine on a regular basis, bookmark the first page of results. You can then return to the page at any point in the future and the search engine will re-run the search for you. Unfortunately, this will not always work, due to technicalities which are a pain to explain (which means that I don't fully understand them myself). Check the URL of your page, and if it contains the phrase 'cgi-bin' and has the words you entered within the string, you'll be able to re-use it. AltaVista is a good example here. If you're not sure, take a copy of the URL (cut and paste it) and wander off elsewhere. Paste the URL into the Go to box and if it takes you there and re-runs the search, you can bookmark it secure in the knowledge that you can run it whenever you want, and get updated results.

Plug in utilities.

What are they?
Your browser is a reasonably clever creature, in that it is able to identify that the page you want to load is a html page, it can find it (usually!) and display it on the screen in the way that the author of the page (probably) meant that you should see it. However, even browsers have limitations in what they can do, and how they do it. New facilities are constantly coming out onto the Internet and they come out quicker than the browsers can keep up with. Indeed, there is no particular reason why the browsers should be able to automatically read every single file that it comes across - they would need to be an awful lot bigger than they are already.

This is where the plug in utilities come into play. The browser loads the page, and sees that it is supposed to play a sound file, or display a moving image or whatever. It then hands control over to the appropriate utility, which downloads the file and displays it in the correct format. If the utility is not available the browser should indicate this to you, and give you the opportunity of downloading the utility, install it and tell the browser that its available for use. Consequently, when you go to the page next time, the utility will be able to do its job correctly.

There are a large number of places that you can obtain the utilities - the cover discs of Internet magazines is probably the easiest and fastest way of getting these, but if you want to get them directly from the Web, a good site to visit is: Tucows at http://www.tucows.com/ which gives you a very full and comprehensive listing, divided by operating system. You can also visit one of my web pages: http://www.philb.com/webtools.htm which I keep updated with lists of sites which carry the utilities.

Other useful utilities

Soople. A simple interface for using Google.
FURL - an online filing cabinet.
Toolbars - trash or treasure? Interesting article on what they are, comparisions and so on.

A list of some useful search engines

General search engines

A good list of general search engines can be found at http://www.philb.com/webse.htm though this is neither complete or checked that often for accuracy, but you've a fair chance of finding what you're looking for.


Yellow and White pages

BT Enquiries
Electronic Yellow Pages
FOUR11 (Yahoo! People search)
Internet Address Finder
UK Phone numbers
Who's Who Online
Ariadne article: Finding people

Searching for political/country information

CIA World Factbook

UK specific resources

British Government
British Search engines at http://www.philb.com/countryse.htm#uk

Searching newsgroups/mail lists

Google groups http://groups.google.com/
Topica
Mailbase(Old and archival material for Jiscmail)
Jiscmail
Tile.net

Searching specific resources

Acronyms http://www.chemie.fu-berlin.de/cgi-bin/acronym
The Bible
MediaUK http://www.mediauk.com/
Library of Congress http://lcweb.loc.gov/
Quotations

Searching for images/music

Lyrics.com
Picons (icons/images)
http://www.tasi.ac.uk/resources/searchengines.html
http://search.yahoo.com/images?&ei=UTF-8&p= Yahoo now has over 1 billion images in its search directory

Searching for software

Download.com
Shareware.com
Tucows

Virtual libraries

The WWW Virtual Library
ADAM (art and design)
ALEX (catalogue of electronic texts)
Biz/ed (business and economics)
BUBL (everything!)
Countryside Recreation Network
EELS (engineering electronic library)
HISTORY
HUMBUL (humanities)
NETEG (Netskills Network Education and Training)
OMNI (medicine)
OWL (orthopaedic)
RUDI (urban design)
SOSIG (social sciences)
WWEVL (waste water engineering)
Pinakes - excellent gateway to some major Virtual libraries.

New search engines

Fazzle is a multi search engine
Mooter is a graphical search engine
Eurekster for personalised results
Freesearch is a UK based search engine
http://www.yousearched.com/ has been designed for those with various impairments
http://www.ujiko.com/flash.php
http://www.a9.com/ is from Amazon.com
http://www.linkspider.co.uk/ UK meta search engine
http://www.ziggs.com/home.aspx Ziggs is designed to help you identify professionals in different areas. US Biased however.


© Phil Bradley, 2005.

This web page last updated 20th January 2005