Better searching without Google

Course outline

09.30 - 09.45 Introduction to the course, trainer and delegates
09.45 - 10.30 The case against Google
10.30 - 10.45 Which search engine to use - coverage of engines
10.45 - 11.00 Coffee
11.00 - 13.00 Different types of search engines and alternatives to Google
13.00 - 14.00 Lunch
14.00 - 14.30 Re-ranking engines, visual search engines
14.30 - 15.00 Country based information, virtual libraries, academic searching
15.00 - 15.15 Coffee
15.15 - 16.00 Image and audio search, podcast searching, site searching
16.00 - 16.30 Weblog search engines, creating search engines and saving RSS searches
16.30 - 16.45 Roundup and last questions

The case against Google

Which other search engine should I use then?

None of them, and all of them. The search engine decoder is useful in telling you about the links between engines. Be aware of the difference in results:

How can I compare results?

What are the key types of search engine?

Strong alternatives to Google

Yahoo!

The database size is comparable to Google and in many cases the relevance ranking results are superior - especially with images.
Some of the search syntax is far superior to that offered by Google.
My 'How to do anything with Yahoo!' Squidoo lens.

Ask

Ask has made considerable retrograde steps recently and has limited functionality, particularly for UK users. In fact, it's very hard to get into the US version (.com) any longer, but if you follow this link: http://www.ask.com/?o=312&l=dir it'll get you there.
Smart answers is great for quick reference
Image searching is superior to Google
Quick links to narrow and refine searches or to expand them. This Sean Connery search is a good example.
Quick view/binoculars thumbshot (only available in the US)
RSS smart answers which you can test with a search for me, and see the most recent material that I've written about in my weblog
Maps and directions functionality is excellent though this is no longer available in the UK
The Search tools on the home page are excellent

Exalead

Very powerful search engine with a database of circa 8 billion pages.
Strong use of search syntax - proximity search (NEAR and NEAR/x)
Use of wildcard at the end of a sentence - librar* in a word for pattern matching - /psyc*ist/
Approximate spelling and phonetic spelling
Thumbnails of pages
Excellent opportunities for refining searches
Basic start page

Live

Microsoft's search engine.
Has a good current database
Basic functionality is the same as the other search engines, but advanced search does have a couple of interesting wrinkles; specifically re-ranking by currency, popularity and exactness.
Can save searches as RSS feeds by adding &format=rss to a URL that comes up when you've run a search.

Livesearch

From AlltheWeb, which is owned by Yahoo.
Results are suggested as you type.
Refine option makes suggests that actually are useful, though limited.

Accoona

This has now closed.

Major search engine test beds

SearchMash (Google)
AlltheWeb (Yahoo)
AltaVista (Yahoo)
MsDewey (Live/Microsoft)
Ask X (Ask)

How do search engines actually work?

Depends on the type of search engine. Some will emply robot or spider programs that wander around the web, and when they find a new page or site will copy the data back to their home base and will include the information when they next update their index. Other search engines, such as the Directory based services rely in web page authors visiting the engines and registering directly.
Those search engines that employ a ranking service will then also take into account a variety of things about the web page that they have returned to the user at the completition of a search. Some of the things that will be considered are:

Searching using an index or directory based engine

These are much simpler to use, since they are based on a hiarchical approach, going from broad subject headings to narrower ones. Simply drill down through the headings until you get to the section which interests you and view the websites listed. Alternatively, you can make use of the search facility that they provide. With Yahoo! for example this search facility will find not only subject headings but also individual sites. The major disadvantages of this type of engine are that they only index a very small percentage of the published websites, and they may not be arranged in a sensible way with regards the hierarchy.

Some examples:
Yahoo! Directory
http://www.dmoz.org/

Searching using a multi/meta search engine

The only multi/meta search engines that I ever use are Ixquick at www.ixquick.com, for its slight emphasis on UK based sites, and Kartoo at www.kartoo.combecause its rather different!
The advantage of using a multi search engine is that you will obtain a much more comprehensive overview of available pages, much more quickly than you'll ever get if you search one engine then another and so on. The major disadvantage is that you can really only use a low common denominator when it comes to searching; advanced syntax will not work, because many engines will not understand them. It's best to stick to phrase, + and - searching.
Alternatively, try Clusty at http://clusty.com/ or iZito at http://www.izito.com/
Others worth trying are:
Joongel
ZapMeta
Surfwax
Trovando
Whonu is quite complicated from the look of it, but worth a try.
Zuula Search goes through some of the major search engines for you
Fast Eagle which used to be calledCrossEngine which used to be called MrSapo.

Searching using a site specific search engine

These will find results/resources that you cannot find elsewhere because the data is contained in a database that can only be accessed via a specific search box on the website. All that traditional search engines can do is to index the content on the page, they can't run searches themselves, and then index the content that they find. Consequently, if you want to find specific information, such as a phone number or genealogical information you can use a search engine to find a likely website, and then use the search engine on the site to find the specific data you require. Some multi search engines such as ez2find may well suggest specific resources.

There are also specific search engines that you can use to search the hidden or the invisible web:

Personalised search engines

Community based searching

Delicious great for social bookmarking and also add /tag/searchterm at the end for specific searching
FURL is another social bookmarking service
digg has been around for a while now and is coming in for criticism, but worth trying for popular/current material. Also try digg swarm to see where people are going now.
reddit is also good for the serendipity material
Slideshare for powerpoint presentations
Spurl is like Furl
Trexy allows you to save 'trails' with their toolbar
Social Mention
TuSavvy

Re-Ranking search results

Exalead allows users to limit/re-rank by date
Grokker allows you to search and re-rank according to date

Using Visual Search Engines

Not all search engines display information in a textual manner - some of them allow you to view the data in rather different ways. These very often appeal to teenagers and students.

Kartoo provides a visual indication of importance of results and links between them. This was just about the first of this type of search engine and in many ways is the best. It allows you to easily limit a search, get an overview of a specific site and see the relationship between different sites.
Mooter provides an initial visual representation.
WebBrain displays a visual representation above, and links below
Quintura offers a tag cloud based on results
Ujiko provides an interesting approach.
SearchCrystal uses a dartboard approach
Touchgraph graphs Google results
RedZee uses a scrolling thumbnail approach

Searching for country based information

If you need to find a search engine that will concentrate on a particular country or region I've got a page on my site that covers exactly that! Some search engines are better than others at this, and I'd suggest both Exalead and Accoona as ways of doing a good geographical search.

Using Virtual Libraries: gateways to get what you need quicker.

What is a Virtual Library (VL)?

As the Internet has grown, so has the information to be found upon it. However, this leads to two major problems - how to find it, and how to assess the information when you finally get there. A VL is the answer to both of these questions. They are designed to offer quick and easy ways of finding quality information that can assist researchers in their work.

A VL is an online catalogue or directory of top quality information resources which can be found on the Internet. Quite often, a VL will allow users to read descriptions of those resources which they can assess, and then to go directly to those resources in order to use them. A VL will point to these resources and the user can go to them, confident in the knowledge that they have been selected and assessed by an information professional, making it the electronic networked equivalent of an academic research library.

Where can I find a list of VL's?

There are literally hundreds of VL's scattered around the globe, covering general subject areas, very specific subject areas, with high coverage, low coverage and so on. There is no standardisation or strict definition as to what a VL is or is not. However, there are some good starting points.

Academic searches

Infomine for scholarly internet resource collections
Scirus provides access to scientific information
Google Scholar provides access to scholarly articles
SMEALSearch for Academic business information (Now known as BizSeer)
Healia is designed for professionals, but great for all of us.
SearchMedica for GPs
Live Booksearch
Shakespeare searched from Clusty
Live Search academic is from Microsoft

Hidden or invisible or deep web resources

Turbo10 allows you to search over 800 specialised search engines
Complete Planet has access to over 70,000 resources
OAISter has an emphasis on academic information
Direct Search from Gary Price is no longer updated, but still has useful information

Image search engines

Yahoo Search has over 1.5 billion images
Fagan Finder lists several dozen good engines
Flickr is an excellent resource, though most images are copyright, so be careful when considering using them
Stockphotos is a resource for copyright free images, but still check the licence information
Picsearch has almost 2 billion images
Devilfinder is not bad, but display is poor
Cydral is a small search engine, I don't personally use it.
Morguefile provides access to stock royalty free images
Pixsy searches images and videos and displays by relevance or date
Like - find things that look like other things. Still a test version
Retrievr - find things by drawing them.
Xcavator is another drawing type search engine, still in demo only mode

Audio search

Yahoo audio search
The ear-archives are mainly television/film related
Commercial breaks and beats to find out what music was used on television adverts
What's that called? is similar to the previous engine
FindSounds is a search engine to find sounds, sound effects and musical instrument samples
Check with other search engines to see if they have an audio option.

Searching podcasts

PodZinger
Blinkx (also other multimedia search)
Podscope finds words spoken in podcasts

Information on specific sites

The Wayback machine may be able to show you an archived version of a site
Better Whois shows ownership for top level domains
Alexa provides traffic details and related sites

Local search

UpMyStreet has information on a wide variety of different aspects of particular areas.
StreetMap is mainly for maps and directions
Freesearch isn't one that I've used myself.
Live Local Search from Microsoft
Sherriff Ratings allows you to find and comment on companies
UK House prices does what it says on the tin.
UK Local is a Yellow Pages type affair

Book search engines

Online Books Page 25,000+ books
Bibliomania a smaller collection, dealing with classic texts
eBrary is a commecial product
Project Gutenberg has over 19,000 titles available

Weblog search engines

Making your own search engines

There has recently been a rise in utilities that allow you to create and store your own searches and let other people make use of them. This is particularly useful if you are dealing with a group of clients who want to do their own searching, but are not sure which resources are the best ones to use. The following is a list of a few that you might want to try for yourself:



Powered by Rollyo

Questions and Answers services

There are a variety of these available, though they are very variable in quality, and it's potluck if you're going to find an answer to your question. Wouldn't use these for serious research, but for simple, quick answers, or opinions, they may be worth trying.

Yahoo answers
Yahoo UK answers
Otavo quests
Ask Philosophers
Ask metafilter
Wondir
Answerbag
Microsoft Live QnA
Yedda

Saving searches using RSS

RSS is short for 'Rich Site Summary' or 'Really Simple Syndication' and is a way of getting a newsfeed, or a weblog, or updates to a page for example into a common format that can be used in a variety of different settings in a variety of different ways. Some search engines, such as Live, Google News, Icerocket, AllTh.at allow users to save searches into an RSS format.
These searches have to be saved into another resource, such as a start page like Pageflakes or an RSS reader like Bloglines.

More search engines


This page last updated on October 7th 2008