Contents:
Search engines
Virtual libraries
Intelligent Agents
Weblogs
More effective ways of
searching
A list of useful
search engines
The number of different types of search engines break down into several major types:
Depends on the type of search engine. Some will emply robot or spider
programs that wander around the web, and when they find a new page or site will
copy the data back to their home base and will include the information when
they next update their index. Other search engines, such as the Directory based
services rely in web page authors visiting the engines and registering
directly.
Those search engines that employ a ranking service will then also
take into account a variety of things about the web page that they have
returned to the user at the completition of a search. Some of the things that
will be considered are:
This cheat sheet is taken in part from the one provided by Google at http://www.google.com/help/cheatsheet.html
| Search Example | What it means | |
| Vacation Russia | Find all pages containing Vacation and Russia, though not necessarily next to each other | |
| Russia OR England | Find all pages containing Russia or England | |
| "holidays in Europe" | Find all pages that contain that exact search string | |
| virus -computer | Find pages that contain virus, but exclude any that mention computer | |
| ~guide | Finds the word guide and any similar words as a synonym search | |
| blind * mouse | both words with other words in between | |
| Advanced Search | What it means | Example |
| define: | Defines a word, with associated website link | define:library |
| site: | Limits the search to one particular site | site:www.philb.com |
| site:.uk | Limits the search to sites that end in www.something.uk | |
| site:.ac.uk | Limits the search to sites that end in www.something.ac.uk | |
| site:.gov.uk | Limits the search to sites that end in www.something.gov.uk | |
| [#]..[#] | Search within a range of numbers | DVD player $100..150 |
| date: | Search only a range of months | Olympics date: 3 |
| safesearch: | Exclude adult-content | safesearch: sex education |
| link: | Linked pages | link:www.philb.com |
| related: | Related pages | related:www.philb.com |
| 23 in roman numerals | Changes numbers to Roman Numerals | |
| 1 inch in mm | Changes inches to millimetres | |
| movie: | Information on movies | movie:lost boys |
| term a term b term b etc | More precise searching (Google Sinker) | dog cat cat cat cat cat |
Basic search functionality - type in a number of keywords -
generally up to about 4 or 5 to focus your search, though the total number of
search words is limited to 32
Use '-' to exclude terms from your search, by
placing it immediately before the word you wish to exclude, such as Everton
-Liverpool
Phrase search by using "..." to search for an exact phrase - up
to 10 words in length, such as "Everton Football Club"
Exclude phrases by
using the minus sign immediately before the phrase, such as -"Liverpool
Football Club"
Advanced search functionality - options to run AND OR NOT and
phrase searches.
Language options, which are reasonably full.
File
formats - varied, and better than other engines. These include Adobe,
PowerPoint, Word, Spreadsheet
Date - limited and not impressive in
comparison to Alltheweb
Occurrences - specific places on the page. Most of
these are automatically taken into consideration by relevance ranking, so are
of limited use.
Domain - specific site, domain, country or combination of
the last two.
Similar pages - can be helpful to broaden out a
search
Links - also broadens out a search
Multimedia - fine with images, but very limited with other multimedia. A better choice would be Yahoo video search at: http://video.search.yahoo.com/
Groups - the best (only!) resource for searching for newsgroup information
Directory - Yahoo! lookalike approach
News - over 4,000 resources constantly checked and updated
Google Toolbar - very effective, and it provides extra functionality such as page rank and page information.
Google Alerts - Google can automate up to 1,000 searches for you per day and email the results to you.
Google Suggest - Google will suggest terms for you as you type, and show you how many results you will get.
Soople is an easy to use advanced interface that works with Google. It clearly explains how to search and use all of the advanced search features that Google doesn’t clearly explain. Simply choose your options and run the search. You will then be taken to the Google results page and can continue as normal.
These are much simpler to use, since they are based on a hiarchical approach, going from broad subject headings to narrower ones. Simply drill down through the headings until you get to the section which interests you and view the websites listed. Alternatively, you can make use of the search facility that they provide. With Yahoo! for example this search facility will find not only subject headings but also individual sites. The major disadvantages of this type of engine are that they only index a very small percentage of the published websites, and they may not be arranged in a sensible way with regards the hierarchy.
Some examples:
Yahoo!
Directory
http://www.dmoz.org/
The only multi/meta search engines that I ever use are
Ixquick at www.ixquick.com, for its
slight emphasis on UK based sites, ez2 at www.ez2www.comfor its emphasis on the big
search engines, and Kartoo at
www.kartoo.combecause its rather different!
The advantage of using a
multi search engine is that you will obtain a much more comprehensive overview
of available pages, much more quickly than you'll ever get if you search one
engine then another and so on. The major disadvantage is that you can really
only use a low common denominator when it comes to searching; advanced syntax
will not work, because many engines will not understand them. It's best to
stick to phrase, + and - searching.
Alternatively, try Clusty at
http://clusty.com/ or iZito at
http://www.izito.com/index_search.htm
These don't really work very well - they never have and quite frankly I think that most of us are getting to the stage when we think that they never will. What we want instead are search engines that are able to give us precise answers to specific questions. It's worth trying out questions on different search engines to see how they cope with the answers - some will do better than others.
If you need to find a search engine that will concentrate on a particular country or region I've got a page on my site that covers exactly that!
What is a Virtual Library (VL)?
As the Internet has grown, so has the information to be found upon it. However, this leads to two major problems - how to find it, and how to assess the information when you finally get there. A VL is the answer to both of these questions. They are designed to offer quick and easy ways of finding quality information that can assist researchers in their work.
A VL is an online catalogue or directory of top quality information resources which can be found on the Internet. Quite often, a VL will allow users to read descriptions of those resources which they can assess, and then to go directly to those resources in order to use them. A VL will point to these resources and the user can go to them, confident in the knowledge that they have been selected and assessed by an information professional, making it the electronic networked equivalent of an academic research library.
A VL will provide you with access to a wide variety of different resources. SOSIG (the VL for the Social Sciences and now part of Intute) for example describes and links to resources such as:
Each VL will point to resources which are appropriate for the subject it covers. SOSIG is again a good example; it will point you towards sources which cover:
Why and when should a VL be used?
Its worth considering using a VL in a number of different situations. If you have a very clear idea of what exactly you are looking for, you can use the appropriate VL to get you to the information you need very quickly, with the minimum fuss. Alternatively, if you are unsure of exactly what you want, but you know it is in a particular subject area it is worth spending time with the appropriate VL in order to clarify your ideas and focus yourself a little more clearly. As VL's are produced by hand, rather than electronically in the way that a search engine such as AltaVista is you can rest assured that you are going to be going direct to top quality information resources in which you can put some level of trust, rather than in aimless wandering around the Web.
However, if you want a comprehensive view of a subject area and are trying to find everything in a subject area, a VL will be a useful starting point, but it won't tell you everything. You may also need to do a search on one of the search engines for that overall picture. You may also find it worthwhile taking a moment to see when the VL of your choice was last updated - since this is done by hand you may find that the sources are not as current as those you will find using a search engine. On the other hand, since search engines are also working under a backlog it might be six of one and half a dozen of another.
Where can I find a list of VL's?
There are literally hundreds of VL's scattered around the globe, covering general subject areas, very specific subject areas, with high coverage, low coverage and so on. There is no standardisation or strict definition as to what a VL is or is not. However, there are some good starting points.
Intelligent agents can be defined as pieces of software that must conform to a certain number of points:
The very first intelligent agents were, as you might guess, very basic
indeed, and hardly deserved the term 'intelligent'. The one which most people
recognise as the forerunner of what we have today is a program called Eliza at
http://www-ai.ijs.si/eliza/eliza.html
'She' was designed to be a psychotherapist, and did little except echo back
comments that you made to her. Although you can ask Eliza questions, she
generally throws the question back to you, so the conversation is rather
one-sided! However, since she was written in only just over 200 lines of code
she is still quite impressive, and its possible to sit and chat to her for
several minutes before you fall asleep from boredom!
Another version is
ALICE - An Artificial Linguistic Computer Entity at
http://www.pandorabots.com/pandora/talk?botid=f5d922d97e345aa1
which I personally didn't find very effective, but which is still worth taking
a look at.
These are known as Chatterbots for obvious reasons.
If
you don't like any of these, you can visit the Botspot Chatterbot page at
http://www.botspot.com/search/s-chat.htm
to find some others.
There are a variety of agents which will learn from your likes and
dislikes and will then attempt to make suggestions based upon your preferences.
There are several nice examples of these on the Internet at the moment, and in
particular I liked:
Alexandria Digital Literature Library at
http://www.alexlit.com/This agent asks
you to rate a number of books that you have read, and once you have input data
rating a minimum of 40 titles it will be able to suggest other titles that it
thinks you would probably enjoy reading. I tried the system out, and it seemed
very top heavy with science fiction titles, but there is an option of choosing
your own favourite authors and rating those as well. I was quite impressed with
the results that were returned to me.
If you don't like this version, you
may wish to explore the The Readers Robot which can be found at
http://www.tnrdlib.bc.ca/rr.html
The Amazon bookshop at http://www.amazon.com/ has a facility which
will update you every time books are added to its catalogue that match your own
particular interests. It is pushing a point rather a lot to call it an
'intelligent agent', since it is very basic, but having said that, it is a
useful feature of their website.
You can see another more detailed
collection of Web 2.0 based utilities at
http://www.philb.com/iwantto/discover.htm
Search engines are becoming more intelligent, and a case can be made that they are moving from very basic and primative systems through to quite advanced pieces of software. One important aspect here is that they should be able to take a query, understand what is being asked and provide appropriate information.
Copernic Agent http://www.copernic.com/
WebFerret at
http://www.ferretsoft.com/ This isn't
exactly an intelligent agent, but it is a useful tool which acts like a
multi-search agent and interrogates a variety of search engines for you and
displays the results neatly and compactly. There is a commercial version
available, but you can also download a freebie.
The Autonomy suite of
programs at http://www.autonomy.com/
There are a variety of different products, such as: The Daily briefing, which
monitors a wide variety of different sites to seek out news in accordance with
the users interests and displays these in an html format. Live Alert, which is
a real time updating service, User profiling to find out exactly what will be
of interest to a particular person and then produces content accordingly and
Communities, which is software designed to put people with similiar interests
in touch with each other - perfect for a large company intranet.
FirstStop
Websearch http://www.firststopwebsearch.com/
is desktop software that searches multiple search engines and websites
simultaneously for a more comprehensive Internet experience. This award
winning, customizable multi-search engine has been described as the "FASTEST
no-nonsense meta search for the net". (That's their quote, not mine by the
way). I've played around with it a little bit and it seems to do exactly what
it says on the tin. There are various commercial versions that are reasonably
priced, and it's possible to get a free version under certain
circumstances.
Watch that page at http://www.watchthatpage.com/ is a
free service, with daily reports.
TrackEngine Useful little pull down link
that you keep on your toolbar.
http://www.boutell.com/morning
Morning Paper automatically visits your favorite web sites every so often to
find out what's new, and presents a summary of what's new on each page as part
of a "newspaper" which it displays in your web browser.
Tracerlock used to be a free service that
will save your favorite search engine queries and web sites, check them
periodically, and send you email whenever there are new or updated web pages.
It's just moved to a commercial product however.
Change Detection at
http://www.changedetection.com/monitor.html
is a simple and free service, also worth looking at.
Infominder at
http://www.infominder.com/ has
received good press.
Website watcher at http://aignes.net/ is shareware software that
resides on your own computer. (Now a commercial product)
CRAYON, or CReAte Your Own Newspaper allows you to do exactly that. Quick and effective, well worth using. At http://crayon.net/
Botspot at http://botspot.com/ is THE
single source for information on intelligent agents.
The UMBC Agent Web
http://www.cs.umbc.edu/agents
A weblog is a website or page that is the product of (generally) an
individual or of non-commercial origin that uses a date limited or diary
format, and which is updated either daily or at least regularly with new
information about a subject, range of subjects, or personal details.
This
information may have been written by the author of the log, obtained from other
sources on the web, contributed by others, or a combination of those. They are
consequently usually topical and timely, and can be viewed as a developing
commentary on a situation, event or subject.
Weblogs are also referred to as
logs, Blogs, Web logs and so on. There appears to be no single standard way or
referring to them.
There are a variety of different types of weblog, all doing different
things. The single most popular weblog is Slashdot which is the work of programmer
and graphic artist Rob Malden and some of his colleagues. Slashdot is an
extended weblog, in that it carries discussion threads which are contributed to
by various individuals, and on many subject areas, such as games, hardware,
programming and so on. To this extent, it may appear to be more akin to a
portal, rather than a diary.
At the other end of the spectrum is the Weblog
of Jenny Levine The Shifted
Librarian which is a personal weblog of an information professional.
Despite their differences, they have several key elements in common:
It could be argued that the first webpages, the creation of Tim
Berners-Lee, were themselves a weblog while he was documenting the origins and
growth of the environment he was creating.
However, it has only been
towards the end of the last decade, 1997-98 that people started to create
weblogs. The name ‘weblog’ was coined by Jorn Barger in December
1997. In 1998 the first list of weblogs was created at
http://www.camworld.com/ Another
listing, http://www.jjg.net/portal/tpoowl.htmllisted
those weblogs that existed in the early days. This listing has not been updated
since 12thOctober 2000, so it is of little use now as anything other
than historical value. Peter Merholz
established the pronounciation ‘wee-blog’, which then was shortened
to blog, and the author or editor in turn became a ‘blogger’.
Weblogs shortly then began to expand as more people created them. Brigitte
Eaton produced an early listing of every weblog that she was aware of at
http://portal.eatonweb.com/ and the
listing currently stands at 62095.
A small sampling of some other blogs that you might like to take a look at:
Explodedlibrary.info at
http://explodedlibrary.typepad.com/salonblog/
Gary
Price – ResourceShelf at http://www.resourceshelf.com/
Phil
Bradley’s blog at http://www.philbradley.typepad.com/
and check out my Blogroll for a complete listing of what I take.
Jill
Bradley's blog - a Reflexologist in
Billericay Essex, at http://essextherapies.blogspot.com/
Lots
more librarian weblogs at
http://www.pageflakes.com/philipbradley.ashx?page=4541261
Using a general search engine
The easiest approach is simply
to go to Google or some other search engine and run a search for weblog
<insert subject area of interest>. For example, a search at Google for
weblog librarian results in over 50,000 results, so you might want to add in a
few more terms to narrow that down a little further. Google has a directory
category that covers weblogs at:
http://directory.google.com/Top/Computers/Internet/On_the_Web/Weblogs/Personal/?tc=1/Yahoo
also has an offering at:
http://dir.yahoo.com/Computers_and_Internet/Internet/World_Wide_Web/Weblogs/
(This is in my opinion a better collection than the Google offering)
Using a blog specific search engine.
Technorati at http://www.technorati.com/ is another highly thought of engine.
Feedster at http://www.feedster.com/ Feedster's search engine scans the Blogosphere constantly, providing a fresh look at the more than 30 million sources it tracks. Each day, Feedster adds millions of new content from existing feeds as well as finding tens of thousands of new sources.
Icerocket blog search at http://www.icerocket.com/
Daypop at http://www.daypop.com/.
NOTE: This looks like it might have died, as of November 2006.
Daypop
searches 35,000 news sites, weblogs and RSS feeds for current events and
breaking news. It crawls ‘the living web’ daily, and search options
exist to allow searchers to search News, Weblogs, both or RSS headlines. All
the usual search options are available – “phrase searching with
quotes” +including –excluding. You can also search for a specific
link to a URL with link:www.mysite.com or in the Advanced search function,
limit to a country or language. There are also options to check the top 40
links, top news stories, top posts, word bursts, news bursts, and top
weblogs.
Detod at http://blawgs.detod.com/
Specialised
engine for searching legal blogs. As well as a search facility it also lists
top stories (current to a few minutes).
BlogSphere news aggregator at http://www.alpern.org/weblog/php/blogsearch/writeup.html
Big long list from Ari Paparo at http://www.aripaparo.com/archive/000632.html though it may not be that current.
Fagan Finder Blogs and RSS search at http://www.faganfinder.com/blogs/ is probably a better bet.
Weblog directories
Another way of finding the right blog(s) for you. These directories work in the same way that Yahoo does – listing types, rather than being a search engine.
Library weblogs http://www.libdex.com/weblogs.html
This
site is primarily designed to list weblogs by, for or about librarians.
Globe of blogs http://www.globeofblogs.com/
This
lists blogs by location and topic. Quite small, with only 5,000+ listed.
Diarist http://www.diarist.net/
This tends to be
more for personal blogs.
Diaries and Journals http://www.worldimage.com/diaries/
Very
small collection of personal blogs.
Weblogs http://www.weblogs.com/
Huge collection,
though not well arranged or organised.
LiveJournal http://www.livejournal.com/
Arranged
by region, community or interest. Also has an option to start your own.
Acme Book News http://www.acmebook.com/
BookNews
http://futureofthebook.com/
Engineering
libraries – news for http://www.englib.info/
Eprintblog
(academic bias) http://www.bloglines.com/blog/eprintblog
Liblog
– library and technology oriented blog
http://www.rcpl.info/services/liblog.html
Research
Buzz – search engines and databases
http://www.researchbuzz.org/wp/
Scholarly
Electronic Publishing weblog http://info.lib.uh.edu/sepb/sepw.htm
Amphetadesk http://www.disobey.com/amphetadesk/
This
is a good tool; it’s fast, quick, effective and free. It’s a download
and sits on your desktop.
Abilon http://www.activerefresh.com/abilon.php
They
no longer offer a package, but they have a great listing
Bloglines http://www.bloglines.com
By far and away
the best IMO
Feedreader http://www.feedreader.com/
Download
desktop aggregator. I think it’s free, but it’s difficult to tell
from it’s home page!
Fyuze http://www.fyuze.com/
Online news
aggregator. Free, but you need to register.
Google RSS reader http://www.google.com/reader/view/
If
you like Google I suppose.
Newsgator http://www.newsgator.com/
An aggregator
that works with MS Outlook. I don’t use Outlook, so can comment no
further.
NewsMonster http://www.newsmonster.org/
Free
download. Works with websites and news sites and weblogs
Syndirella http://yole.ru/projects/syndirella/
Free
download. Another news aggregator.
Wildgrape News desk http://www.wildgrape.net/
Aggregates
rss feeds. Free, but requests donations.
Blog Easy http://www.blogeasy.com/
Simple, easy
and free way to quickly create your own blog.
Blogger http://www.blogger.com/
This is a free
tool, but there is a commercial version as well. Blogger can host your site, or
it can be configured to update on your own server.
EasyBlog http://www.elka.cz/easyblog/howto.htm
Download
the software onto your computer and update your blog onto your own server.
Electric Diary http://www.electricdiary.com/main.aspx
Emphasis
is on writers and creating communities, but everyone is welcome to create a
free blog.
LiveJournal http://www.livejournal.com/
Community
feel, easy and free to set up a blog. Can update it either on their site or
download a small utility to do it on your own computer.
Moveable Type http://www.movabletype.org/
Well
regarded software package that you download and create your blog from your own
computer. Free, but donations welcome.
Radio UserLand http://radio.userland.com/
Commercial
product and very well regarded by the blogging community.
Xanga http://www.xanga.com/
Cheap
(free!) and cheerful. Easy to set up and run. Allows for comments to
people’s weblogs (if they wish). The whole design is for a
‘community’ feel.
Blogging @ your library – feature article
http://www.sls.lib.il.us/reference/por/features/2003/blogging.html
Stephen
Cohen’s presentation on an introduction to RSS and blogging
http://www.librarystuff.net/presentations/neasist04282003_files/frame.htm
Guardian
article on working with newsreaders.
http://www.guardian.co.uk/online/story/0,3605,781838,00.html
Guardian
article on weblogs. Nice, concise, to the point and with a good listing of
weblogs.
http://www.guardian.co.uk/weblog/special/0,10627,744914,00.html
A
short history of weblogs
http://www.rebeccablood.net/essays/weblog_history.html
Matt
Maldre’s comparison of blog systems if you want to create your own.
http://www.spudart.org/etc/blogresearch/
RSS
info. Good collection of aggregators, some of which I’ve not mentioned.
Worth a look.http://blogspace.com/rss/readers
RSS
technical specifications that you probably don’t want to know about!
http://web.resource.org/rss/1.0/spec
Danny
Sullivan (Searchenginewatch) article on weblogs
http://www.searchenginewatch.com/sereport/article.php/2175281
There has recently been a rise in utilities that allow you to create and store your own searches and let other people make use of them. This is particularly useful if you are dealing with a group of clients who want to do their own searching, but are not sure which resources are the best ones to use. The following is a list of a few that you might want to try for yourself:
A good list of general search engines can be found at http://www.philb.com/webse.htm though this is neither complete or checked that often for accuracy, but you've a fair chance of finding what you're looking for.
BT
Enquiries
Electronic Yellow
Pages
FOUR11 (Yahoo! People
search)
Internet Address Finder
UK Phone numbers
Who's Who Online
Ariadne article:
Finding people
British Government
British
Search engines at http://www.philb.com/countryse.htm#uk
Google groups http://groups.google.com/
Jiscmail
Tile.net
Acronyms http://www.chemie.fu-berlin.de/cgi-bin/acronym
The
Bible http://www.biblegateway.com/
MediaUK
http://www.mediauk.com/
Library of
Congress http://lcweb.loc.gov/
Quotations
Lyrics.com
Picons
(icons/images)
http://www.tasi.ac.uk/resources/searchengines.html
http://search.yahoo.com/images?&ei=UTF-8&p= Yahoo now has over 1 billion images in its search directory
Download.com
Shareware.com
Tucows
The WWW Virtual Library
ADAM (art and design)
ALEX (catalogue of
electronic texts)
Biz/ed (business and
economics)
BUBL (everything!)
EELS (engineering electronic library)
HISTORY
WWEVL (waste water
engineering)
Pinakes -
excellent gateway to some major Virtual libraries.
Fazzle is a multi search
engine
Mooter is a graphical search
engine
Freesearch is a UK based
search engine
http://www.yousearched.com/ has been
designed for those with various impairments
http://www.ujiko.com/flash.php
http://www.ziggs.com/home.aspx Ziggs
is designed to help you identify professionals in different areas. US Biased
however.
© Phil Bradley, 2005, 2006.