Creating custom search boxes for library use- a second look

A couple of months back , I wrote a post entitled Creating custom search boxes for library use. This is one of my top 10 most popular blog posts and also one of the posts which I’m most proud of because it is one of my few posts that I feel is pretty original.

In that post, I figured out a way to create search widgets/boxes for practically any database, which can be embedded in many places including subject guides.

However since then, circumstances have conspired to make the post a little out-dated

Firstly the example given on Scopus, no longer works as Scopus changed their urls. That of course is easily fixed. Secondly, I figured out a slightly better way to improve the stability of the widget.

A more stable search widget?

You can read through the original post again on how to create a custom search-box for EconLit (via OVIDSP). At the risk of quoting myself
I have being a big fan of Opensearch plugins since I discovered them and I even created a big bunch of them here for almost every database we support on various platforms.

Once you have created a opensearch plugin, you know exactly what format the url should be sent to get the result. For instance, I know that to send a keyword query to EconLit (OvidSP) with the term TEST, you should send the following string.

http://gateway.ovid.com.libproxy1.nus.edu.sg/ovidweb.cgi?T=JS&NEWS=N&PAGE=titles&SEARCH=TEST.mp&D=econ

Creating custom search boxes for library use

The slight improvement to this would be to send the following string instead,

http://libproxy1.nus.edu.sg/login?url=http://gateway.ovid.com/ovidweb.cgi?T=JS&NEWS=N&PAGE=titles&SEARCH=TEST.mp&D=econ

Both methods should work, but I’m told that the later string would avoid caching problems.

So the rest follows as before and so the final code you should use is as follows

<script type=”text/javascript”>
function econlitSearchGo(){
var url=”
http://libproxy1.nus.edu.sg/login?url=http://gateway.ovid.com/ovidweb.cgi?T=JS&NEWS=N&PAGE=titles&SEARCH=“;
var url2=”.mp&D=econ
var searchInputeconlit = document.getElementById(“econlitSearchInput”);
window.open(url + encodeURIComponent(searchInputeconlit.value) + url2);
}</script>

<div>

<div id=”enterText” style=”position: absolute; left: -1000em; width: 20em;”>Enter your search terms:</div>
<input type=”text” id=”econlitSearchInput” size=”30″ onKeyPress=”handleKeyPress(event,this.form)” />
<input type=”button” value=”Search” onclick=”econlitSearchGo()”/>

</div>


nter your search terms:
ter your search terms:

If the javascript above looks tough to understand, refer to the original post again, it isn’t really that hard to understand and modify.

Conclusion

So there you have it, my improved custom search widget. If you know a bit more javascript you can do more fancy tricks, for example selecting 2 or more searches from a list and search them in different tabs comes to mind.

Using library 2.0 tools for technical services

Technical services in libraries which encompasses back-room work like acquisitions, document delivery and cataloging are often deemed as the less ‘sexy’ part of librarianship.

Yet, satisfying and even delighting library patrons depends critically on having an efficient and effective technical services departments which explains why many libraries are embarking on projects to streamline and improve their services. While the biggest gains probably come from radical redesign, some gains can be gotten from minor changes. In this post I will share some ideas to do that.

Librarians have being introducing tools like Libx toolbars, opensearch plugins, bookmarklets greasemonkey scripts (ezproxy script, amazon script) etc but the focus always seems to be for library users.

But it occurs to me that such tools are even more effective for technical services work.

Libx for instance automatically converts any isbn on a page to a clickable link, and clicking on it will do a xisbn search (it searches not just the given isbn, but also related isbns) of the item in your catalogue. While this is no doubt useful to users, one suspects that a typical library staff assigned to say acquisitions would use this dozens if not hundreds of times in the course of his/her work for checking orders that come in. Though each use saves them probably 5 seconds (not to mention reduces errors), the time savings can add up when they process thousands of books a year.

Another advantage of using opensearch plugins or Libx toolbar in technical services is that they can serve as a pilot, before you push it out to users!

Opensearch plugins

Obvious idea, add opensearch plugins for sites you use often for cataloguing, purchasing etc. This could be anything from your own library catalogue to other libraries like British Library, Book vendors, jobbers (Amazon, Blackwell books, Bookfinder, BookData Online etc)that do not support z39.50

You can also add sites that handle journal abbreviations such as JAbbr etc.

This is particularly effective if you need to do the same search a dozen times across different sites.

Say for instance, you need to process a request for document delivery. You need to

1. Check if the article exists in your collection and if not
2. Check your vendors to see if they have it to place an order.

Let’s assume you don’t have a openurl resolver and you need to manually check the source title against your OPAC.

Set up, the opensearch plugins for your library opac and other vendors/libraries you use in Internet explorer 7 or 8 or Firefox. (Tip : Add to searchbar Firefox addon makes adding new search providers a snap in Firefox)

Copy and paste, the source title (issn would be better, but most users don’t give you that), into the searchbox and then select your library search. After checking it doesn’t exist, you then click on the pull down button again and select the vendor you are checking (e.g. CISTI). The search results will appear.

If they don’t have it, continue with the other vendors down the list.

Notice how you don’t have to cut and paste the same search manually several times, just select another search provider and the search is run! Internet Explorer 7 is pretty handy in the way it handles opensearch because the search is automatically run with the terms in the searchbar whenever you change search providers.

You might notice that I’m actually using Firefox in the video. Firefox users who want to achieve the same effect should install “search on engine change” addon.

There are various other firefox addons that allow you to do even more interesting stuff, in particular you can do a search such that it searches the same term across several search providers opening a new tab for each.

Try Firefox search sidebar (search several search providers at one time) , searchwith (adding searches to your context menu) to search multi-servives at one time. This is a very cheap method to achieve a poor man’s federated search.

Install Libx

The Libx toolbar has a host of useful functions. But probably the most useful one of all would be the autolinking of isbns. Any page with an isbn is recognized and converted into a clickable link. Clicking on it will do a search of your library catalogue. It even searches related isbns (different publishers, different editions) if your opac supports that or lists related isbns in a sidebar that you can search with another click if your opac doesn’t.

Say you get a book order for a certain title. You typically search your catalogue first to see if the book already exists, then they search vendor sites to look for pricing, availability etc.

If you combine* the use of Libx and opensearch, you have two choices.

1. Use opensearch to search your catalogue (by title probably, isbn usually isn’t given but even if it is you probably want to check for hardback, paperback editions) then search the vendor the site. Then you can double check that the book doesn’t exist in your collection by clicking on the isbn given on the vendor site.

2. Use opensearch to search a commonly used vendor first , then click on the given isbn which brings you to the opac search of that isbn and related isbn.

#2 looks better to me. In particularly since it shows related isbn searches (american vs british publishers, paperback/hardback even different editions etc).

Using Googledocs for book orders

Did you know you can use Googledocs to create simple forms? The data will be automatically added to a spreadsheet. The Unquietlibrary uses this to create a simple form created this way for processing book orders.

How about using googledocs has a form for reporting problems with electronic resources that are down?

Using wikis to inform of cataloging subject heading changes

Rather than having a meeting to discuss the changes or additions, as we have in the past, one of the cataloguers suggested we add this information to the wiki. As each cataloguer has a chance to read the bulletin, they can add to the list of new, changed or old subject headings.

The cataloguing librarian

Other ideas

Incidentally, my love affair with all things library 2.0 began while I was embarking on a project to streamline technical services, so I have a soft spot for such ideas. Are there other interesting creative ways technical services staff are using web 2.0 in your library? I’m interested in hearing about it.

* Why not use Libx alone? While Libx allows you to add library catalogues, support of other searches are limited compared to opensearch plugins.

Does your library have a Firefox add-on collection?

Libraries that are big on Library 2.0 tend to offer a bunch of browser plugins/addons in the effort to reach out to the users who don’t feel the need to visit the Library Portal.

They offer custom toolbars, some prefer Libx, others are big on Conduit toolbars. Many progressive libraries are big fans of the firefox addon Zotero for citation management, many more offer opensearch plugins. All of these are available as Firefox add-ons of course.

Even Librarians who just use the standard add-ons, tend to have a list of add-ons that they can’t live without and love to recommend these to their patrons.

While submitting new Firefox addons to the official http://addons.mozilla.org web site isn’t a particularly new thing, there was no way to group all your favourite add-ons together and offer them in one place.

Until recently that is when Firefox revamped their website, allowing “developers” (all you need to do is register, no programming required!) to offer customized Firefox Add-on collections.

What libraries could do is to upload their unique firefox addons onto the Mozilla addon site and then bundle them together with other useful standard addons as a collection and offer them together to users.

As always I checked to see if any libraries had this idea and indeed some had (It’s hard to have a really original idea, librarians are really creative!).

As of writing these collections include the “Law Librarian recommended Add-ons” (University of Wisconsin-Madison) , “Swem Library“(Earl Gregg Swem Library), “Copenhagen Digital Library” , “Recommended for Library staff” (Ada Community Library) .

Of them all, the first is probably the most interesting and many of the ideas here is owed to that collection. They don’t have many subscribers yet though.

Managing the collection is quite simple particularly if one uses the Mozilla Firefox Add-on Collector , as you can add to your collection add-ons that are installed in your browswer(See feature list and video demos of creating a collection and setting up a collection that updates based on your installed addons).

Below is a screenshot of how you can select add-ons that have already being installed in your browser to be added to your custom collection.

Users can subscribe to a collection via RSS feed or better yet if they install the Mozilla Firefox Add-on Collector, they will be notified whenever the collection updates.

The last is a interesting feature, particularly if you are offering your own custom add-ons and constantly update them. Do note that add-ons you offer in the collection must be hosted on the Mozilla add-on site, so you will have to submit to them first.

So what can you add to your library collection? Some ideas

Opensearch plugins

I was looking at the Law Librarian recommended Add-ons and to my surprise I noticed that opensearch plugins (known as search engine add-ons in Firefox such as this) could be added to the collection as well.

If your library supports opensearch plugins for your library catalogue and subscribed databases (customized using the necessary ezproxy link), you can submit them to be added on the Firefox add-on site then add them to your collection. See this example

Custom toolbars and search related toolbars

Many libraries offer custom toolbars such as Libx, Conduit toolbars as well as other custom toolbars for download (see examples here, here ). Those can go into your collection.

How about a Book Burro toolbar? Or maybe OCLC’s Openurl referrer? Some libraries have add-ons that display availability of items listed on Amazon. You can also add ezproxy related addons

I’m playing with a pilot/experiment WebMynd add-on that includes library catalogue results alongside the default results whenever the user searches Google.com, Yahoo.com etc and that could be added as well.

There are also quite a lot of unique custom made for library add-ons being demoed at various “Library Labs” that could conceivably be added for those libraries.

Citation related addons

Zotero is the obvious choice here. Law Librarian recommended Add-ons collection also includes many interesting Zoterio plugins I was not aware of including Zotero Plugin for MONK Project , SEASR Analytics for Zotero , Zotz .

There are also sticky note/web annotation/scrapbook related add-ons like Diigo which I favour.

Others

There’s a ton of other addons you can add to your collection, that can help making research easier or are just so useful you can’t leave them out (e.g Adblock, Autopager), see for example this list.

Definitely add greasemonkey if you are offering greasemonkey scripts.

Conclusion

One thing I’m curious about is whether it would be possible or even legal to upload add-ons that are slightly customized. E.g A Zotero add-on
with specific options setup for your institution users (e.g. Openurl resolver settings set to the correct url).

Are there any more libraries creating Firefox Collections? What do you add? I’m interested to hear from you.


Creating web search widgets for any database – 3 different options

In an earlier post, i talked about how libraries are attempting to increase accessibility to their databases and catalogues via the use of opensearch plugins, custom toolbars, bookmarklets and even smart keywords.

Another way to improve accessibility is by embedding web widgets. You can embed all kinds of web widgets but here I will talk about web search widgets. Essentially these are javascript code for portable search boxes. I won’t sell you on how you can use this, but “but the general idea is to put it anywhere and everywhere your users may be

By cutting and pasting simple javascript, one can put the search box for catalogues and databases where-ever they want. Users no longer have to go to the official JSTOR page (for instance), just to search JSTOR. In fact, users can even cut and paste the same code and put it on their webpages to get the search box.

Vendors have begun promoting this. Proquest offers an easy to use search widget creator that allow librarians to easily generate javascript code to cut and paste to put where-ever they want on their portal (also CSA Illumnia) Ebscohost also allows this as well.

Note : Scopus takes a different tack and offers desktop widgets via Yahoo! widgets. But here I will focus on simple web widgets – simple javascript that one can cut and paste and put on any webpage.

But what if libraries want to create search web widgets for other databases that the vendors have not yet seen fit to support this? Currently, there are two approaches to handle this, in this post I will introduce a third method that AFAIK has not being mentioned before.

To get you motivated here’s one example of what you could do. I embed them into my subject guides


Approach one : Leveraging on database searches already supported by Conduit toolbar.

The highly innovative and creative librarian from Netherlands Guus van den Brekel is a big fan of Conduit toolbars. As I mentioned in a previous post this is a custom toolbar that can be configured to support quick access to search almost any database.

He realised that once a toolbar is setup to support searching say JSTOR, you can easily convert it to a web search widget. The instructions on how to do so are here.

My main concern with this method are as follows

1) The search doesn’t go directly to the database but is routed through Conduit servers, which might lead to privacy concerns.

2) The results shown will be framed. While this is an advantage for users who can quickly switch being databases. Some vendors might frown on sites framing their material.


Approach Two : Leveraging on existing federated search solution

Very recently at the Computers in Libraries 2009 conference, Nina McHale of Auraria Library, gave an amazing presentation entitled “Help your Library become omnipresent without spending a dime” (slides) , talking about what they have done with widgets. Definitely worth reading if you haven’t. But her presentation doesn’t go much into the technical details except to advise to check with your “web folk”. Looking at examples given here and in the slides, one notices/guesses that they are leveraging on their federated search solution from serial solutions? (I might be wrong)

If so I can see two disadvantages with this approach. Not every library will have a federated search solution. More seriously, I believe most federated search vendors charge you for each database you add, so this solution will restrict you to creating search widgets for only these few databases you are supporting.

Is there a better approach?

I have being a big fan of Opensearch plugins since I discovered them and I even created a big bunch of them here for almost every database we support on various platforms.

Once you have created a opensearch plugin, you know exactly what format the url should be sent to get the result. For instance, I know that to send a keyword query to EconLit (OvidSP) with the term TEST, you should send the following string.

http://gateway.ovid.com.libproxy1.nus.edu.sg/ovidweb.cgi?T=JS&NEWS=N&PAGE=titles&SEARCH=TEST.mp&D=econ

How do I know that? For every opensearch plugin, you will have a string in the xml file that says something like the following

<Url type=”text/html” template=”http://gateway.ovid.com.libproxy1.nus.edu.sg/ovidweb.cgi?T=JS&amp;NEWS=N&amp;PAGE=titles&amp;SEARCH={searchTerms}.mp&amp;D=econ“/>

where {searchTerms} represents the search term that you type in.

Would it be possible to leverage on this knowledge to send the search directly from a web widget? I was pretty sure this was possible, but my javascript knowledge is practically zero, but looking at the sample web search widget produced by the Proquest Search Widget allowed me to figure it out.

Creating a simple web search widget by copying and pasting

After tweaking it a bit, this is what I came up with by modifying the Proquest widget version. It can probably be improved but so far it works fine.

<script type=”text/javascript”>
function econlitSearchGo(){
var url=”http://gateway.ovid.com.libproxy1.nus.edu.sg/ovidweb.cgi?T=JS&amp;NEWS=N&amp;PAGE=titles&amp;SEARCH=“;
var url2=”.mp&amp;D=econ
var searchInputeconlit = document.getElementById(“econlitSearchInput”);
window.open(url + encodeURIComponent(searchInputeconlit.value) + url2);
}</script>

<div>

<div id=”enterText” style=”position: absolute; left: -1000em; width: 20em;”>Enter your search terms:</div>
<input type=”text” id=”econlitSearchInput” size=”30″ onKeyPress=”handleKeyPress(event,this.form)” />
<input type=”button” value=”Search” onclick=”econlitSearchGo()”/>

</div>

Don’t panic!

Okay if you are like me and am not a javascript god, this looks quite scary.
But it isn’t really. Look at the line that says

window.open(url + encodeURIComponent(searchInputeconlit.value) + url2);

It just means open in your browser a new window with a url made up of whatever is in

1) the variable url = http://gateway.ovid.com.libproxy1.nus.edu.sg/ovidweb.cgi?T=JS&NEWS=N&PAGE=titles&SEARCH=

followed by

2) the search value entered into the box = For instance say TEST is entered in the search box.

followed by

3) the variable url2 =.mp&D=econ
As such on typing TEST and clicking on the search button, the following url will be opened

http://gateway.ovid.com.libproxy1.nus.edu.sg/ovidweb.cgi?T=JS&NEWS=N&PAGE=titles&SEARCH=TEST.mp&D=econ

and this of course is exactly what you want to type in the browser url address bar!

One more example

Here’s one more example, you created a Scopus opensearch plugin. You look into the xml file and you see this

<url type=”text/html” method=”GET” template=”http://www.scopus.com.libproxy1.nus.edu.sg/scopus/search/submit/quick.url?searchtext={searchTerms}&amp;origin=Browse&amp;javascript=t”>

So you know the string you need to send is
http://www.scopus.com.libproxy1.nus.edu.sg/scopus/search/submit/quick.url?searchtext={searchTerms}&origin=Browse&javascript=t

So what you do is to just copy and paste the same javascript as above, except to replace with the appropriate values

<script type=”text/javascript”>
function ScopusSearchGo(){
var url=”http://www.scopus.com.libproxy1.nus.edu.sg/scopus/search/submit/quick.url?searchtext=“;
var url2=”&amp;origin=Browse&amp;javascript=t
var searchInputscopus = document.getElementById(“ScopusSearchInput”);
window.open(url + encodeURIComponent(searchInputscopus.value) + url2);
}</script>

<div>

<div id=”enterText” style=”position: absolute; left: -1000em; width: 20em;”>Enter your search terms:</div>
<input type=”text” id=”ScopusSearchInput” size=”30″ onKeyPress=”handleKeyPress(event,this.form)” />
<input type=”button” value=”Search Scopus” onclick=”ScopusSearchGo()”/>

</div>

Easy as pie. ūüôā

Some notes

1. If the search string ends with just {searchTerms} and there is nothing after it, you can just leave the line that specifies url2 as follows

var url2=””

2. It is not strictly necessary to change the variable name e.g. searchInputscopus instead of searchInputeconlit, if there is one widget per page, but this may cause problems if widgets sharing the same name are on the same page.

3. Note to beginners the examples above incorpoate the ezproxy string that you have to change to your institution.

Advantages

1. Compared to the above two mentioned methods, search goes directly to database.

2. Relatively easy to create search widget once you have the opensearch plugin.

3. Cost nothing. No requirements at all, except client side javascript.

Disadvantage

1. It is difficult to instruct users on how to create their own search widget for the database they want. It is unlikely the user would want to go through the method mentioned above. One advantage of approach two above (using serial solutions federated search) is that you can use easily understood codes like db=jst to represent JSTOR, making it easy for users to change to whatever search they wanted.

One possible improvement would be to to implement a gatekeeper page, as in “Plug your users into libraries resources with OpenSearch Plug-ins ” . Instead of the opensearch string going direct to the database it will go to a special gatekeeper php page that will redirect the search. That way, one could create nice aliases to use, it would go the php page which would then redirect to the correct url. Another advantage is that the gatekeeper page allows one to keep statistics of how often the searchplugin is used, and if the url string needs to be changed, all you need to do is to change the php string and all users will automatically be updated.

Conclusion

So what do you think? Is this a good idea?

The next step would be I think to figure out if this can be extended so that it can be used to create desktop widgets, put on startpages like netvibes, pageflakes, put on social networking sites like Facebook etc by putting it through Widgetbox etc



Top 10 opensearch searchplugins used at the Information Desk

Introduction

I have being heavily using opensearch searchplugins to help speed up access to common used searches at the reference desk. I take it for granted, most people already have Google, Wikipedia, Youtube, Amazon etc. But what else should you add?

As WordPress strips out all javascript, I am unable to provide a direct clickable link to install the opensearch plugins mentioned below. Sorry about that. This page lists them all and more.

1. SearchPlugin to my Institution’s OPAC

This one is obvious.¬† I have searchplugin’s to our new Encore system which does a keyword search , and another to do a search by Call number.¬† The latter is necessary because Encore still doesn’t do call number searches yet and a lot number of our patrons like to give me call numbers to verify.

Of course, you could always keep a tab open on your OPAC search page, but being able to interrupt your work and instantly look up something without hunting for a tab is good.

2. SearchPlugin to Google Scholar and Google Books

In earlier posts, I have explained I’m a big fan of Google scholar and Google books. Naturally I have searchplugins for them. Use Google scholar to verify citations, hunt for obscure terms when your other sources don’t find much and a lot more. Google books. is great for verifying contents of books (those with limited preview or full view). Strangely I find myself doing it more often for books that we own. That sounds a bit odd, but when you can’t leave the desk to verify if a certain book really has the chapter the patron needs (and the catalogue doesn’t have the tables of contents), Google books is a god sent.¬† Also great for remote reference to help patrons decide if they really want to come down physically to borrow the book.

We don’t yet have a openurl resolver up yet, so when clicking on¬† Google scholar results sends me to the vendor page, I log-in directly with a bookmarklet that adds our ezproxy string to the url.

I’ve considered adding other members of the Google family such as Google Groups (great for tech support), or Google news, but I seldom use them at my reference desk.

3. SearchPlugin for directions

Want to know the way to the Zoo from the library? To the airport? I’m the last person you should ask, since I was born without a sense of direction. To solve this problem, I have a searchplugin from gothere.sg that allows you to key in any postal code or road name or building in Singapore, and it will automatically work out the best route to go there from the library, whether by driving, or by public transport (Bus and MRT). This can then be easily printed. You can also flip it around with a click so it goes from any location to the Library.

There are many similar services here in Singapore, but they either don’t provide directions for public transport (e.g. Google Maps, Red nano) or they allow you to only search by Street or Postal or building name and not all three together in one search. (e.g. Streetdirectory.com)

4. SearchPlugin for Google site search of my institution

Another no-brainer.¬† Often you need to just search for things on your institution’s domain. If a custom google site search¬† for the domain exists, create a searchplugin for it.¬† or if it does not exist and you don’t want to create one you can create a normal Google search plugin, but remember to insert site:www.institutiondomain.edu before creating.

If your institution has a specialized intranet search engine (e.g google search appliance), you might want to create a searchplugin for that too.¬† In the same category is a searchplugin for searching your institution’s staff directory.

5. SearchPlugin for newspaper searches

Patrons tend to like to ask questions that require searching of Straits Times, our national newspaper. Since I haven’t yet figured out a way to get the plugin working for Factiva, I created one for Lexis-Nexis Academic which searches our local newspapers. But what about older archived issues not covered? Recently our National Library scanned all back issues of Straits Time (up to 1982)¬† at newspapers.nl.sg I created a search plugin for that,¬† it shows only abstracts, but that is good enough to verify for users whether they want to go to our microflim collection to get the full text.

6. SearchPlugin for Thesis

Another common query is for thesis.  I generally add PQDT. Some others I occasionally use also are  NDLTD, Hong Kong University Thesis etc.

7. SearchPlugin for Open Access archives

They are several you could add.  Google scholar already covers quite a bit, but I could envision adding Citeseerx, Arxiv, Biomed central etc depending on your needs. Others you could add include Open J Gate, DOAJ, OAIster to search by article. etc.

8. SearchPlugin for digitalized book collections

Try adding the free Open content alliance (via Internet Archive), Universal Digital Library, China-US Million book project. If your institution has subscriptions you could add Springer Books, NetLibrary etc.

9. SearchPlugin for scholarly search engines.

Try adding some high quality web search engines, Federated search engines that index academic material only such as Scirus,  Mednar, Biznar, Science.gov etc.

10. SearchPlugin for anything else

Be creative, add book vendors like Bookdata online, Worldcat etc. Or Social sites/research 2.0 sites  like  Librarything, Bookjetty, Citeulike,  Slideshare etc.

I personally also add Scopus, EconLit, Blackwell-Wiley, Project Muse, for specific reasons to do with the interaction between Google scholar and the proxy bookmarklet that I won’t go into now.

Conclusion

Yes, I have a big bunch of searchplugins in my browser. In fact I have 2 lists one for IE7 , one for Firefox. The later I use at the reference desk, the former I use when I’m in the back-room doing cataloguing, acquisitions etc with the appropriate search-plugins. But that is another post.

So dear librarians, what search tricks do you use (need not be opensearch plugins, could be smart keywords) do you commonly use at the desk?

Opensearch vs custom toolbar vs smart keyword vs bookmarklet (II)

In a previous post, I discussed the different ways, one can add support to searching OPAC and other library subscribed databases. The four methods were Opensearch plugins , custom toolbars (Conduit toolbar , Google toolbar and Libx) , Smart keyword searches and Search bookmarklets

I will begin with Opensearch.

Opensearch

This is by far the most popular method. Many libraries have used them (see this) . This became possible only when browsers began to include a built in search bar. The idea was simple, instead of just hardcoding search engines in the searchbar why not allow users to add their own?

For a time sherlock plugins (which worked like opensearch plugins) were popular with the Firefox/Firebird/Pheonix crowd but it was strictly meant for power users because of the lack of support for Internet Explorer.

When Internet explorer 7 and Firefox 2.0 both began supporting Opensearch in Oct 2006, interest in Opensearch plugins exploded.

The future of opensearch seems secure as IE 8 adds various features like web accelerators and newer versions of opensearch (don’t worry your existing ones will continue to work) that further enhance use of opensearch plugins.

Various powerful firefox extensions exist that further enhance use of opensearch plugins.

Try Add to searchbar (makes adding new search providers a snap) , Firefox search sidebar (search several search providers at one time) , search on engine change (becomes IE7 like, switching engines will automatically search),  searchwith (adding searches to your context menu) and much more.

Creating and testing

I won’t go into how to create basic ones because there are many guides available (e.g this guide) as well as opensearch plugin builders (mycroft, enhanceie ,Microsoft’s etc)¬† where all you need to do is to follow basic instructions without programming.

I do recommend the following testing scheme

1. Create the opensearch plugin. Include favicon if required (I prefer to link it locally).

2. Clear your browser cache, cookies etc, log-out, restart your browser and test the opensearch plugin again.

3. Wait an hour or so, and retry the opensearch plugin again.

Step 3 is recommended, because often, the url string includes a sessionid, and the url you send will continue to work for a period until the server starts to reject it. With experience you can spot it quickly (usually long random string), but until you can, including #3 in your testing is a good idea.

Also be aware of the slight differences in the way IE 7, IE 8 and Firefox handle opensearch plugins.

Difficulties

I have personally tested this method more than the other methods, and I find that I can create working opensearch plugins for roughly 90-95%% of our subscribed database including our OPAC.

Many databases are harder to create for because they use post methods (which do not work in IE7/8 because Microsoft in their infinite wisdom refuse to support them, though Opensearch does provide for them and Firefox supports them) or are doing some quirkly refreshing of screens so it is hard to capture the actual url.

Workarounds I know of to solve problematic databases

1. frmget bookmarklet

Use this to convert the form to use GET method instead of POST method. Though many forms are by default using POSTmethod, they work fine when converted to Get method. To do this, Click on the frmget bookmarklet before capturing the url as per normal.

2. Use Needlesearch (a Firefox addon)

Some databases, do some quirkly refreshing of screen, and the final url you see is not the one you need. There are many methods to capture the actual url you need but I use Needlesearch (a Firefox addon)

3. Find a direct Url

Often steps #1 or #2  (or even #1 and #2 combined) will cut through any difficulties. But some will require special handling beyond that.

Many databases add a sessionid key that causes problems when you try to create a opensearch plugin using the normal method.¬† Fortunately, they also provide a directurl, or jumpstart url you can use. I will list the ones I’m aware of. Most came from the excellent blog, Library toolbars

  • OvidSp – use jumpstart url
  • Web of science – use this url string for normal searches or citied reference searches.
  • Lexis- See this
  • Ebscohost – Probably easiest way is to do a search, then click on “alert/Save/share” and copy the Permalink (see below).

That said, the some databases still defeat me, Factiva and IEEE Xplorer are two examples. Anyone managed them?

Pros and Cons

Pros :

1. Well supported

There are many sites offering lists of opensearch plugins for use (few library related ones though), and lots of well designed mechanism for easily creating plugins.

Use any one of the following web forms (mycroft, enhanceie ,Microsoft’s etc) or you could create your own from the scratch (basic idea).

2. Supports both IE 7+ and Firefox as a built-in feature in.

This means it does not require user to install large clunky toolbars, which can cause technical support problems.

For example some security programs like anti-spyware programs don’t like Conduit due to privacy concerns, firewalls might block toolbars from “phoning home” etc.

Another minor concern is that IE 7+ and Firefox already have searchbars, so adding third party toolbars just makes the browser cluttered. (Yes, I know you can remove the default searchbars in Firefox and Internet explorer..)

3. Greater ease of use.

Use of  Search bookmarklets gets  a bit difficult once you have more than a few. Smart keyword searches are unsuitable for all but powerusers used to remembering commands

Cons

1. Firefox version does not support Searches using post method.

I had to use workarounds (see above)

2. Does not support IE 6 and older versions, much less Safari, Opera.

Expect lots of complains from Mac users who love Safari even though Firefox works fine.

There’s a lot more to say about opensearch which I predict will become huge (e.g Microsoft support in Windows 7 , using program launcher like¬†FARR to search ) but next up are the custom toolbars. Conduit toolbar , Google toolbar and Libx

Opensearch vs custom toolbar vs smart keyword vs bookmarklet (I)

Introduction

How can libraries encourage the “google generation”¬† to use anything else besides Google?¬† I have no clue either but for sure increasing accessibility to such searches is a key.

Think about it. How many users do you think know the library portal url straight off? How many bookmark it? In my experience precious few (most in my institution seem to know the university homepage url and access the library portal from a link there). Compare this to the number of people who do not know the url for google (KISS is a good idea!). Worse yet the availability of Google toolbar, default google searches in browser toolbars makes accessing google even easier.

Accessibility problems

It is difficult enough to convince users that they might like to try say Scopus instead of Google. Even if they were convinced to try Scopus, would users be hardworking enough to

  1. Go to our library portal (assuming they can remember it)
  2. Search for the database they want on the portal (assuming they can find it among the hundreds we provide)
  3. Authenticate (assuming they didn’t forget the password)
  4. Wait for the search page to load, enter the search terms (assuming they don’t struggle with the unfamiliar interface with tons of options)
  5. Enter the search term and finally wait for the results

We can’t do anything about #3, but there are methods to mitigate #1, #2, #4 and #5 , and as usual the commerical world has gotten there before us.

One method would be to provide Opensearch plugins

Others include providing custom toolbars ( e.g. Conduit toolbars, Google toolbar (details). Libx and others), Smart keyword searches and Search bookmarklets.

What these methods share in common is a way to search your favourite database or search engine with any arbitary search term, from whatever page you are on, without needing to go to the search page first. This solves #1 and #2.

They also share in common the following feature, the search they carry out goes directly to the search results page (using the search options defined earlier) bypassing the search form.

For users who just mostly use the default search at first anyway, it is a waste of time to display the search web form for them to set options. They just get confused anyway. This solves #4 and #5

Oh sure, there are times this is a disadvantage, because you want to set special search options, turn on limits etc, but chances are 9 out of 10, for a quick dirty search the general basic default search would be sufficient, and if it isn’t you can easily refine your search later right?

Comparison of methods

I know for many of the readers the use of opensearch, Libx/conduit toolbars is old hat. However, I hope in my future posts about my experiences using them will be of interest.

Browsers supported? POST Method supported? Popularity
Opensearch IE7+, Firefox 2.0+, Chrome**** Firefox only High
Conduit toolbar IE6+, Firefox Yes Low
Google toolbar IE 6+, Firefox Yes High
Libx IE 6+, Firefox No?** Low
Smart keyword Firefox only* Yes Low
Search bookmarklets In theory all browsers Yes*** Low

* Opera has a similar feature, IE supports this if
configured using the free TweakUI

** Post method only works for some “custom catalogues”.

*** Possible but complicated but see this

**** Limited support, your offered opensearch plugins will create a smart keyword. Also see this.

Browsers supported

Obviously, it is important for the method chosen to support as many browsers as possible. In this area the custom toolbars , Google toolbar, Conduit toolbars and Libx, support the two major  browsers Internet explorer and Firefox.

Custom toolbars add an additional searchbar, unlike Opensearch plugins which utilize the already existing searchbar in IE7 and Firefox. As a result, opensearch plugins are not usable with IE 6 and older browsers as they do not come with a searchbar. This can be an issue if many systems are still on IE 6. On the plus side the new rising browser Chrome by Google is supported! (limited though, your offered opensearch plugins will create a smart keyword. Also see this).

Smart Keywords are perhaps most limited, supporting mainly Firefox, though it is possible to support it using the free TweakUI.

So far, Mac users who insist on using Safari, are out of luck. However, it is potentially possible for the same search bookmarklet to work even all browsers even those as different as IE and Safari.

Post method supported?

Search engines can use two different methods to send queries. One is the “GET” method and the other the “Post” Method. (see details) . I have found that while a vast majority of opacs and library subscribed databases use the “GET” method many don’t (i.e. Pubmed, Lexis etc). Unfortunately, not all methods support the POST method, and in the case of Opensearch in though there is a provision for it, Microsoft in their infinite wisdom decided not to support it in IE 7/8 although Firefox does!

There are work arounds though, certain search engines actually work if you convert it to work with GET, you can do this using the frmget bookmarklet first, before trying to create the opensearch plugin.

In future posts I will share further thoughts on each of the methods, their strengths, weakness, the problems I faced creating searches for subscribed databases.