OpenSearch for Wikka
What is OpenSearch?
At its simplest, OpenSearch is an open specification that allows search engines to publish what the (HTTP) interface to their engine looks like, and search clients to use that specification to provide a compatible (user) interface to that search engine and retrieve the results. Or, According to the OpenSearch.org official website:
OpenSearch is a collection of simple formats for the sharing of search results.
Search engines may also use the specification to tell what the license for their search results is: an open license would allow search clients to create a single interface to multiple open-license search engines, much like today's "meta" search engines.
All the search engine needs to do is publish a description file in XML format that specifies its interface; and to enable autodiscovery it can link to this description file via a <link> tag in the <head> section of its website. Any application that supports OpenSearch can thus find this description file, and use it to create an interface to using that search engine without requiring the end user to go to the search engine's website.
A search client application can be a desktop application (such as a browser: both Internet Explorer 7+ and Firefox 2+ 1) support it) but it can also be a web-based application. An application needs to download the XML description file, and interpret it to provide a search interface for the search engine it belongs to.
For a more detailed description, see OpenSearch.
Who uses OpenSearch?
Even though the OpenSearch specification is officially still a draft (but quite stable), it is already widely supported.Search engines
Did you notice how two in that list are applications that the Wikka Wiki project itself is using? We use WordPress for our blog, and Trac for our issue tracking. And did you notice there's a wiki site in that list? Wikipedia is a major one (Mediawiki has a plugin for that), but not the only one. For more see OpenSearch software.
Many major search engines already support OpenSearch: Amazon, of course, since they are the inventor, but also Yahoo!, Google, and may more. But any website or web publishing software that provides its own search function can be a search provider for OpenSearch as well. Notable examples are WikiPedia (in all languages), WordPress (via a plugin), Trac, Youtube, and Mozilla Add-ons.More OpenSearch providers can be found here (just a start):
- OpenSearch search engine directories (a list of lists)
- Tredosoft's list of search providers (don't be confused by the reference to IE7: it works equally well with Firefox!)
- Firefox Search Tools add-ons (although Firefox calls it "plugins", if you download from a link, what you get is the actual XML description file for the search engine: a nice way to study description files!)
Search clients
As I've already hinted, several browsers have built-in support for OpenSearch and autodiscovery, allowing you to add extra search engines to the list of engines you can use:- Firefox 3 (see the footnotes 1)); there is also a plugin that works with Firefox 1.0-3.0 as well as Seamonkey
- Internet Explorer 7 (and later - see the footnotes 2))
- Camino (since version 1.6)
- it also seems that the Google Chrome browser (just in beta) also supports it, (it will actually automatically add a search engine once you've used it - which provides the user with rather less control than the other browsers do; it's possible it uses autodiscovery for this, without explicitly mentioning it)
Enter Wikka
As you may have gathered by now, although the OpenSearch specification allows many variations, if you have a website that provides search functionality, it is actually very easy to enable that for OpenSearch. While the usual format of an OpenSearch result is a feed (Atom or RSS), the specification also provides for search results in the form of HTML pages - and that's exactly what Wikka already does in its search functionality. This means that any Wikka installation can be an OpenSearch search provider - all it takes is two steps:- Create and upload an XML description file
- Enable autodiscovery with two simple additions to the <head> section
Ready? Let's go!
Create a description file
You can read the details of the syntax of the description file here:
First, a little preparation:
- This is possibly the hardest part of the exercise: You need to decide how your search function(s) will be known to the world, and you have only 16 or fewer characters of plain text available. That "ShortName" is what your search will be known by in the dropdown search menu of a supporting browser (unless you also provide a LongName, so make sure it is distinctive. If you want to support both the plain text search and the expanded text search you can, but your choice of "ShortName" becomes even harder!
- You also need a description of your search engine: up to 1024 plaintext characters - no markup allowed!
- While the ShortName element is required, you can also provide an optional LongName, for which you have 48 or fewer characters of plain text available. It seems that Firefox will use the ShortName, even if a LongName is provided, in the search dropdown (I don't know how IE7 is handling this since I don't have access to it.)
- Optional: A Contact email; you could use your admin email address, but remember that it will be visible in cleartext. If you are not comfortable with that, create a special throw-away address for the purpose, or just do not provide a contact
- An icon would be nice, although it is not strictly necessary. Two formats are recommended (you can provide both):
- A 16x16 small icon - this can be type "image/x-icon" or "image/vnd.microsoft.icon" (the Microsoft ICON format);
If you already have a favicon for your site, you can simply use that, or you can create a special "search" variant if you like; if you create a special one, just upload it to you images directory.
- A 64x64 image of type "image/jpeg" or "image/png"
- The default "input format" for OpenSearch is UTF-8; if you've set up your site to use UTF-8, you won't need to specify anything, but the default for Wikka is ISO-8859-1, which you would need to specify, as well as any other encoding.
- Finally, decide on a location for your XML description file. In a short discussion with DarTar on the #wikka IRC channel, we agreed that libs/opensearch would be a good location: while it is not a library in the sense of a set of routines in a programming language, you can provide a library if search description files here.
- For the autodiscovery you should provide a description in the title attribute, that describes what an end user would be adding to their browser. Although this isn't part of the specification, the content of this title attribute should actually match the ShortName: browser implementations of autodiscovery will compare the text of the title attribute to the ShortName of already-installed search engines: if there is no match, the browser will still offer to add it, even if it has been added already. (That is because it cannot be detected until the description file has been loaded - for autodiscovery a browser only looks at the <link> tag itself.)
After that preparation, you're ready to create your XML description file. First, grab the following template:
search description template
- <?xml version="1.0" encoding="UTF-8"?>
- <OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
- <ShortName>[[your | ShortName]]</ShortName>
- <LongName>[[**your | LongName]]</LongName>
- <Description>[[a | description, up to 1024 characters of plain text, no markup]]</Description>
- <Contact>[[**your | Contact email]]</Contact>
- <Url type="text/html"
- template="[[your | base_url]]TextSearch[[separator ?|&]]phrase={searchTerms}"/>
- <Image height="64" width="64" type="image/png">[[**path to a 64x64 image</Image>
- <Image height="16" width="16" type="image/vnd.microsoft.icon">[[**path | to a 16x16 icon]]</Image>
- <Query role="example" searchTerms="page" />
- <SyndicationRight>open</SyndicationRight>
- <Language>en-us</Language>
- <InputEncoding>[[**encoding | if not UTF-8]]</InputEncoding>
- <OutputEncoding>[[**encoding | if not UTF-8]]</OutputEncoding>
- </OpenSearchDescription>
This is conformant with the latest version of the specification, the OpenSearch 1.1 current draft.
Now you can edit it; the bits you need to fill in are enclosed in [[...]]. If it starts with **, it is optional: leave out the whole tag if you do not provide a value. The elements you absolutely must provide are ShortName, Description and Url; and of course the OpenSearchDescription root element. ShortName and Description are described above; Url is the search URL "template" the browser will use to construct a search from the user's input. Let's go through it line by line (elements are linked to the official description):
- <?xml... ?> is the processing instruction: required
- OpenSearchDescription required: the root element
- ShortName required: fill in your short name here
- LongName optional: if you don't have a long name, leave out this whole line
- Description required: fill in your search engine description here
- Contact optional: your contact email, if you provide one; otherwise leave out the whole line
- Url required: this is the template the search client will use to search your site. Fill in your base URL (the base_url value from your wikka.config.php file. The next bit is "TextSearch" for the basic search; if you want to provide an interface to the expanded search instead, use "TextSearchExpanded". Then a separator: ? if you use URL rewriting (rewrite_mode 1 in wikka.config.php), & if you do not
- Image optional: you can have zero, one or two of these; fill in the (relative or absolute) path to the image(s) you prepared, and make sure the type attribute matches your image (see the type descriptions above)
- Query optional: this can provide a search client with a way to test your search engine; "page" is a good search term to use as it will provide a result even on a fresh Wikka install
- SyndicationRight optional: possible values are open|limited|private|closed; the default is "open". You'll probably want to use "open" (in which case you can leave out the whole line), but see the documentation for the precise semantics
- Language optional: one element for each language that your search engine supports (in RFC 3066 format); the default is "*" for "any language"
- InputEncoding optional: must match the character set your site is encoded in (ISO-8859-1 for a standard Wikka install); default is UTF-8
- OutputEncoding optional: must match the character set your site is encoded in (ISO-8859-1 for a standard Wikka install); default is UTF-8
That's it - it's easier than it looks: just go through the steps carefully, you can't really go wrong. Name the file after the type of search it describes, like TextSearch.xml or TextSearchExpanded.xml (you can have both).
A minimal description file, with only the required tags, would look like this:
minimal description template
- <?xml version="1.0" encoding="UTF-8"?>
- <OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
- <ShortName>[[your | ShortName]]</ShortName>
- <Description>[[a | description, up to 1024 characters of plain text, no markup]]</Description>
- <Url type="text/html"
- template="[[your | base_url]]TextSearch[[separator ?|&]]phrase={searchTerms}"/>
- </OpenSearchDescription>
Make sure you understand what the defaults are though - for most Wikka installs you will at least need to override the default UTF-8 for InputEncoding and OutputEncoding.
To install these file(s) in your Wikka installation you need to:
- create a opensearch directory under the libs directory in your installation path (if you run an old version of Wikka and don't have a libs directory, simply create that first);
- upload your XML description file(s) into [yourpath]/libs/opensearch.
- copy the .htaccess file you find in the images or css directory to your opensearch directory (without this, a browser won't be able to download the XML file!)
Provide for autodiscovery
For this, you'll need to edit the file header.php. Depending on the version of your Wikka installation, it's either in the templates directory (version 1.1.6.4 or later), or the actions directory (earlier versions). The process is the same, regardless of where it is located.Open (a local copy of) the file in a text editor, and make the following changes:
- Find the section where the start of the HTML is produced (in 1.1.6.5 this starts on line 7); two lines below that is the <head> element; change it from
to
- <head>
- <head profile="http://a9.com/-/spec/opensearch/1.1/">
- For each of your XML description files, insert a link tag that indicates its location, for instance just before the link tags for the favicon; the href attribute must point to the location of your XML file. You can use variables, too. For instance like this for (plain) TextSearch:
- <link rel="search" type="application/opensearchdescription+xml" href="<?php echo $site_base;?>libs/opensearch/TextSearch.xml" title="<?php echo $this->GetConfigValue('wakka_name');?> Text Search"/>
Make sure that your title attribute matches the ShortName in your description file. Although the OpenSearch specification states only "The "title" attribute may contain a human-readable plain text string describing the search engine", in practice the strings should match so as not to confuse browsers (they compare the title attribute with the ShortName of already-installed search engines to decide whether of not to offer to install a new one).
If you have done this locally, upload your modified header.php to your server again.
Over to the browser
This section is still a stub - expect some screenshots here later.Now, if you go to your wiki with a browser like Firefox, you'll note that the dropdown button for the search box get a blue "glow" around its down arrow. Click it, and you'll see an option to add your own search engine(s) to the list. All you need to do is click that menu item, and from then on you can search your site directly from the browser, without going to the site first. And any of your visitors can do the same.
Footnotes
- Actually, Firefox 2 did not really support the OpenSearch specification, but a proprietary variant.
- While that document suggests that "the only difference is the root element and XML namespace" that is certainly not the case in comparison with the current OpenSearch 1.1 draft version. Also, Creating OpenSearch plugins for Firefox suggests that the Param elements they use are an OpenSearch extension, but the document they link to actually describes a Parameter element as an extension: there is no Param element in OpenSearch.
- An extra complication is that adding these Mozilla-specific elements, attributes and namespace to an OpenSearch description (as illustrated on WikkaOpenSearch and the Mozilla documentation) will make it no longer work with Internet Explorer: the document is no longer conformant (and no longer "open")!
- Meanwhile, Firefox 3 does support the actual OpenSearch specification, and the definition file (template) provided on this page works without a hitch in Firefox 3.
- The upshot is that if you wanted to provide backwards compatibility with Firefox 2, you'd have to provide a compliant OpenSearch description file as well as a Firefox proprietary (i.e., not-open) description file - and then indicate the difference in the only 16 characters you have for your ShortName and autodiscovery title attribute...
- Since I don't have access to Internet Explorer 7, I'm hoping for someone to test the instructions in this page (or a proof of concept implementation of it), and report the result here!
