Yesterday and again this morning, discussions on the #koha irc channel turned to searching and a decision was made to develop a "google-like query syntax" (GQS) for searches. In general, we want to develop a much better search back-end that: - can optimize queries - is accessible to multiple front-ends (Z39.50 server, Catalog Searches, Circ/Patron searches, reports, etc.) - handles security better than the current model - 'lowers the bar' for developers who want to work on localized improvements for searches - easily spans the range from very simple to very complex queries We're looking at two passes. The first pass would parse incoming search requests and compile GQS queries. This pass would be performed by the SearchParser::Foo family of modules (e.g., SearchParser::Basic, SearcParser::Z3950, etc.). The second pass would convert the GQS into SQL, optimize the SQL query for the Koha DB structure, and handle the returning of data to the application. We intend to create these new modules following the Perl OO style to provide the easiest possible access to developers. I volunteered to look up the google search syntax and make a first cut at how we might use it for our query syntax. Would someone be willing to look at Z39.50 searches to see what additions they' d need? (Joshua?) Are there other kinds of searches that will need special terms or operators? If so, what are they? We still need to deal with constraining searches/queries to acceptable kinds of queries against specific tables and fields. We can work on hammering that out after we agree on the syntax we need. Here's my take on the syntax: The basic unit of a search is a term. By default I think we want to match terms within a field rather than exactly matching. Terms are any of the following: aString term term = term & term = term AND term term | term = term OR term term ^ term = term XOR term (term) -term # not including term #term # begins with term term% # ends with term -#term -term% = term
term
< term
= term
<= term We also want to deal with the special case of: "foo * bar" There are some special operators we want to deal with. Some of these operators could be combined, any of the operators could be negated (with the syntax '-[operator]'). They should apply to the entire search: locally => this item exists in the local branch available => the item is not lost, checked out, or on hold onhold => the item is marked as reserved lost => the item is marked as lost circulating => the item can be checked out There are also operators which only act on a term (which of these should be the default, 'keyword: term'?): title: term exacttitle: term subject: term author: term exactauthor: term keyword: term branch: term marc-[recordnumber][-sub]: term # these should allow some wildcarding, like: # marc-6X0-a # marc-58X-a # # nobody in the discussion was/is a MARC expert, does # this syntax work? Some examples: title: ("looking glass") -(author: carrol%) # any title containing "looking glass" where the author doesn't # end with carrol "animal husbandry" OR (raising AND (rabbits OR chickens)) "animal husbandry" | (raising (rabbits | chickens)) # these are equivalent subject: knights AND marc-526-c >= 3 AND marc-526-c <= 4 # subject includes knights and AR reading level is between # third and fourth grade available locally subject: knights AND marc-526-c >= 3 AND marc-526-c <= 4 # as above, but local copies are available for checkout Pat Eyler Kaitiaki/manager migrant Linux sys admin the Koha project ruby, shell, and perl geek http://www.koha.org http://pate.eylerfamily.org