Paul a écrit: NPL had a tech meeting today focusing on the opac searching and we have reached some tentative conclusions about how to proceed. Running some test searches using marc_subfield_table we realized that a search model based on that table is inadequate for our needs. For example, a search on 'patrick o'brian' using the 'like' syntax produces no results if the database entry is stored as 'o'brian, patrick' (when author is stored in the 100a that is the format). On the other hand, a search using the current marc_word model fails for reasons we have already talked about (marc_word does not keep track of single characters, &c.). But if the marc_word table did index single charcters, a search model based on marc_word would work very well. For example, a search on 'o'brian, patrick' or 'patrick o'brian' would both return the correct records. So our idea is to re-create our marc_word table so that it indexes all characters from the tags and subfields that we want to use for searches (we don't need all of them as you pointed out; for instance, we will never use 300 for a search). So we have three basic tasks: 1.) write a script to re-create marc_word using the parameters we choose for searching and including all characters. 2.) fix Biblio.pm so that it will include all characters when it adds records to marc_word (currently we add to our holdings using a modified version of bulkmarcimport.pl that relies on Biblio.pm) 3.) write a clean-up script to delete all the tags and subfields from marc_word that we will never use (like 300) Does that sound like a sound plan to you Paul? Do you have any scripts that will speed up the process of re-building our marc_word table--if not we will write one ourselves. Can you make the changes to Biblio.pm that will force it to index single characters? One final point about search results. Currently the marc searching does not pass all the variables to the template so that we can choose what values to display (for example, Lord of the Rings: The Two Towers currently displays as 'Lord of the Rings:' without the subtitle). I suggest that we setup a method of easily making marc fields available to the template so that each library can decide exactly what marc fields they want to display for the initial search results. Comments? Suggestions? Joshua On Wed, May 26, 2004 at 04:38:37PM +0200, paul POULAIN wrote:
Stephen Hedges a écrit :
At what point does marc_word become so big and clunky that it becomes a liability instead of an asset? NPL's marc-word file is full of 'junk' entries like "(pa." (picked up when an ISBN number has "(pa.)" after it to denote paperback) and other such MARC oddities. Our stopword file should ideally be expanded to catch all of this junk, but I haven't done that yet. Now we're talking about adding punctuation marks and single letters! I agree with Joshua that this is what should be done if we're going to depend on using marc_word and expect to get any meaningful search results. My question is: maybe it would be more efficient to just use marc_subfield_table for these searches and forget about marc_word?
you're right stephen... I have an other idea that could be coded quickly : in the MARC framework, we could add a checkbox called "do NOT index this subfield". If checked, the subfield wouldn't be stored in marc_word (but stored in marc_subfield_table) (Needs a script to clean the DB too, should be quite easy : foreach subfield in marc_subfield_structure { if checkbox checked { delete from marc_word where subfield= this one } } ...)
-- Paul POULAIN Consultant indépendant en logiciels libres responsable francophone de koha (SIGB libre http://www.koha-fr.org)
------------------------------------------------------- This SF.Net email is sponsored by: Oracle 10g Get certified on the hottest thing ever to hit the market... Oracle 10g. Take an Oracle 10g class now, and we'll give you the exam FREE. http://ads.osdn.com/?ad_id149&alloc_id66&opÌk _______________________________________________ Koha-devel mailing list Koha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/koha-devel