Bugs related to Zebra indexing for Unimarc
Hi Some time ago I wrote a few patchs for making unimarc indexing more precise, especially 7XX and 6XX fields. One has just been signed off by Nick Clemens (thanks!), a MARC21 librarian. Could some UNIMARC people make the QA for it? http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 (for 6XX fields: subjects) During the test, we discovered that it was not possible to index some fields depending on the value of MARC indicators. It would be a nice improvement, so I created a bug for that (but I won't work on it...) http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12813 He was about to sign the 2d patch about 7XX fields (authors) http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9352 But there is an issue that I am not sure to be able to fix: The patch index only relevant subfields in author index (so, no more the profession of the author, its date, or its address, because those information are not here to be searched by users, but to disambiguate authors). As a result, each subfield is indexed separately in "phr" and "word" index. So the complete name is no more searchable with "au:phr" index Ex : in master, you can retreive a record with "Doe John" in "au:phr" index if you have 700$aDoe$bJohn with the patch, you get 0 record I don't want to spend days on that... so do you know if the problem occurs also for MARC21? Is there a way to resolve it by using some (new?) template in *koha-indexdefs-to-zebra.xsl ? Regards, Mathieu Saby*
Hi Matieu, Il 26/08/2014 10:08, Mathieu Saby ha scritto:
He was about to sign the 2d patch about 7XX fields (authors) http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9352 But there is an issue that I am not sure to be able to fix: The patch index only relevant subfields in author index (so, no more the profession of the author, its date, or its address, because those information are not here to be searched by users, but to disambiguate authors). As a result, each subfield is indexed separately in "phr" and "word" index. So the complete name is no more searchable with "au:phr" index Ex : in master, you can retreive a record with "Doe John" in "au:phr" index if you have 700$aDoe$bJohn with the patch, you get 0 record
I don't want to spend days on that... so do you know if the problem occurs also for MARC21?
I think that a solution is possible, but is not easy. As first step for au:phr we need: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="abdg"> ... </index_data_field> a different entry from the entry of au:word that is: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="bdg"> ... </index_data_field> Then we need a XSLT procedure that create a phrase instead of single subunits. In fact DOM indexing is XSLT. In MARC21 all subfield in 100 (equivalent of 700 in Unimarc) are indexed in one shot. In biblio-koha-indexdefs.xml: <index_data_field xmlns="http://www.koha-community.org/schemas/index-defs" tag="100"> <target_index>Author:w</target_index> <target_index>Author:p</target_index> <target_index>Author:s</target_index> <target_index>Author-title:w</target_index> <target_index>Author-name-personal:w</target_index> <target_index>Name:w</target_index> <target_index>Name-and-title:w</target_index> <target_index>Personal-name:w</target_index> </index_data_field> The result in biblio-zebra-indexdefs.xsl is: <xslo:template mode="index_data_field" match="marc:datafield[@tag='100']"> <z:index name="Author:w Author:p Author:s Author-title:w Author-name-personal:w Name:w Name-and-title:w Personal-name:w"> <xslo:variable name="raw_heading"> <xslo:for-each select="marc:subfield"> <xslo:if test="position() > 1"> <xslo:value-of select="substring(' ', 1, 1)"/> </xslo:if> <xslo:value-of select="."/> </xslo:for-each> </xslo:variable> <xslo:value-of select="normalize-space($raw_heading)"/> </z:index> </xslo:template>
Is there a way to resolve it by using some (new?) template in *koha-indexdefs-to-zebra.xsl ?
Probsbly is, but we need to write a new template Bye Zeno Tajoli -- Dr. Zeno Tajoli Soluzioni per la Ricerca Istituzionale - Automazione Biblioteche z.tajoli@cineca.it fax +39 02 2135520 CINECA - Sede operativa di Segrate
Le 26/08/2014 11:21, Zeno Tajoli a écrit :
Hi Matieu,
Thank you for your answer
Il 26/08/2014 10:08, Mathieu Saby ha scritto:
He was about to sign the 2d patch about 7XX fields (authors) http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9352 But there is an issue that I am not sure to be able to fix: The patch index only relevant subfields in author index (so, no more the profession of the author, its date, or its address, because those information are not here to be searched by users, but to disambiguate authors). As a result, each subfield is indexed separately in "phr" and "word" index. So the complete name is no more searchable with "au:phr" index Ex : in master, you can retreive a record with "Doe John" in "au:phr" index if you have 700$aDoe$bJohn with the patch, you get 0 record
I don't want to spend days on that... so do you know if the problem occurs also for MARC21?
I think that a solution is possible, but is not easy.
As first step for au:phr we need:
<index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="abdg"> ... </index_data_field>
a different entry from the entry of au:word that is: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="bdg"> ... </index_data_field>
Why not "abdg" for the second entry? For the first entry (phrase indexing), don't we need to define a new element instead of index_subfields (something like index_subfields_as_phrase)? like: <index_subfields_as_phrase xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="abdg"> or maybe to use an attribute like "index_as_phrase" so that the new xsl template could be applied only if we want so? like; <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="abdg" index_as_phrase="yes"> Are we allowed to use new elements or attributes in *-koha-*.xml files ? There is a namespace defined in the file : xmlns:kohaidx="http://www.koha-community.org/schemas/index-defs". But I cannot find the definitions either on koha-community.org, or elsewhere. Do you know if it is avaiable somewhere? Excuse me if my remarks are silly, I don't really know XSLT ;-) Mathieu
Then we need a XSLT procedure that create a phrase instead of single subunits.
In fact DOM indexing is XSLT.
In MARC21 all subfield in 100 (equivalent of 700 in Unimarc) are indexed in one shot.
In biblio-koha-indexdefs.xml: <index_data_field xmlns="http://www.koha-community.org/schemas/index-defs" tag="100"> <target_index>Author:w</target_index> <target_index>Author:p</target_index> <target_index>Author:s</target_index> <target_index>Author-title:w</target_index> <target_index>Author-name-personal:w</target_index> <target_index>Name:w</target_index> <target_index>Name-and-title:w</target_index> <target_index>Personal-name:w</target_index> </index_data_field>
The result in biblio-zebra-indexdefs.xsl is:
<xslo:template mode="index_data_field" match="marc:datafield[@tag='100']"> <z:index name="Author:w Author:p Author:s Author-title:w Author-name-personal:w Name:w Name-and-title:w Personal-name:w"> <xslo:variable name="raw_heading"> <xslo:for-each select="marc:subfield"> <xslo:if test="position() > 1"> <xslo:value-of select="substring(' ', 1, 1)"/> </xslo:if> <xslo:value-of select="."/> </xslo:for-each> </xslo:variable> <xslo:value-of select="normalize-space($raw_heading)"/> </z:index> </xslo:template>
Is there a way to resolve it by using some (new?) template in *koha-indexdefs-to-zebra.xsl ?
Probsbly is, but we need to write a new template
Bye Zeno Tajoli
Hi all Il 26/08/2014 12:48, Mathieu Saby ha scritto:
As first step for au:phr we need:
<index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="abdg"> ... </index_data_field>
a different entry from the entry of au:word that is: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="bdg"> ... </index_data_field>
Why not "abdg" for the second entry?
I fact my only idea is that there is an error at line 317 of your path. The line is: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="bdg"> but for phrase it needs to be so: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="abdg"> Why for phrases and not for word ? Because there is the line 309 in the patch (304 in master): <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="a"> So the word's indexes for 700$a are OK, but not phrase.
Are we allowed to use new elements or attributes in *-koha-*.xml files ? I think yes, namespace declaration is present only to be complian with xml tools, but it is not really present.
There is a namespace defined in the file : xmlns:kohaidx="http://www.koha-community.org/schemas/index-defs". But I cannot find the definitions either on koha-community.org, or elsewhere. Do you know if it is avaiable somewhere? No, I think that kohaidx="http://www.koha-community.org/schemas/index-defs" and xmlns="http://www.koha-community.org/schemas/index-defs" are here only for the sake of xsltproc.
Bye Zeno Tajoli -- Dr. Zeno Tajoli Soluzioni per la Ricerca Istituzionale - Automazione Biblioteche z.tajoli@cineca.it fax +39 02 2135520 CINECA - Sede operativa di Segrate
Le 26/08/2014 15:43, Zeno Tajoli a écrit :
Hi all Il 26/08/2014 12:48, Mathieu Saby ha scritto:
As first step for au:phr we need:
<index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="abdg"> ... </index_data_field>
a different entry from the entry of au:word that is: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="bdg"> ... </index_data_field>
Why not "abdg" for the second entry?
I fact my only idea is that there is an error at line 317 of your path. The line is: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="bdg">
but for phrase it needs to be so: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="abdg">
Why for phrases and not for word ? Because there is the line 309 in the patch (304 in master): <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="a">
So the word's indexes for 700$a are OK, but not phrase.
OK, in fact my patch was just trying to preserve the indexing of $a (only) in "author:sort" index Currently it does: index 700$a in auhor:w, author:p, author:s and index 700$bdg in author:w, author:p I could do index 700$a in author:s and index 700$abdg in author:w, author:p But don't know if it is best ;-) Mathieu
Are we allowed to use new elements or attributes in *-koha-*.xml files ? I think yes, namespace declaration is present only to be complian with xml tools, but it is not really present.
There is a namespace defined in the file : xmlns:kohaidx="http://www.koha-community.org/schemas/index-defs". But I cannot find the definitions either on koha-community.org, or elsewhere. Do you know if it is avaiable somewhere? No, I think that kohaidx="http://www.koha-community.org/schemas/index-defs" and xmlns="http://www.koha-community.org/schemas/index-defs" are here only for the sake of xsltproc.
Bye Zeno Tajoli
Hello I wrote a patch for managing "groups of subfields" and indexing them as a whole in a phrase index : Bug 9352 (the 3rd patch). Mathieu Le 26/08/2014 15:54, Mathieu Saby a écrit :
Le 26/08/2014 15:43, Zeno Tajoli a écrit :
Hi all Il 26/08/2014 12:48, Mathieu Saby ha scritto:
As first step for au:phr we need:
<index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="abdg"> ... </index_data_field>
a different entry from the entry of au:word that is: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="bdg"> ... </index_data_field>
Why not "abdg" for the second entry?
I fact my only idea is that there is an error at line 317 of your path. The line is: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="bdg">
but for phrase it needs to be so: <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="abdg">
Why for phrases and not for word ? Because there is the line 309 in the patch (304 in master): <index_subfields xmlns="http://www.koha-community.org/schemas/index-defs" tag="700" subfields="a">
So the word's indexes for 700$a are OK, but not phrase.
OK, in fact my patch was just trying to preserve the indexing of $a (only) in "author:sort" index Currently it does: index 700$a in auhor:w, author:p, author:s and index 700$bdg in author:w, author:p
I could do index 700$a in author:s and index 700$abdg in author:w, author:p
But don't know if it is best ;-)
Mathieu
Are we allowed to use new elements or attributes in *-koha-*.xml files ? I think yes, namespace declaration is present only to be complian with xml tools, but it is not really present.
There is a namespace defined in the file : xmlns:kohaidx="http://www.koha-community.org/schemas/index-defs". But I cannot find the definitions either on koha-community.org, or elsewhere. Do you know if it is avaiable somewhere? No, I think that kohaidx="http://www.koha-community.org/schemas/index-defs" and xmlns="http://www.koha-community.org/schemas/index-defs" are here only for the sake of xsltproc.
Bye Zeno Tajoli
-- Mathieu Saby Élève conservateur d'État - DCB 23 ENSSIB
participants (2)
-
Mathieu Saby -
Zeno Tajoli