[Bug 9828] New: Zebra indexes useless subfields in UNIMARC 6XX
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Bug ID: 9828 Summary: Zebra indexes useless subfields in UNIMARC 6XX Classification: Unclassified Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: minor Priority: P5 - low Component: Searching Assignee: gmcharlt@gmail.com Reporter: mathieu.saby@univ-rennes2.fr Most of 6XX may contain a $2 that identifies the system used for indexing. It should not be indexed. In French libraries, $2 contains "rameau". So searching books about the music composer "Rameau" retreive thousands of records! For some 6XX fiels, other subfields should not be indexed, for example 600$t (dates of persons). And indexing of $j,$x,$y,$z subdivision could be improved by indexing them in specific indexes M. Saby -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathieu.saby@univ-rennes2.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|gmcharlt@gmail.com |mathieu.saby@univ-rennes2.f | |r -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathieu.saby@univ-rennes2.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #1 from mathieu saby <mathieu.saby@univ-rennes2.fr> --- Additionnally, subject indexing could be improved by using specific indexes for each 6XX if possible. In ccl.properties, a specific index is defined, but not used in record.abs : Subject-name-personal, alias su-na Other aliases are defined, but as synonyms to Subject index : su-to, su-geo and su-ut. We can use index Subject-name-personal and create new specific indexes by using existing bib1 attributes. M. Saby -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #2 from mathieu saby <mathieu.saby@univ-rennes2.fr> --- Created attachment 16179 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16179&action=edit [PATCH] Bug 9828: More specific indexing of UNIMARC 6XX fields Most of 6XX may contain a $2 that identifies the system used for indexing. It should not be indexed. In French libraries, $2 contains "rameau". So searching books about the music composer "Rameau" retreive thousands of records! For some 6XX fiels, other subfields should not be indexed, for example dates of persons and familiy, or adresses. In unimarc guide, 600$t,601$t,602$t are said to exist but to be "not used". I keep them indexed. Additionnally, subject indexing could be improved by using specific indexes for each 6XX if possible : In ccl.properties : - su-to, su-geo and su-ut are defined as aliases of Subject. - a specific index are defined, but not used in record.abs : Subject-name-personal, alis su-na We can use this index and create new specific indexes by using existing bib1 attributes. We could also index $j,$x,$y,$z subdivision in specific indexes. This patch does the following changes : 1) Add comments in record.abs 2) For all 6XX : Not indexing $2 (LSCH, Rameau...), $3 and $5 3) Not indexing specific subfields 600 : Personal name used as a subject // see Marc21 600 not indexing c (additional elements),f (dates),p (address/affiliation) 602 : Family name used as a subject // see Marc21 600 3X not indexing f (dates) 616 : Trademark not indexing c,f 3) For all 6XX : index $j,$x,$y,$z in several indexes in addition to the specfific index for their 6XX field: # 6XX$j : Genre/form : indexed in Subject, Subject-subdivision, Subject-genre-form # 6XX$x : Subject : indexed in Subject, Subject-subdivision # (could be topical subject or genre/form subject, so don't index in Subject-topical) # 6XX$y : Geographical subject : indexed in Subject, Subject-subdivision, Subject-name-geographical # 6XX$z : Chronological subject : indexed in Subject, Subject-subdivision, Subject-chronological 4) Define in ccl.properties some specific indexes : Subject-name-conference 1=1073 => alias su-conf Subject-name-corporate 1=1074 => alias su-corp Subject-genre-form 1=1075 => alias su-genre and su-form Subject-geographical 1=1076 => alias su-geo Subject-chronological 1=1077 => alias su-chrono Subject-title 1=1078 => alias su-ut and su-ti Subject-topical 1=1079 => alias su-to 5) Adding new aliases in Search.pm : su-chrono, su-form, su-genre, su-corp, su-conf, su-ti 6) Using these new indexes in record.abs for 600 : all field in Subject and Subject-Personal-Name all subfields except subdivisions in Personal-name 601 : all field in Subject, Subject-name-conference and Subject-name-corporate and Subject-name-conf all subfields except subdivisions in Corporate-name and Conference-name 602 : same as 600 but could be improved later 604 : all field in Subject and Subject-title ; $a in Subject-Personal-Name all subfields except subdivisions in Name-and-Title 605 : all field in Subject and Subject-title 606 : all field in Subject and Subject-topical 607 : all field in Subject and Subject-geographical all subfields except subdivisions in Name-geographic 608 : all field in Subject and Subject-genre-form To test : A. In a GRS 1 environment 1) Apply the patch 2) Rebuild zebra 3) Create a record A with - the string "bz9828" in 600$c 600$f 600$p, 602$f, 616$c, 616$f - the string "subform" in 602$j 4) Create a record B with the string "subgeo" in 606$y 5) Create a record C with the string "subdate" in 606$z 6) try to search "su:bz9828". You should have no results 7) try to search "su-geo:subgeo". You should have 1 result : record A 8) try to search "su-genre:subform". You should have 1 result : record B 9) try to search "su-chrono:subdate". You should have 1 result : record C 10) on existing records, try su-ut, su-to, su-na, su-form, su-corp, su-geo indexes, and see it results are relevant B. In a DOM environment same operations M. Saby -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathieu.saby@univ-rennes2.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathieu.saby@univ-rennes2.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathieu.saby@univ-rennes2.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16179|0 |1 is obsolete| | --- Comment #3 from mathieu saby <mathieu.saby@univ-rennes2.fr> --- Created attachment 17164 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=17164&action=edit [PATCH] Bug 9828: More specific indexing of UNIMARC 6XX fields Fixing typos in test plan -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |8962 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 I'm just a bot <gitbot@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |gitbot@bugs.koha-community. | |org When did the bot| |2013-09-29 last check this| | --- Comment #4 from I'm just a bot <gitbot@bugs.koha-community.org> --- Patch applied cleanly, go forth and signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 I'm just a bot <gitbot@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Patch doesn't apply When did the bot|2013-09-29 00:00:00 |2013-10-25 last check this| | --- Comment #5 from I'm just a bot <gitbot@bugs.koha-community.org> --- Applying: Bug 9828: More specific indexing of UNIMARC 6XX fields Using index info to reconstruct a base tree... M C4/Search.pm M etc/zebradb/biblios/etc/bib1.att M etc/zebradb/ccl.properties M etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml M etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl M etc/zebradb/marc_defs/unimarc/biblios/record.abs Falling back to patching base and 3-way merge... Auto-merging etc/zebradb/marc_defs/unimarc/biblios/record.abs Auto-merging etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl Auto-merging etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml Auto-merging etc/zebradb/ccl.properties CONFLICT (content): Merge conflict in etc/zebradb/ccl.properties Auto-merging etc/zebradb/biblios/etc/bib1.att Auto-merging C4/Search.pm Patch failed at 0001 Bug 9828: More specific indexing of UNIMARC 6XX fields The copy of the patch that failed is found in: /home/christopher/git/koha/.git/rebase-apply/patch When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort". -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathieu.saby@univ-rennes2.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #17164|0 |1 is obsolete| | --- Comment #6 from mathieu saby <mathieu.saby@univ-rennes2.fr> --- Created attachment 22379 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22379&action=edit [PATCH] Bug 9828: More specific indexing of UNIMARC 6XX fields Rebased patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathieu.saby@univ-rennes2.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #7 from mathieu saby <mathieu.saby@univ-rennes2.fr> --- A followup is probably needed for QueryParser. I don't know how to do that. Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathieu.saby@univ-rennes2.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|8962 | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 I'm just a bot <gitbot@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- When did the bot|2013-10-25 00:00:00 |2013-11-23 last check this| | --- Comment #8 from I'm just a bot <gitbot@bugs.koha-community.org> --- Patch applied cleanly, go forth and signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Nick Clemens <nick@quecheelibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Patch doesn't apply CC| |nick@quecheelibrary.org --- Comment #9 from Nick Clemens <nick@quecheelibrary.org> --- The sandbox you've requested is not ready. Some problems occurred applying patches from bug 9828: <h1>Something went wrong !</h1>Applying: Bug 9828: More specific indexing of UNIMARC 6XX fields Using index info to reconstruct a base tree... Falling back to patching base and 3-way merge... Auto-merging C4/Search.pm Auto-merging etc/zebradb/biblios/etc/bib1.att Auto-merging etc/zebradb/ccl.properties Auto-merging etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml CONFLICT (content): Merge conflict in etc/zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml Auto-merging etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl CONFLICT (content): Merge conflict in etc/zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl Auto-merging etc/zebradb/marc_defs/unimarc/biblios/record.abs Failed to merge in the changes. Patch failed at 0001 Bug 9828: More specific indexing of UNIMARC 6XX fields When you have resolved this problem run git bz apply --continue. If you would prefer to skip this patch, instead run git bz apply --skip. To restore the original branch and stop patching run git bz apply --abort. Bug 9828 - Zebra indexes useless subfields in UNIMARC 6XX -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #10 from mathieu saby <mathsabypro@gmail.com> --- Hello You cannot test this patch on a sandbox, because you need to rebuild zebra. If you want to test it, tell me, I will rebase the patch M. Saby -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #11 from Nick Clemens <nick@quecheelibrary.org> --- Ah, if can't do sandbox I will have to setup a koha instance with UNIMARC, once I have one ready to go I will let you know I am ready for rebase -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #12 from Nick Clemens <nick@quecheelibrary.org> --- I have a UNIMARC instance setup with dom indexing. I can confirm that before patch: - 6XX $2 is indexed - su-to, su-geo and su-ut return the same results as su - su-na does not return results - su-chrono, su-form, su-genre, su-corp, su-conf, su-ti return incorrect results if any Happy to test patch in dom if you rebase, I don't have a grs1 setup, but it is now deprecated so maybe is not needed for this patch? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #22379|0 |1 is obsolete| | --- Comment #13 from mathieu saby <mathsabypro@gmail.com> --- Created attachment 30954 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=30954&action=edit [PATCH] Bug 9828: More specific indexing of UNIMARC 6XX fields -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #30954|[PATCH] Bug 9828: More |[PATCH 1/2] Bug 9828: More description|specific indexing of |specific indexing of |UNIMARC 6XX fields |UNIMARC 6XX fields -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #14 from mathieu saby <mathsabypro@gmail.com> --- [New commit on 18 Aug 2014 : rebased, and DOM indexing only] Issues to fix : Most of 6XX may contain a $2 that identifies the system used for indexing. It should not be indexed. In French libraries, $2 contains "rameau". So searching books about the music composer "Rameau" retreive thousands of records! For some 6XX fiels, other subfields should not be indexed, for example dates of persons and family, or adresses. In Unimarc guide, 600$t,601$t,602$t are said to exist but to be "not used". I keep them indexed. Additionnally, subject indexing could be improved by using specific indexes for each 6XX if possible : In ccl.properties : - su-to, su-geo and su-ut are defined as aliases of Subject. - a specific index is defined, but not used in record.abs : Subject-name-personal, alias su-na We can use these indexes and create new specific indexes by using existing bib1 attributes. We could also index $j,$x,$y,$z subdivision in specific indexes. This patch does the following changes : 1) For all 6XX : Not indexing $2 (LSCH, Rameau...), $3 and $5 2) Suppressing the indexing of some specific subfields, depending on the field: 600 : Personal name used as a subject // see Marc21 600 not indexing c (additional elements),f (dates),p (address/affiliation) 602 : Family name used as a subject // see Marc21 600 3X not indexing f (dates) 616 : Trademark not indexing c,f 3) For all 6XX : index $j,$x,$y,$z in several indexes in addition to the specfific index for their 6XX field: 4) Define in ccl.properties some specific indexes : Subject-name-conference 1=1073 => alias su-conf Subject-name-corporate 1=1074 => alias su-corp Subject-genre-form 1=1075 => alias su-genre and su-form Subject-geographical 1=1076 => alias su-geo Subject-chronological 1=1077 => alias su-chrono Subject-title 1=1078 => alias su-ut and su-ti Subject-topical 1=1079 => alias su-to 5) Adding new aliases in Search.pm : su-chrono, su-form, su-genre, su-corp, su-conf, su-ti 6) Using these new indexes in for 600 : Subject and Subject-Personal-Name ; all subfields except subdivisions in Personal-name 601 : Subject, Subject-name-conference and Subject-name-corporate and Subject-name-conf ; all subfields except subdivisions in Corporate-name and Conference-name 602 : same as 600 but could be improved later 604 : Subject and Subject-title ; $a in Subject-Personal-Name ; all subfields except subdivisions in Name-and-Title 605 : Subject and Subject-title 606 : Subject and Subject-topical 607 : Subject and Subject-geographical ; all subfields except subdivisions in Name-geographic 608 : Subject and Subject-genre-form To test : A. In a UNIMARC-DOM indexing environment 1) Apply the patch 2) Rebuild zebra 3) Create a record A with some values in critical fields, for example: - the string "test9828" in 600$c 600$f 600$p, 602$f, 616$c, 616$f, 606$2,600$2 - the string "subform" in 600$j 4) Create a record B with the string "subgeo" in 606$y 5) Create a record C with the string "subdate" in 606$z 6) try to search "su:test9828". You should have no results 7) try to search "su-genre:subform". You should have 1 result : record A 8) try to search "su-geo:subgeo". You should have 1 result : record B 9) try to search "su-chrono:subdate". You should have 1 result : record C 10) on existing records, try su-ut, su-to, su-na, su-form, su-corp, su-geo indexes, and see it results are relevant Indexing of subjects could maybe be improved later -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #15 from mathieu saby <mathsabypro@gmail.com> --- Created attachment 30955 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=30955&action=edit [PATCH 2/2] Bug 9828 : Add and fix comments in UNIMARC biblio-koha-indexdefs.xml 2d patch : Only cosmetic : - the references to lines record.abs are now useless and outdated - some comments added in record.abs could be usefull in biblio-koha-indexdefs.xml No change expected, only comments -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #16 from mathieu saby <mathsabypro@gmail.com> --- For testing, I forgot a step : between "apply the patch" and "rebuild zebra", of course you need to copy the modified files from your source directory to the actual zebra configuration directory of your machine. But I suppose you know how to do that. Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #17 from mathieu saby <mathsabypro@gmail.com> --- And like Nick suggested, I removed the part of code related to GRS-1, as it is now more or less deprecated. So, please test the patch in a DOM UNIMARC environment. Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #18 from Nick Clemens <nick@quecheelibrary.org> --- Supposing I know how to do things: generally not a good idea :-) I figured it out it pretty quick though -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Nick Clemens <nick@quecheelibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Nick Clemens <nick@quecheelibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #30954|0 |1 is obsolete| | --- Comment #19 from Nick Clemens <nick@quecheelibrary.org> --- Created attachment 30960 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=30960&action=edit [SIGNED OFF] Bug 9828: More specific indexing of UNIMARC 6XX fields [New commit on 18 Aug 2014 : rebased, and DOM indexing only] Issues to fix : Most of 6XX may contain a $2 that identifies the system used for indexing. It should not be indexed. In French libraries, $2 contains "rameau". So searching books about the music composer "Rameau" retreive thousands of records! For some 6XX fiels, other subfields should not be indexed, for example dates of persons and family, or adresses. In Unimarc guide, 600$t,601$t,602$t are said to exist but to be "not used". I keep them indexed. Additionnally, subject indexing could be improved by using specific indexes for each 6XX if possible : In ccl.properties : - su-to, su-geo and su-ut are defined as aliases of Subject. - a specific index is defined, but not used in record.abs : Subject-name-personal, alias su-na We can use these indexes and create new specific indexes by using existing bib1 attributes. We could also index $j,$x,$y,$z subdivision in specific indexes. This patch does the following changes : 1) For all 6XX : Not indexing $2 (LSCH, Rameau...), $3 and $5 2) Suppressing the indexing of some specific subfields, depending on the field: 600 : Personal name used as a subject // see Marc21 600 not indexing c (additional elements),f (dates),p (address/affiliation) 602 : Family name used as a subject // see Marc21 600 3X not indexing f (dates) 616 : Trademark not indexing c,f 3) For all 6XX : index $j,$x,$y,$z in several indexes in addition to the specfific index for their 6XX field: 4) Define in ccl.properties some specific indexes : Subject-name-conference 1=1073 => alias su-conf Subject-name-corporate 1=1074 => alias su-corp Subject-genre-form 1=1075 => alias su-genre and su-form Subject-geographical 1=1076 => alias su-geo Subject-chronological 1=1077 => alias su-chrono Subject-title 1=1078 => alias su-ut and su-ti Subject-topical 1=1079 => alias su-to 5) Adding new aliases in Search.pm : su-chrono, su-form, su-genre, su-corp, su-conf, su-ti 6) Using these new indexes in for 600 : Subject and Subject-Personal-Name ; all subfields except subdivisions in Personal-name 601 : Subject, Subject-name-conference and Subject-name-corporate and Subject-name-conf ; all subfields except subdivisions in Corporate-name and Conference-name 602 : same as 600 but could be improved later 604 : Subject and Subject-title ; $a in Subject-Personal-Name ; all subfields except subdivisions in Name-and-Title 605 : Subject and Subject-title 606 : Subject and Subject-topical 607 : Subject and Subject-geographical ; all subfields except subdivisions in Name-geographic 608 : Subject and Subject-genre-form To test : A. In a UNIMARC-DOM indexing environment 1) Apply the patch 2) Rebuild zebra 3) Create a record A with some values in critical fields, for example: - the string "test9828" in 600$c 600$f 600$p, 602$f, 616$c, 616$f, 606$2,600$2 - the string "subform" in 600$j 4) Create a record B with the string "subgeo" in 606$y 5) Create a record C with the string "subdate" in 606$z 6) try to search "su:test9828". You should have no results 7) try to search "su-genre:subform". You should have 1 result : record A 8) try to search "su-geo:subgeo". You should have 1 result : record B 9) try to search "su-chrono:subdate". You should have 1 result : record C 10) on existing records, try su-ut, su-to, su-na, su-form, su-corp, su-geo indexes, and see it results are relevant Indexing of subjects could maybe be improved later Signed-off-by: Nick Clemens <nick@quecheelibrary.org> All seems to work as expected, I am not super-familiar with UNIMARC but I wonder if in su-corp and su-conf the subdivisions might be useful (e.g. France-Gendarmie / Staatsbibliothek-Berlin) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #20 from Nick Clemens <nick@quecheelibrary.org> --- Supposing I know how to do things: generally not a good idea :-) I figured it out it pretty quick though -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Nick Clemens <nick@quecheelibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #30960|0 |1 is obsolete| | --- Comment #21 from Nick Clemens <nick@quecheelibrary.org> --- Created attachment 30961 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=30961&action=edit [SIGNED OFF] Bug 9828: More specific indexing of UNIMARC 6XX fields [New commit on 18 Aug 2014 : rebased, and DOM indexing only] Issues to fix : Most of 6XX may contain a $2 that identifies the system used for indexing. It should not be indexed. In French libraries, $2 contains "rameau". So searching books about the music composer "Rameau" retreive thousands of records! For some 6XX fiels, other subfields should not be indexed, for example dates of persons and family, or adresses. In Unimarc guide, 600$t,601$t,602$t are said to exist but to be "not used". I keep them indexed. Additionnally, subject indexing could be improved by using specific indexes for each 6XX if possible : In ccl.properties : - su-to, su-geo and su-ut are defined as aliases of Subject. - a specific index is defined, but not used in record.abs : Subject-name-personal, alias su-na We can use these indexes and create new specific indexes by using existing bib1 attributes. We could also index $j,$x,$y,$z subdivision in specific indexes. This patch does the following changes : 1) For all 6XX : Not indexing $2 (LSCH, Rameau...), $3 and $5 2) Suppressing the indexing of some specific subfields, depending on the field: 600 : Personal name used as a subject // see Marc21 600 not indexing c (additional elements),f (dates),p (address/affiliation) 602 : Family name used as a subject // see Marc21 600 3X not indexing f (dates) 616 : Trademark not indexing c,f 3) For all 6XX : index $j,$x,$y,$z in several indexes in addition to the specfific index for their 6XX field: 4) Define in ccl.properties some specific indexes : Subject-name-conference 1=1073 => alias su-conf Subject-name-corporate 1=1074 => alias su-corp Subject-genre-form 1=1075 => alias su-genre and su-form Subject-geographical 1=1076 => alias su-geo Subject-chronological 1=1077 => alias su-chrono Subject-title 1=1078 => alias su-ut and su-ti Subject-topical 1=1079 => alias su-to 5) Adding new aliases in Search.pm : su-chrono, su-form, su-genre, su-corp, su-conf, su-ti 6) Using these new indexes in for 600 : Subject and Subject-Personal-Name ; all subfields except subdivisions in Personal-name 601 : Subject, Subject-name-conference and Subject-name-corporate and Subject-name-conf ; all subfields except subdivisions in Corporate-name and Conference-name 602 : same as 600 but could be improved later 604 : Subject and Subject-title ; $a in Subject-Personal-Name ; all subfields except subdivisions in Name-and-Title 605 : Subject and Subject-title 606 : Subject and Subject-topical 607 : Subject and Subject-geographical ; all subfields except subdivisions in Name-geographic 608 : Subject and Subject-genre-form To test : A. In a UNIMARC-DOM indexing environment 1) Apply the patch 2) Rebuild zebra 3) Create a record A with some values in critical fields, for example: - the string "test9828" in 600$c 600$f 600$p, 602$f, 616$c, 616$f, 606$2,600$2 - the string "subform" in 600$j 4) Create a record B with the string "subgeo" in 606$y 5) Create a record C with the string "subdate" in 606$z 6) try to search "su:test9828". You should have no results 7) try to search "su-genre:subform". You should have 1 result : record A 8) try to search "su-geo:subgeo". You should have 1 result : record B 9) try to search "su-chrono:subdate". You should have 1 result : record C 10) on existing records, try su-ut, su-to, su-na, su-form, su-corp, su-geo indexes, and see it results are relevant Indexing of subjects could maybe be improved later Signed-off-by: Nick Clemens <nick@quecheelibrary.org> All seems to work as expected, I am not super-familiar with UNIMARC but I wonder if in su-corp and su-conf the subdivisions might be useful (e.g. France-Gendarmie / Staatsbibliothek-Berlin) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Nick Clemens <nick@quecheelibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #30955|0 |1 is obsolete| | --- Comment #22 from Nick Clemens <nick@quecheelibrary.org> --- Created attachment 30962 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=30962&action=edit [SIGNED OFF] Bug 9828 : Add and fix comments in UNIMARC biblio-koha-indexdefs.xml Only cosmetic : - the references to lines record.abs are now useless and outdated - some comments added in record.abs could be usefull in biblio-koha-indexdefs.xml No change expected, only comments Signed-off-by: Nick Clemens <nick@quecheelibrary.org> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #23 from mathieu saby <mathsabypro@gmail.com> ---
All seems to work as expected, I am not super-familiar with UNIMARC but I wonder if in su-corp and su-conf the subdivisions might be useful (e.g. France-Gendarmie / Staatsbibliothek-Berlin)
For 601 field, the j, x, y, z subfield are mentionned in UNIMARC standard. See http://multimedia.bnf.fr/unimarcb_trad/B601-6-2013.pdf The best would be to index separately meetings and organizations, which are both in 601 field, but with different "indicator" value. I'm trying to write a followup for that. M. Saby -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #24 from mathieu saby <mathsabypro@gmail.com> --- I also need to add a followup for Queryparser -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #25 from mathieu saby <mathsabypro@gmail.com> --- Created attachment 31029 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31029&action=edit [PATCH 3/3] Bug 9828 : Followup for Queryparser and deletion of useless 6XX$9 This followup - changes some indexes in Queryparser configuration file - supresses some clearly useless 6XX$9 in biblio-koha-indexdefs.xml and adds 2 new ones, probably useless (not sure of that) - change the name of index Subject-geographical to Subject-name-geographical in ccl.properties (to match bib1.att) the xsl file zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl was generated with the following command: xsltproc zebradb/xsl/koha-indexdefs-to-zebra.xsl zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml > zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl To test : 1) Apply the 3 patches 2) copy the modified files from the source directory to the directory where you store the config files for Zebra and Queryparser The files modified by the 3 patches and that need to be copied are: etc/zebradb/biblios/etc/bib1.att etc/zebradb/ccl.properties etc/searchengine/queryparser.yaml etc/zebradb/ccl.properties .../unimarc/biblios/biblio-koha-indexdefs.xml .../unimarc/biblios/biblio-zebra-indexdefs.xsl 3) Rebuild Zebra 4) Create a record A with some values in critical fields, for example: - the string "test9828" in 600$c 600$f 600$p, 602$f, 616$c, 616$f, 606$2,600$2 - the string "subform" in 600$j 4) Create a record B with the string "subgeo" in 606$y 5) Create a record C with the string "subdate" in 606$z WITHOUT QP activated in sysprefs ("Don't try to use QP"): 6) try to search "su:test9828". You should have no results 7) try to search "su-genre:subform". You should have 1 result : record A 8) try to search "su-geo:subgeo". You should have 1 result : record B 9) try to search "su-chrono:subdate". You should have 1 result : record C 10) on existing records, try su-ut, su-to, su-na, su-form, su-corp, su-geo indexes, and see it results are relevant WITH QP activated in sysprefs: Same tests -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Needs Signoff --- Comment #26 from mathieu saby <mathsabypro@gmail.com> --- Would you be kind enough to test it again, with the 3rd patch applied on top of the 2 first ones? You need to test without QP activated : it should work as in your previous test and with QP activated (it should work also). (Results with QP are sometimes slightly different, I cannot remember why) M. Saby -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #27 from mathieu saby <mathsabypro@gmail.com> --- Note that it is currently impossible to make a more precise indexing of some fields depending on the value stored in the 2 indicators (for field 601 for example). This need an change in / etc / zebradb / xsl / koha-indexdefs-to-zebra.xsl M. Saby -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Nick Clemens <nick@quecheelibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Nick Clemens <nick@quecheelibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #31029|0 |1 is obsolete| | --- Comment #28 from Nick Clemens <nick@quecheelibrary.org> --- Created attachment 31113 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=31113&action=edit [SIGNED OFF] Bug 9828 : Followup for Queryparser and deletion of useless 6XX$9 This followup - changes some indexes in Queryparser configuration file - supresses some clearly useless 6XX$9 in biblio-koha-indexdefs.xml and adds 2 new ones, probably useless (not sure of that) - change the name of index Subject-geographical to Subject-name-geographical in ccl.properties (to match bib1.att) the xsl file zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl was generated with the following command: xsltproc zebradb/xsl/koha-indexdefs-to-zebra.xsl zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml > zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl To test : 1) Apply the 3 patches 2) copy the modified files from the source directory to the directory where you store the config files for Zebra and Queryparser The files modified by the 3 patches and that need to be copied are: etc/zebradb/biblios/etc/bib1.att etc/zebradb/ccl.properties etc/searchengine/queryparser.yaml etc/zebradb/ccl.properties .../unimarc/biblios/biblio-koha-indexdefs.xml .../unimarc/biblios/biblio-zebra-indexdefs.xsl 3) Rebuild Zebra 4) Create a record A with some values in critical fields, for example: - the string "test9828" in 600$c 600$f 600$p, 602$f, 616$c, 616$f, 606$2,600$2 - the string "subform" in 600$j 4) Create a record B with the string "subgeo" in 606$y 5) Create a record C with the string "subdate" in 606$z WITHOUT QP activated in sysprefs ("Don't try to use QP"): 6) try to search "su:test9828". You should have no results 7) try to search "su-genre:subform". You should have 1 result : record A 8) try to search "su-geo:subgeo". You should have 1 result : record B 9) try to search "su-chrono:subdate". You should have 1 result : record C 10) on existing records, try su-ut, su-to, su-na, su-form, su-corp, su-geo indexes, and see it results are relevant WITH QP activated in sysprefs: Same tests Signed-off-by: Nick Clemens <nick@quecheelibrary.org> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #29 from mathieu saby <mathsabypro@gmail.com> --- New bug created for indicators: bug 12813 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Paul Poulain <paul.poulain@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA CC| |paul.poulain@biblibre.com --- Comment #30 from Paul Poulain <paul.poulain@biblibre.com> --- QA comments: * NOT UNIMARC-specific things: - this patch fixes 2 obvious mistakes in bib1.att. It means it also fixes a bug in MARC21 that no-one ever saw : -att 1075 Subject-genre/form -att 1076 Subject-name-gerographical +att 1075 Subject-genre-form +att 1076 Subject-name-geographical - this patch adds the following attributes: +su-corp Subject-name-corporate +su-ti Subject-title => those 2 additions are harmless, i've checked that there is no reference to that in marc21/normarc directories no other comment, test plan works. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Paul Poulain <paul.poulain@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #30961|0 |1 is obsolete| | Attachment #30962|0 |1 is obsolete| | Attachment #31113|0 |1 is obsolete| | --- Comment #31 from Paul Poulain <paul.poulain@biblibre.com> --- Created attachment 32111 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=32111&action=edit Bug 9828: More specific indexing of UNIMARC 6XX fields [New commit on 18 Aug 2014 : rebased, and DOM indexing only] Issues to fix : Most of 6XX may contain a $2 that identifies the system used for indexing. It should not be indexed. In French libraries, $2 contains "rameau". So searching books about the music composer "Rameau" retreive thousands of records! For some 6XX fiels, other subfields should not be indexed, for example dates of persons and family, or adresses. In Unimarc guide, 600$t,601$t,602$t are said to exist but to be "not used". I keep them indexed. Additionnally, subject indexing could be improved by using specific indexes for each 6XX if possible : In ccl.properties : - su-to, su-geo and su-ut are defined as aliases of Subject. - a specific index is defined, but not used in record.abs : Subject-name-personal, alias su-na We can use these indexes and create new specific indexes by using existing bib1 attributes. We could also index $j,$x,$y,$z subdivision in specific indexes. This patch does the following changes : 1) For all 6XX : Not indexing $2 (LSCH, Rameau...), $3 and $5 2) Suppressing the indexing of some specific subfields, depending on the field: 600 : Personal name used as a subject // see Marc21 600 not indexing c (additional elements),f (dates),p (address/affiliation) 602 : Family name used as a subject // see Marc21 600 3X not indexing f (dates) 616 : Trademark not indexing c,f 3) For all 6XX : index $j,$x,$y,$z in several indexes in addition to the specfific index for their 6XX field: 4) Define in ccl.properties some specific indexes : Subject-name-conference 1=1073 => alias su-conf Subject-name-corporate 1=1074 => alias su-corp Subject-genre-form 1=1075 => alias su-genre and su-form Subject-geographical 1=1076 => alias su-geo Subject-chronological 1=1077 => alias su-chrono Subject-title 1=1078 => alias su-ut and su-ti Subject-topical 1=1079 => alias su-to 5) Adding new aliases in Search.pm : su-chrono, su-form, su-genre, su-corp, su-conf, su-ti 6) Using these new indexes in for 600 : Subject and Subject-Personal-Name ; all subfields except subdivisions in Personal-name 601 : Subject, Subject-name-conference and Subject-name-corporate and Subject-name-conf ; all subfields except subdivisions in Corporate-name and Conference-name 602 : same as 600 but could be improved later 604 : Subject and Subject-title ; $a in Subject-Personal-Name ; all subfields except subdivisions in Name-and-Title 605 : Subject and Subject-title 606 : Subject and Subject-topical 607 : Subject and Subject-geographical ; all subfields except subdivisions in Name-geographic 608 : Subject and Subject-genre-form To test : A. In a UNIMARC-DOM indexing environment 1) Apply the patch 2) Rebuild zebra 3) Create a record A with some values in critical fields, for example: - the string "test9828" in 600$c 600$f 600$p, 602$f, 616$c, 616$f, 606$2,600$2 - the string "subform" in 600$j 4) Create a record B with the string "subgeo" in 606$y 5) Create a record C with the string "subdate" in 606$z 6) try to search "su:test9828". You should have no results 7) try to search "su-genre:subform". You should have 1 result : record A 8) try to search "su-geo:subgeo". You should have 1 result : record B 9) try to search "su-chrono:subdate". You should have 1 result : record C 10) on existing records, try su-ut, su-to, su-na, su-form, su-corp, su-geo indexes, and see it results are relevant Indexing of subjects could maybe be improved later Signed-off-by: Nick Clemens <nick@quecheelibrary.org> All seems to work as expected, I am not super-familiar with UNIMARC but I wonder if in su-corp and su-conf the subdivisions might be useful (e.g. France-Gendarmie / Staatsbibliothek-Berlin) Signed-off-by: Paul Poulain <paul.poulain@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #32 from Paul Poulain <paul.poulain@biblibre.com> --- Created attachment 32112 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=32112&action=edit Bug 9828 : Add and fix comments in UNIMARC biblio-koha-indexdefs.xml Only cosmetic : - the references to lines record.abs are now useless and outdated - some comments added in record.abs could be usefull in biblio-koha-indexdefs.xml No change expected, only comments Signed-off-by: Nick Clemens <nick@quecheelibrary.org> Signed-off-by: Paul Poulain <paul.poulain@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #33 from Paul Poulain <paul.poulain@biblibre.com> --- Created attachment 32113 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=32113&action=edit Bug 9828 : Followup for Queryparser and deletion of useless 6XX$9 This followup - changes some indexes in Queryparser configuration file - supresses some clearly useless 6XX$9 in biblio-koha-indexdefs.xml and adds 2 new ones, probably useless (not sure of that) - change the name of index Subject-geographical to Subject-name-geographical in ccl.properties (to match bib1.att) the xsl file zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl was generated with the following command: xsltproc zebradb/xsl/koha-indexdefs-to-zebra.xsl zebradb/marc_defs/unimarc/biblios/biblio-koha-indexdefs.xml > zebradb/marc_defs/unimarc/biblios/biblio-zebra-indexdefs.xsl To test : 1) Apply the 3 patches 2) copy the modified files from the source directory to the directory where you store the config files for Zebra and Queryparser The files modified by the 3 patches and that need to be copied are: etc/zebradb/biblios/etc/bib1.att etc/zebradb/ccl.properties etc/searchengine/queryparser.yaml etc/zebradb/ccl.properties .../unimarc/biblios/biblio-koha-indexdefs.xml .../unimarc/biblios/biblio-zebra-indexdefs.xsl 3) Rebuild Zebra 4) Create a record A with some values in critical fields, for example: - the string "test9828" in 600$c 600$f 600$p, 602$f, 616$c, 616$f, 606$2,600$2 - the string "subform" in 600$j 4) Create a record B with the string "subgeo" in 606$y 5) Create a record C with the string "subdate" in 606$z WITHOUT QP activated in sysprefs ("Don't try to use QP"): 6) try to search "su:test9828". You should have no results 7) try to search "su-genre:subform". You should have 1 result : record A 8) try to search "su-geo:subgeo". You should have 1 result : record B 9) try to search "su-chrono:subdate". You should have 1 result : record C 10) on existing records, try su-ut, su-to, su-na, su-form, su-corp, su-geo indexes, and see it results are relevant WITH QP activated in sysprefs: Same tests Signed-off-by: Nick Clemens <nick@quecheelibrary.org> Signed-off-by: Paul Poulain <paul.poulain@biblibre.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #34 from mathieu saby <mathsabypro@gmail.com> --- Note: France -- Gendarmerie is encoded as $aFrance $bGendarmerie http://multimedia.bnf.fr/unimarcb_trad/B601-6-2013.pdf M. Saby -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Failed QA CC| |tomascohen@gmail.com --- Comment #35 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Patches don't apply cleanly, please rebase. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #36 from mathieu saby <mathsabypro@gmail.com> --- Of course, because of the new code for facets ;-) I will rebase this week end. Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #37 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to mathieu saby from comment #36)
Of course, because of the new code for facets ;-) I will rebase this week end.
Of course, sorry for that Mathieu. Please feel free to contact me regarding this and the other UNIMARC bug, I'd like to have them pushed soon. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #32111|0 |1 is obsolete| | --- Comment #38 from mathieu saby <mathsabypro@gmail.com> --- Created attachment 32720 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=32720&action=edit [PATCH 1/3] Bug 9828: More specific indexing of UNIMARC 6XX fields -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #32112|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #32113|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #32720|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #39 from mathieu saby <mathsabypro@gmail.com> --- Created attachment 32725 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=32725&action=edit [PATCH 1/3] Bug 9828: More specific indexing of UNIMARC 6XX fields -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #40 from mathieu saby <mathsabypro@gmail.com> --- Created attachment 32726 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=32726&action=edit [PATCH 2/3] Bug 9828 : Add and fix comments in UNIMARC biblio-koha-indexdefs.xml -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #41 from mathieu saby <mathsabypro@gmail.com> --- Created attachment 32727 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=32727&action=edit [PATCH 3/3] Bug 9828 : Followup for Queryparser and deletion of useless 6XX$9 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #42 from mathieu saby <mathsabypro@gmail.com> --- Rebased I hope I did not break anything... I saw an issue in 995$b/c, which are not indexed in the same way in facets and in ordinary index. I will send a message to the french list, as it is not linked with this patch. Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master --- Comment #43 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Patch pushed to master. Thanks Mathieu! -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |In Discussion --- Comment #44 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Mathieu, I tried to email you but it bounced. Your patches seem to affect MARC21 (and probably NORMARC) search behaviour. I think it is because you removed: Subject 1=21 su Subject -su-to Subject and probably because you only patched the UNIMARC biblio-koha-indexdefs.xml file. I'm reverting it, but I want it pushed again soon. So, please take a look at the output of this with your patches applied: prove t/db_dependent/QueryParser.t t/db_dependent/Search.t to find out. I think this kind of problem is one of the reasons we need better regression tests, lacking a lot in UNIMARC. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Failed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #45 from mathieu saby <mathsabypro@gmail.com> --- I will try to have a look at that. But not obvious, because I don't have a MARC21 install... Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #46 from mathieu saby <mathsabypro@gmail.com> --- Could you precise how the MARC21 was affected? No result at all? To much results? Issue with the facets maybe? I suppressed those 3 aliases -su-to Subject -su-geo Subject -su-ut Subject But I redefined them later in ccl.properties. So, MARC21 should not be broken What I see from MARC21 config file: - 650 field is indexed in "Subject" index only - "su-to" is used only for building the facets on 650 So the facets will probably be more precise than before. Is this the behavior you noticed? If so, it is not a bug. Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #47 from mathieu saby <mathsabypro@gmail.com> --- By the way, the facet code for unimarc could slighty be improved, but it can be done in an other bug... -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #48 from mathieu saby <mathsabypro@gmail.com> --- The test t/Queryparser fails because of a wrong "+ authority-number:" -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #49 from mathieu saby <mathsabypro@gmail.com> --- Created attachment 32882 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=32882&action=edit [PATCH 4/4] Bug 9828 - Delete a duplicate line in queryparser.yaml the line "authority-number:" was wrongly duplicated This change should make t/Queryparser.t happy -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #50 from mathieu saby <mathsabypro@gmail.com> --- I added a followup to silent t/Queryparser Could you tell me if it is OK, and what is the issue for MARC21 ? Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #51 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to mathieu saby from comment #50)
I added a followup to silent t/Queryparser
Could you tell me if it is OK, and what is the issue for MARC21 ?
Mathieu, what I noticed is that $ prove t/db_dependent/Search.t was failing with your patches. Will try with the new ones. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_3_18_candidate -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de --- Comment #52 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- It would be nice to see this get into 3.18. I can confirm that t/db_dependent/Search.t was failing - hope it can be fixed. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #53 from mathieu saby <mathsabypro@gmail.com> --- Search.t is not failing anymore with my new patches ;-) Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Failed QA --- Comment #54 from Tomás Cohen Arazi <tomascohen@gmail.com> --- I'm failing this one so maybe any MARC21/NORMARC dev can do the changes needed for them. I'll try to, but not sure i'll have the time. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #55 from mathieu saby <mathsabypro@gmail.com> --- Are you sure there is still a problem? I thought I had fixed it... Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Paul Poulain <paul.poulain@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Passed QA --- Comment #56 from Paul Poulain <paul.poulain@biblibre.com> --- (In reply to Tomás Cohen Arazi from comment #54)
I'm failing this one so maybe any MARC21/NORMARC dev can do the changes needed for them. I'll try to, but not sure i'll have the time.
Tomas, I don't understand what you mean here. In my comment 30, I pointed that this patch ALSO fixed some MARC21/NORMAC errors, not that it need some work to fix MARC21/NORMAC errors ! So ? (back to passed QA because this patch is really useful for UNIMARC !) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 mathieu saby <mathsabypro@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Signed Off --- Comment #57 from mathieu saby <mathsabypro@gmail.com> --- Hello Paul & Tomas - Paul, since your comment 30 the patch has been changed several times... - Tomas, see my comment 53, i rewrote the patch, so there should not be any problem in unit tests... So, in my opinion this patch has been signed off, but need a NEW look by QA, especially to check if UT are broken, and if everything works well with MARC21 install. So, I think the best status is "Signed off" Mathieu -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #32882|0 |1 is obsolete| | --- Comment #58 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 34537 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=34537&action=edit Bug 9828 - Delete a duplicate line in queryparser.yaml the line "authority-number:" was wrongly duplicated This change should make t/Queryparser.t happy Signed-off-by: Jonathan Druart <jonathan.druart@biblibre.com> t/Queryparser.t and t/Search.t pass. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Failed QA Keywords|rel_3_18_candidate | --- Comment #59 from Tomás Cohen Arazi <tomascohen@gmail.com> --- I'm sorry, but it does break the tests, still. What I did was: - a clean dev install choosing MARC21 - loaded the default DB structure with the default data - Run $ prove t/db_dependent/Search.t => BOOM. I fail it, but plan to help on the MARC21 side as soon as I get back to the office from vacation. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9828 --- Comment #60 from mathieu saby <mathsabypro@gmail.com> --- (In reply to Tomás Cohen Arazi from comment #59)
I'm sorry, but it does break the tests, still.
What I did was: - a clean dev install choosing MARC21 - loaded the default DB structure with the default data - Run $ prove t/db_dependent/Search.t => BOOM.
I fail it, but plan to help on the MARC21 side as soon as I get back to the office from vacation.
Sorry for that :/ I don't think I will have to fix that before long. Mathieu -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org