http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14456 --- Comment #1 from Jacek Ablewicz <ablewicz@gmail.com> --- Created attachment 40596 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=40596&action=edit Bug 14456 - EmbedSeeFromHeadings record filter shouldn't process MARC holding fields If the system preference IncludeSeeFromInSearches is enabled, records exported for zebra indexing are being additionally processed by EmbedSeeFromHeadings record filter (right now used only in rebuild_zebra.pl script). This filter embeds 'see from' fields (extracted from authority records linked with the given biblio via $9 subfields) into target MARC record, which is then subsequently indexed in zebra. Currently all fields containing $9 are getting the same exact treatment by this filter. But on the export stage when the filter is applied, MARC record being processed already does have holdings data fields added in the previous stage (usually 952 / 995, depending on the MARC format). Problem is that holdings data fields use to have $9 subfields in them as well (mapped to item.itemnumber by default). As a consequence, some (great many in the typical setup) records exported for zebra indexing may have surplus "see from" fields added erroneously in semi-random fashion, so biblio searches would often return some completely unexpected additional results. EmbedSeeFromHeadings record filter should not process holdings fields when dealing with MARC records intended for zebra indexing. To reproduce: 1) database with as many sample or real-world biblio, item and authority records as possible is recommended for testing purposes 2) enable IncludeSeeFromInSearches 3) export a bunch of biblio records for zebra (e.g.: misc/migration_tools/rebuild_zebra.pl -I -b -x -k -length=1000), inspect the result xml records in /tmp/<whatever> file; observe that at the end of many records, here and there some extra "see from" (= 1st indicator: 'z') fields tend to appear, which shouldn't be there ;) To test: 4) apply patch 5) redo 3) 6) compare results from 3) and 5) with diff -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14456 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14456 --- Comment #2 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 41127 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41127&action=edit [SIGNED OFF] Bug 14456: (regression test) Holdings fields should be skipped by EmbedSeeFromHeadings This patch introduces a regression test for the situation in which an itemnumber on a record matches the authid of an authority record and the current implementation of the EmbedSeeFromHeadings filter wrongly includes new holding fields with authority data... To test: - Apply the patch - Run: $ prove t/db_dependent/RecordProcessor_EmbedSeeFromHeadings.t => FAIL: The test 'Holdings fields not processed to introduce See-from heading' fails. Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14456 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14456 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14456 --- Comment #5 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 41142 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=41142&action=edit [PASSED QA] Bug 14456: EmbedSeeFromHeadings record filter shouldn't process MARC holding fields If the system preference IncludeSeeFromInSearches is enabled, records exported for zebra indexing are being additionally processed by EmbedSeeFromHeadings record filter (right now used only in rebuild_zebra.pl script). This filter embeds 'see from' fields (extracted from authority records linked with the given biblio via $9 subfields) into target MARC record, which is then subsequently indexed in zebra. Currently all fields containing $9 are getting the same exact treatment by this filter. But on the export stage when the filter is applied, MARC record being processed already does have holdings data fields added in the previous stage (usually 952 / 995, depending on the MARC format). Problem is that holdings data fields use to have $9 subfields in them as well (mapped to item.itemnumber by default). As a consequence, some (great many in the typical setup) records exported for zebra indexing may have surplus "see from" fields added erroneously in semi-random fashion, so biblio searches would often return some completely unexpected additional results. EmbedSeeFromHeadings record filter should not process holdings fields when dealing with MARC records intended for zebra indexing. To reproduce: 1) database with as many sample or real-world biblio, item and authority records as possible is recommended for testing purposes 2) enable IncludeSeeFromInSearches 3) export a bunch of biblio records for zebra (e.g.: misc/migration_tools/rebuild_zebra.pl -I -b -x -k -length=1000), inspect the result xml records in /tmp/<whatever> file; observe that at the end of many records, here and there some extra "see from" (= 1st indicator: 'z') fields tend to appear, which shouldn't be there ;) To test: 4) apply patch 5) redo 3) 6) compare results from 3) and 5) with diff Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> I introduced a regression test for this. You should run the tests without/with the patch and verify that the patch actually fixes the problem. Good job Jacek! I'm sure writing the regression test would take less time than such a detailed commit message! Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14456 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable CC| |chris@bigballofwax.co.nz --- Comment #7 from Chris Cormack <chris@bigballofwax.co.nz> --- Pushed to 3.20.x will be in 3.20.3 -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14456 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtj@kohaaloha.com --- Comment #9 from Mason James <mtj@kohaaloha.com> --- Pushed to 3.16.x, will be in 3.16.14 -- You are receiving this mail because: You are watching all bug changes.