[Bug 35117] New: get_marc_components and get_marc_volumes should be more consistent
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Bug ID: 35117 Summary: get_marc_components and get_marc_volumes should be more consistent Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: martin.renvoize@ptfs-europe.com QA Contact: testopia@bugs.koha-community.org Depends on: 35099 Bug 35099 highlighted a number of inconsistencies between these two parallel methods.. we should bring them closer. Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35099 [Bug 35099] Cannot load records with invalid marcxml -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 --- Comment #1 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 157516 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=157516&action=edit Bug 35117: Improve consistency This patch improves the consistency between get_components_query and get_get_volumes_query methods so they both check for MARC21. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 --- Comment #2 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- No real test plan as such.. just brings a little more consistency between these two parallel/similar sets of code. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |martin.renvoize@ptfs-europe |ity.org |.com -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Bug 35117 depends on bug 35099, which changed state. Bug 35099 Summary: Cannot load records with invalid marcxml https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35099 What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #157516|0 |1 is obsolete| | --- Comment #3 from David Nind <david@davidnind.com> --- Created attachment 167327 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=167327&action=edit Bug 35117: Improve consistency This patch improves the consistency between get_components_query and get_get_volumes_query methods so they both check for MARC21. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #4 from David Nind <david@davidnind.com> --- I have no idea about the actual code, but the patch still applies and all the tests pass. prove t/* ..... All tests successful. Files=200, Tests=9395, 497 wallclock secs ( 1.81 usr 0.39 sys + 1068.18 cusr 200.62 csys = 1271.00 CPU) Result: PASS -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 --- Comment #5 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- I can't for the life of me remember what I was doing here at this point :(. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 --- Comment #6 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Martin Renvoize from comment #5)
I can't for the life of me remember what I was doing here at this point :(.
LOL -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- What about 001 not found when pref enabled? if ( C4::Context->preference('UseControlNumber') ) { my $pf001 = $marc->field('001') || undef; Then you go to: my ( $error, $query, $query_str ) = $builder->build_query_compat( undef, [$searchstr], undef, undef, [$sort], 0 ); while $searchstr is undefined.. We should probably not look for [ undef ] ? In that unusual case, we should be looking for the title, right? So the 001 defined test should go to the toplevel if ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |34482 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=34482 [Bug 34482] Improve MARC21 7xx linking fields in bibliographic records [Omnibus] -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |41713 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41713 [Bug 41713] warnings sometimes generated when using ShowComponentRecords feature -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=39287, | |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=29441, | |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=31608, | |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=43113, | |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=26480 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #167327|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 --- Comment #8 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 211456 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=211456&action=edit Bug 35117: Improve consistency This patch improves the consistency between get_components_query and get_get_volumes_query methods so they both check for MARC21. Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 --- Comment #9 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 211457 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=211457&action=edit Bug 35117: (QA follow-up) Search by title when there is no 001 to link on With UseControlNumber enabled, get_components_query and get_volumes_query only built a search when the record has a 001. get_components_query then went on to search for an undefined value, which warns: Use of uninitialized value in concatenation (.) or string at Koha/SearchEngine/Elasticsearch/QueryBuilder.pm while get_volumes_query returned nothing at all. Test for the 001 first, so that a record without one is searched for by title, as it is when UseControlNumber is disabled. Both methods now do the same. The previous patch added the bib-level limit to the title search of get_components_query without updating the test that checks it, so do that too, and add tests for a record without a 001 and for flavours other than MARC21. To test: 1. prove t/db_dependent/Koha/Biblio.t => SUCCESS: Tests pass! 2. Enable UseControlNumber and find a record that is a host item (analytics) whose record has no 001 => It shows its component parts, found by title, and there is no warning in the logs 3. Sign off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff --- Comment #10 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Hi Marcel, Thanks again for the review. I have rebased Nick's patch on current main and attached a follow-up for your point about a missing 001: with UseControlNumber on and no 001, get_components_query and get_volumes_query now fall back to the title search, as they do when UseControlNumber is off, so a search is never built from an undefined value. I added tests for it, and fixed a test the original patch had broken. While preparing this I went through the related bugs and found two overlaps worth agreeing on: - Bug 41713 fixes the same warnings from the other end, by passing an empty string when there is no search string, and changing detail.pl. With this follow-up a search string is always built, so I would keep the unit tests and Pedro's two points from 41713 and drop the "// ''" change. I have put the details on that bug. - Bug 39287 adds a more specific index for the volumes search, which is the other half of making the two methods consistent. I think it is worth getting that one moving too. One caveat on the title fallback: it uses the same title phrase search as UseControlNumber off, which bugs 43113 and 26480 report as too broad. I think a broad match is better than searching for an undefined value, but if you would rather return nothing when there is no 001 I am happy to change it. I am setting this back to Needs Signoff. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |44064 See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=44063 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=44064 [Bug 44064] Match linking $w on the (OrgCode)control number form when UseControlNumber is enabled -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|34482 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=34482 [Bug 34482] Improve MARC21 7xx linking fields in bibliographic records [Omnibus] -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 --- Comment #11 from David Nind <david@davidnind.com> --- I attempted to test: 1. The tests pass. 2. For step 2 in the QA follow-up patch, I could reproduce the warnings before the patch, but didn't get the Analytics/Component tab showing after the patch. I don't think I've set things up correctly - any hints would help. Testing notes (using KTD): 1. I set things up similar to what I did for bug 44063. 2. Start up KTD with Elasticsearch: ktd --search-engine es9 up 3. Set UseControlNumber to Use and ShowComponentRecords to both. 4. For the BKS framework make these tags and fields visible in the editor: - 001 - 003 - 773$w 5. Host record: - Programming Perl (262) - Edit record and click 003 to add a value 6. Component records (change 000 position 7 to a, click in the 003 to add a value, add 17259930 to 773$w): - CGI programming with Perl (11) - Effective Perl programming (139) - Intermediate Perl (126) 7. The host record (Programming Perl (262)) shows the Analytics link and the Components tab. 8. Remove the 001 value from the host record: ==> Result: No analytics link and components tab and warnings in the log files 8. Refresh the host records page in the staff interface and OPAC 8.1 Warnings in /var/log/koha/kohadev/plack-intranet-error.log [2026/11/10 21:14:58] [WARN] Use of uninitialized value in concatenation (.) or string at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm line 353. [2026/11/10 21:14:58] [WARN] Use of uninitialized value in concatenation (.) or string at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm line 353. [2026/11/10 21:14:58] [WARN] Use of uninitialized value $comp_query_str in concatenation (.) or string at /kohadevbox/koha/catalogue/detail.pl line 260. 8.2 Warnings in /var/log/koha/kohadev/plack-opac-error.log [2026/11/10 21:16:12] [WARN] Use of uninitialized value in concatenation (.) or string at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm line 353. [2026/11/10 21:16:12] [WARN] Use of uninitialized value in concatenation (.) or string at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm line 353. [2026/11/10 21:16:12] [WARN] Use of uninitialized value $comp_query_str in concatenation (.) or string at /kohadevbox/koha/opac/opac-detail.pl line 633. 9. After applying the patch, reindexing (koha-elasticsearch --rebuild -d -b -a kohadev) and refreshing the pages, I don't get the analytics link and components tab. I'm not sure how the host record would know about the component records if there is nothing to link them 8-). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35117 --- Comment #12 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Hi David, Thanks for testing, and sorry about that: step 2 of my test plan was incomplete. When the host has no 001, the components are found by title, using the Host-item index. That index holds the 773$a and 773$t of the component records, not the 773$w. With only 773$w set on your component records, nothing is found by title, which is expected. To test, keep your three component records and also put the host's title (as in its 245$a, for example "Programming Perl") in 773$t. Then remove the 001 from the host record: - Before the patch: no Analytics link or Components tab, and the warnings in the log. - With the patch: the Analytics link and the Components tab show, and there are no warnings. I will fix the test plan. -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org