[Bug 9256] New: debian/templates/koha-conf-site.xml.in is hardcoded to retrieval syntax = usmarc
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Bug ID: 9256 Summary: debian/templates/koha-conf-site.xml.in is hardcoded to retrieval syntax = usmarc Classification: Unclassified Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: major Priority: P5 - low Component: Packaging Assignee: koha-bugs@lists.koha-community.org Reporter: magnus@enger.priv.no CC: robin@catalyst.net.nz Bug 3087 updated koha-conf.xml so the MARC flavour is configurable: - <backend syntax="usmarc" name="F"> + <backend syntax="__ZEBRA_MARC_FORMAT__" name="F"> But debian/templates/koha-conf-site.xml.in was not updated, so it is still hardcoded to usmarc: <retrieval syntax="usmarc" name="F"/> <retrieval syntax="usmarc" name="B"/> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 --- Comment #1 from Magnus Enger <magnus@enger.priv.no> --- There might be similar issues with the other config files affected by bug 3087: C4/Context.pm | 2 + Makefile.PL | 1 + etc/koha-conf.xml | 46 +++++++++++++++--------------- etc/zebradb/etc/usmarc.mar | 4 +- etc/zebradb/retrieval-info-auth-dom.xml | 4 +- etc/zebradb/retrieval-info-auth-grs1.xml | 14 ++++---- etc/zebradb/retrieval-info-bib-dom.xml | 4 +- etc/zebradb/retrieval-info-bib-grs1.xml | 14 ++++---- 8 files changed, 46 insertions(+), 43 deletions(-) -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|major |critical --- Comment #2 from Magnus Enger <magnus@enger.priv.no> --- I'm increasing the severity: as far as I can tell the packages are broken for marcflavour != marc21 now. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Serhij Dubyk <dubyk@library.lviv.ua> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dubyk@library.lviv.ua -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |chris@bigballofwax.co.nz --- Comment #3 from Chris Cormack <chris@bigballofwax.co.nz> --- Magnus, by changing that file manually on your 3.10.0 install did it fix the issue? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 --- Comment #4 from Robin Sheat <robin@catalyst.net.nz> --- I'm a little confused, but it might be my lack of understanding of how this works. My assumption is: __ZEBRA_MARC_FORMAT__ comes out as marc21, normarc, whathaveyou. However usmarc is the binary MARC format, not the schema that lies on top of that. So, wouldn't changing "usmarc" to "normarc", or even "marc21" break things? However, clearly this is wrong, I just don't know what the right version is. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 --- Comment #5 from Magnus Enger <magnus@enger.priv.no> --- I'm afraid I don't completely understand this either, and I have not had the time to investigate thoroughly enough, but it seems pretty clear to me that the changes brought on by bug 3087 needs to be reflected in the koha-conf.xml and zebra config files created by the packages. What I did find out was that if I changed marcflavour from normarc to marc21 and then reindexed, searching worked. If I changed it back to normarc again and reindexed, it did not work. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 --- Comment #6 from Magnus Enger <magnus@enger.priv.no> --- I got it to work now, by reproducing the changes introduced by bug 3087 on the relevant config files. According to my notes, I did something like this: sed -i "s/syntax=\"usmarc\"/syntax=\"normarc\"/" /etc/koha/sites/esme/koha-conf.xml sed -i "s/marc21/normarc/" /etc/koha/zebradb/etc/usmarc.mar sed -i "s/syntax=\"marc21\"/syntax=\"normarc\"/" /etc/koha/zebradb/retrieval-info-bib-dom.xml But my notes might be missing things... So it looks to me like the packages will need to do the same kinds of substitutions that bug 3087 introduced. There is one big question, though: Since I had to edit non-instance specific files like /etc/koha/zebradb/retrieval-info-bib-dom.xml - does that mean we will only be able to support one MARC dialect on one server? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|debian/templates/koha-conf- |Search is broken for 3.10 |site.xml.in is hardcoded to |package-installs when |retrieval syntax = usmarc |marcflavour != marc21 --- Comment #7 from Magnus Enger <magnus@enger.priv.no> --- Trying to find a more descriptive (and fetching) title. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 --- Comment #8 from Chris Cormack <chris@bigballofwax.co.nz> ---
There is one big question, though: Since I had to edit non-instance specific files like /etc/koha/zebradb/retrieval-info-bib-dom.xml - does that mean we will only be able to support one MARC dialect on one server?
If that is true bug 3087 should be reverted. We have broken Koha to get an z3950 server going for external use. We should not break core like that. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |3087 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Paul Poulain <paul.poulain@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |paul.poulain@biblibre.com -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 --- Comment #9 from Magnus Enger <magnus@enger.priv.no> --- So I finally got around to spending some more time on this, and looks like there are three problems here, from the point of view of a package install: * 1. usmarc.mar After bug 3087 this file gets the marcflavour of the installation. I think the solution is to have one file for each marcflavour, e.g. marc21.mar, normarc.mar and unimarc.mar. usmarc.mar is referenced by the different record.abs files for the different marcflavours, so updating them to point to the right .mar file for their flavour should be trivial. Side note: I'm not sure what role the .mar file actually plays in Zebra/Koha. I have tried changing the contents around a bit without any apparent influence on searching. The only documentation I have been able to find is this: 3.9. The MARC (ISO2709) Representation (.mar) Files This file provides rules for representing a record in the ISO2709 format. The rules pertain mostly to the values of the constant-length header of the record. http://www.indexdata.com/zebra/doc/grs-conf.html#grs-mar-files Perhaps Robin is right too, in that it could have been left as it was before Bug 3087? * 2. koha-conf.xml Occurences of hardcoded "usmarc" in debian/templates/koha-conf-site.xml.in need to be replaced with the __ZEBRA_MARC_FORMAT__ placeholder, so it can get the right values when a new site is created:
From <retrieval syntax="usmarc" name="F"/> To <retrieval syntax="__ZEBRA_MARC_FORMAT__" name="F"/>
* 3. retrieval-info-*-*.xml To be exact: retrieval-info-auth-dom.xml retrieval-info-auth-grs1.xml retrieval-info-bib-dom.xml retrieval-info-bib-grs1.xml Similar to koha-conf.xml, the __ZEBRA_MARC_FORMAT__ placeholders in these files now get replaced with the actual marcflavour in a regular installation. In a dev install you will have one copy of each of these files for each of your installations, so verything works as expected. Not so for package installs, where these files are not genereated from templates and so continue to have the hardcoded "usmarc". Another major problem is that these files are common to all the instances on a server (in /etc/koha/zebradb). So it looks like we either have to - make the packages generate these files from templates whenever a new instance is added, and find a way to make them instance-sepecific (not shared), or - create one file for each marcflavour and make whatever references these files reference the files for the right marcflavour As far as I can tell, the files that reference these files are the koha-conf.xml files. It looks like it should not be too hard to make the retrieval-info-*.xml files instance specific and generate them from a template. That way we just have two sets of template files to keep updated, and not 3 versions of each of the files, for different marcflavours. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 --- Comment #10 from Robin Sheat <robin@catalyst.net.nz> --- What you're saying doesn't seem bad or too difficult. One point though: where possible, I'd like to centralise the files as much as reasonable. That is, rather than create an instance-specific version, it'd be preferable to have a file for all the combinations. We could generated these as part of the build process, if need be. My reasoning for this is simply that this means they'll get upgraded as koha gets upgraded. There will be situations where this really isn't feasible, but if it's just a handful more files, sensibly named, it should be OK. If we start having dozens of them (especially if they have to be maintained manually) then it is probably time to template a per-instance one. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |magnus@enger.priv.no |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 --- Comment #11 from Magnus Enger <magnus@enger.priv.no> --- Created attachment 15906 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=15906&action=edit Bug 9256 - Fix search for the packages See the bug for a description of the problem. This patch tries to restore searching for marcflavour != MARC21 as well as allowing instances with different marcflavors to co-exist on the same server. To test: - Do a package install with e.g. the official squeeze-dev packages and create at least two instances, with different marcflavours, e.g.: sudo koha-create --create-db --marcflavor marc21 test1 sudo koha-create --create-db --marcflavor normarc test2 - Run through the web installers for both instances and add a couple of records to each. Wait for the records to be indexed or run indexing manually with sudo koha-rebuild-zebra -f test1 sudo koha-rebuild-zebra -f test2 - Try searching for the records you added. It should work in test1 but not in test2. - Apply the patch and build packages with the build-git-snapshot script - Install the new koha-common package - Create two instances (because of Bug 9754 it is probably best to give the instances different names than the ones you created above, or to do this on a fresh VM or similar) and add records, as described above. Searching should now work equally well for both instances. Please note: Because of Bug 9752 you will have to set marcflavour = NORMARC by hand before you do the searching, if you choose NORMARC as the marc flavour on one of the instances you create. Please note too: I am not confident that this is the perfect solution, so merciless and thorough testing is necessary! ;-) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 --- Comment #12 from Magnus Enger <magnus@enger.priv.no> --- Oh, testing that this does not bring back the problems described by Bug 3087 is probably a good idea too... -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Mirko Tietgen <mirko@abunchofthings.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mirko@abunchofthings.net --- Comment #13 from Mirko Tietgen <mirko@abunchofthings.net> --- Installed Koha-common 3.10.3-1 created instances test1 and test2 with Marc21 and Normarc respectively installed Norwegian language for test2 so I can choose Normarc in the web installer ran web installer created a staff user changed marcflavor to Normarc for test2 added some biblios, reindexed searched test1 (marc21), all looks ok searched test2 (normarc), got message »Feil: Can't call method "raw" on an undefined value at /usr/share/koha/lib/C4/Search.pm line 471.« Upgraded to my own package of master+9256 created instances test3 and test4 with Marc21 and Normarc respectively installed Norwegian language for test4 so I can choose Normarc in the web installer (needs to be done again or the layouts are broken!) ran web installer created a staff user changed marcflavor to Normarc for test4 Unicode is broken in the patron type pulldown when you try to create a new user. I have not looked around more, instead I tried searching. added some biblios, reindexed searched test3 (marc21), all looks ok searched test4 (normarc), all looks ok. (if you forget to change marcflavor syspref to Normarc, you will get the same error as described for test2 instance) So the patch seems to do what it is intended to do. I do not recall having seen broken Unicode when creating a patron in the unpatched version (maybe I didn't pay attention) and I want to have another look at it later but so far it's looking good. I have a repository with the package if somebody else would like to try it but does not feel like going through the pain of building it themselves. I won't be able to test for Unimarc problems for instance. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 --- Comment #14 from Magnus Enger <magnus@enger.priv.no> --- (In reply to comment #13)
Unicode is broken in the patron type pulldown when you try to create a new user. I have not looked around more, instead I tried searching. ... So the patch seems to do what it is intended to do. I do not recall having seen broken Unicode when creating a patron in the unpatched version (maybe I didn't pay attention) and I want to have another look at it later but so far it's looking good.
There is a signed off patch for the Unicode problem on Bug 9772. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Mirko Tietgen <mirko@abunchofthings.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #15906|0 |1 is obsolete| | --- Comment #15 from Mirko Tietgen <mirko@abunchofthings.net> --- Created attachment 16067 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16067&action=edit Bug 9256 - Fix search for the packages See the bug for a description of the problem. This patch tries to restore searching for marcflavour != MARC21 as well as allowing instances with different marcflavors to co-exist on the same server. To test: - Do a package install with e.g. the official squeeze-dev packages and create at least two instances, with different marcflavours, e.g.: sudo koha-create --create-db --marcflavor marc21 test1 sudo koha-create --create-db --marcflavor normarc test2 - Run through the web installers for both instances and add a couple of records to each. Wait for the records to be indexed or run indexing manually with sudo koha-rebuild-zebra -f test1 sudo koha-rebuild-zebra -f test2 - Try searching for the records you added. It should work in test1 but not in test2. - Apply the patch and build packages with the build-git-snapshot script - Install the new koha-common package - Create two instances (because of Bug 9754 it is probably best to give the instances different names than the ones you created above, or to do this on a fresh VM or similar) and add records, as described above. Searching should now work equally well for both instances. Please note: Because of Bug 9752 you will have to set marcflavour = NORMARC by hand before you do the searching, if you choose NORMARC as the marc flavour on one of the instances you create. Please note too: I am not confident that this is the perfect solution, so merciless and thorough testing is necessary! ;-) Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net> Works for me for GRS-1 (package installation out of the box). Could not figure out how to set up DOM indexing and eventually stopped caring about it. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Mirko Tietgen <mirko@abunchofthings.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA Patch complexity|--- |Medium patch -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16067|0 |1 is obsolete| | --- Comment #16 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 16410 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16410&action=edit [PASSED QA] Bug 9256 - Fix search for the packages See the bug for a description of the problem. This patch tries to restore searching for marcflavour != MARC21 as well as allowing instances with different marcflavors to co-exist on the same server. To test: - Do a package install with e.g. the official squeeze-dev packages and create at least two instances, with different marcflavours, e.g.: sudo koha-create --create-db --marcflavor marc21 test1 sudo koha-create --create-db --marcflavor normarc test2 - Run through the web installers for both instances and add a couple of records to each. Wait for the records to be indexed or run indexing manually with sudo koha-rebuild-zebra -f test1 sudo koha-rebuild-zebra -f test2 - Try searching for the records you added. It should work in test1 but not in test2. - Apply the patch and build packages with the build-git-snapshot script - Install the new koha-common package - Create two instances (because of Bug 9754 it is probably best to give the instances different names than the ones you created above, or to do this on a fresh VM or similar) and add records, as described above. Searching should now work equally well for both instances. Please note: Because of Bug 9752 you will have to set marcflavour = NORMARC by hand before you do the searching, if you choose NORMARC as the marc flavour on one of the instances you create. Please note too: I am not confident that this is the perfect solution, so merciless and thorough testing is necessary! ;-) Signed-off-by: Mirko Tietgen <mirko@abunchofthings.net> Works for me for GRS-1 (package installation out of the box). Could not figure out how to set up DOM indexing and eventually stopped caring about it. Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Build packages with the patch and checked that creating instances and search within them works for both MARC21 and NORMARC. All tests and QA script pass. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Jared Camins-Esakov <jcamins@cpbibliography.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master --- Comment #17 from Jared Camins-Esakov <jcamins@cpbibliography.com> --- This patch has been pushed to master. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Chris Cormack <chris@bigballofwax.co.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable --- Comment #18 from Chris Cormack <chris@bigballofwax.co.nz> --- Pushed to 3.10.x will be in 3.10.4 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Stable |RESOLVED Resolution|--- |FIXED --- Comment #19 from Magnus Enger <magnus@enger.priv.no> --- Search works now! (Tested with packages built off current master.) -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |9996 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=12577 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9256 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also|http://bugs.koha-community. | |org/bugzilla3/show_bug.cgi? | |id=12577 | -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org