[Bug 40304] New: Zebrasrv config doesn't consider non-AMD64 CPUs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Bug ID: 40304 Summary: Zebrasrv config doesn't consider non-AMD64 CPUs Change sponsored?: --- Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: trivial Priority: P5 - low Component: Searching - Zebra Assignee: koha-bugs@lists.koha-community.org Reporter: jake.deery@openfifth.co.uk QA Contact: testopia@bugs.koha-community.org Hi folks, As per this Mattermost conversation: https://chat.koha-community.org/koha-community/pl/ft6e47akutn6fbdg5tpwab5zbw I've identified an issue where running t/db_dependent/Search.t fails on AARCH64 machines. Upon investigation, I've found that in the default zebra-biblios-dom.cfg and zebra-authorities-dom.cfg files, the module search path does not include as many architectures as it should. I have rectified this, by copying the lines found in the debian/templates folder to the relevant config files mentioned in the last paragraph. Could I ask someone to check the patch I'm about to push, so we can get a quick conclusion to this bug? Ta, Jake -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #1 from Jake Deery <jake.deery@openfifth.co.uk> --- Created attachment 183783 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183783&action=edit Bug 40304: Zebrasrv configs now work on ARM CPUs This patch adds more module search paths to the zebra-biblios-dom.cfg and zebra-authorities-dom.cfg, so that a larger range of CPU architectures will work with Zebra search unit tests. This will not change Debian package installs, but it will fix any git-based installs. TO TEST (you'll need access to an ARM-based CPU, and, preferably, an x86-64 machine): ON THE ARM-BASED MACHINE: a) start ktd, then ktd --shell b) in ktd, run: prove -v t/db_dependent/Search.t c) notice how the test now fails APPLY PATCH d) rerun step b e) notice how the test now passes ON THE X86-64-BASED MACHINE: f) run steps a-e g) notice how the test passes, both before and after applying the patch SIGN OFF -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Jake Deery <jake.deery@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Jake Deery <jake.deery@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |jake.deery@openfifth.co.uk |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #183783|0 |1 is obsolete| | --- Comment #2 from Paul Derscheid <paul.derscheid@lmscloud.de> --- Created attachment 183784 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183784&action=edit Bug 40304: Zebrasrv configs now work on ARM CPUs This patch adds more module search paths to the zebra-biblios-dom.cfg and zebra-authorities-dom.cfg, so that a larger range of CPU architectures will work with Zebra search unit tests. This will not change Debian package installs, but it will fix any git-based installs. TO TEST (you'll need access to an ARM-based CPU, and, preferably, an x86-64 machine): ON THE ARM-BASED MACHINE: a) start ktd, then ktd --shell b) in ktd, run: prove -v t/db_dependent/Search.t c) notice how the test now fails APPLY PATCH d) rerun step b e) notice how the test now passes ON THE X86-64-BASED MACHINE: f) run steps a-e g) notice how the test passes, both before and after applying the patch SIGN OFF Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@openfifth.c | |o.uk, mtj@kohaaloha.com, | |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au Status|Signed Off |Failed QA --- Comment #3 from David Cook <dcook@prosentient.com.au> --- Sorry to do this guys but I think that this introduces a regression. The path /usr/lib64/idzebra-2.0/modules is removed by the patch, but that's the path used by OSes like OpenSUSE (and I think Fedora/RHEL as well although I don't have any of those on-hand atm). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #4 from Jake Deery <jake.deery@openfifth.co.uk> --- Hi David, Thanks for looking. If this is indeed a problem, Debian packages will also have this broken behaviour. Perhaps this isn't an issue, as Debian packages only go on Debian systems, but nonetheless, perhaps it wouldn't hurt fixing this in both places? I shall test both eventualities, and push a new patch shortly. Ta, Jake. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Jake Deery <jake.deery@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #183784|0 |1 is obsolete| | --- Comment #5 from Jake Deery <jake.deery@openfifth.co.uk> --- Created attachment 183831 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=183831&action=edit Bug 40304: Zebrasrv configs now work on ARM CPUs This patch adds more module search paths to the zebra-biblios-dom.cfg and zebra-authorities-dom.cfg, so that a larger range of CPU architectures will work with Zebra search unit tests. This will not change Debian package installs, but it will fix any git-based installs. TO TEST (you'll need access to an ARM-based CPU, and, preferably, an x86-64 machine): ON THE ARM-BASED MACHINE: a) start ktd, then ktd --shell b) in ktd, run: prove -v t/db_dependent/Search.t c) notice how the test now fails APPLY PATCH d) rerun step b e) notice how the test now passes ON THE X86-64-BASED MACHINE: f) run steps a-e g) notice how the test passes, both before and after applying the patch SIGN OFF Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Jake Deery <jake.deery@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off --- Comment #6 from Jake Deery <jake.deery@openfifth.co.uk> --- That's that change implemented. I've put /usr/lib64/idzebra-2.0/modules first in the sequence, as most folk will be running x86-64, and so it makes sense for Zebrasrv to search there first. Tested on ARM64 and x86-64. If anyone has a MIPS or POWERPC platform lying around, it would be great to get a thumbs up from you folk too – though, it isn't essential at this stage. Ta, Jake. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #7 from David Cook <dcook@prosentient.com.au> --- (In reply to Jake Deery from comment #4)
Thanks for looking. If this is indeed a problem, Debian packages will also have this broken behaviour. Perhaps this isn't an issue, as Debian packages only go on Debian systems, but nonetheless, perhaps it wouldn't hurt fixing this in both places?
(In reply to Jake Deery from comment #6)
That's that change implemented. I've put /usr/lib64/idzebra-2.0/modules first in the sequence, as most folk will be running x86-64, and so it makes sense for Zebrasrv to search there first.
Tested on ARM64 and x86-64. If anyone has a MIPS or POWERPC platform lying around, it would be great to get a thumbs up from you folk too – though, it isn't essential at this stage.
Sorry I should've been more precise before. On 64bit Debian/Ubuntu, you'll find the Zebra modules in /usr/lib/x86_64-linux-gnu/idzebra-2.0/modules/. On 64bit OpenSUSE/SLES (and maybe the Red Hat systems), you'll find them in /usr/lib64/idzebra-2.0/modules/. So the Debian packages didn't have broken behaviour, since - as you say - they'd only be on Debian-based systems. So it would make sense for them to have Debian-specific paths. Your patch that I failed removed /usr/lib64 from the generic Zebra conf, which would create a regression on non-Debian-based systems. I won't fail this latest patch, but you might want to consider not including /usr/lib64 in the Debian package conf (I like the idea of using /usr/lib/x86_64-linux-gnu/idzebra-2.0/modules first in the modulePath config though - as you say it makes sense to search there first.). But the ones in the "./etc" path should support everything since they could be deployed anywhere. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #8 from David Cook <dcook@prosentient.com.au> --- If we want to double-check RPM packaged Zebra, we can look at the following: https://github.com/indexdata/idzebra/blob/master/idzebra.spec There we see the use of the %_libdir macro, so then we do some Googling. I won't include the links here because then Bugzilla will auto-ban me for being a spammer :). But for opensuse, it shows %_libdir goes into /usr/lib(64) For Fedora, it says %_libdir goes to /usr/%{_lib} and %{_lib} is lib64 (or just lib on 32bit platforms. Of course, they're probably assuming x86 for all those... hmmm. -- How'd you install the Zebra on the ARM system? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #9 from David Cook <dcook@prosentient.com.au> --- Okie dokes... I looked up libidzebra-2.0-mod-dom on packages.debian.org and looking at the "list of files" I see all the different paths. That's cool. -- Looking again at OpenSUSE/Fedora... I don't think they include the CPU architecture name in the path. They just do 32bit vs 64bit. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #10 from Jake Deery <jake.deery@openfifth.co.uk> --- Thanks for looking again, David. I did consider leaving the Debian-specific zebra-*-dom.cfg files as they were, but decided to include /usr/lib64, as: * /usr/lib64 does exist on Debian-based systems (even if it isn't used by default) and so may be used if someone decided to compile and install Zebrasrv themselves * It is, from my reading and understanding, a Linux standard path, and so I felt like it should be included for compatibility. Thanks for your time on this. I'll try to get a Mac user's QA on this one today. Ta, Jake. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #11 from David Cook <dcook@prosentient.com.au> --- (In reply to Jake Deery from comment #10)
Thanks for looking again, David. I did consider leaving the Debian-specific zebra-*-dom.cfg files as they were, but decided to include /usr/lib64, as:
* /usr/lib64 does exist on Debian-based systems (even if it isn't used by default) and so may be used if someone decided to compile and install Zebrasrv themselves
* It is, from my reading and understanding, a Linux standard path, and so I felt like it should be included for compatibility.
Oh snap, you're right. I think I'd just looked at Debian 11 and Ubuntu 22.04. I looked at Ubuntu 24.04 just now and I see that there is a /usr/lib64. It's not really used for much except it has a symlink from /usr/lib64/ld-linux-x86-64.so.2 to /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2. But still... fair enough!
Thanks for your time on this. I'll try to get a Mac user's QA on this one today.
Oh you know me... I like the weird stuff haha. (I'm also low key interested in Koha having ARM support.) I didn't think that Mac could run Koha though? I thought Mac users run Koha in Linux virtual machines? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #12 from Jake Deery <jake.deery@openfifth.co.uk> --- Hi David, That's right, but that Linux image is ARM-based Debian 12 (up until about a year ago we were emulating x86-64 due to constraints with Docker – absolute madness). It's been us folk on those ARM-based Macs that noticed this bug in the first place, I think. Sussing out this problem is going to be a huge bonus for running Koha against ARM in Jenkins, too! Ta, Jake. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #13 from David Cook <dcook@prosentient.com.au> --- (In reply to Jake Deery from comment #12)
That's right, but that Linux image is ARM-based Debian 12 (up until about a year ago we were emulating x86-64 due to constraints with Docker – absolute madness).
It's been us folk on those ARM-based Macs that noticed this bug in the first place, I think. Sussing out this problem is going to be a huge bonus for running Koha against ARM in Jenkins, too!
Yes, of course. I'm not a Mac user so I'd totally forgotten there were ARM-based Macs. Makes sense.
From time to time, I get the question about whether or not Koha can run on ARM, and my answer is usually something like "I don't think it can 100%". But I'd love for that to change to "Yes, definitely."
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #14 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- (In reply to David Cook from comment #13)
(In reply to Jake Deery from comment #12)
That's right, but that Linux image is ARM-based Debian 12 (up until about a year ago we were emulating x86-64 due to constraints with Docker – absolute madness).
It's been us folk on those ARM-based Macs that noticed this bug in the first place, I think. Sussing out this problem is going to be a huge bonus for running Koha against ARM in Jenkins, too!
Yes, of course. I'm not a Mac user so I'd totally forgotten there were ARM-based Macs. Makes sense.
From time to time, I get the question about whether or not Koha can run on ARM, and my answer is usually something like "I don't think it can 100%". But I'd love for that to change to "Yes, definitely."
It does run. Been using an ARM mac daily since 2023 :-D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #183831|0 |1 is obsolete| | --- Comment #15 from Tomás Cohen Arazi (tcohen) <tomascohen@gmail.com> --- Created attachment 184240 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=184240&action=edit Bug 40304: Zebrasrv configs now work on ARM CPUs This patch adds more module search paths to the zebra-biblios-dom.cfg and zebra-authorities-dom.cfg, so that a larger range of CPU architectures will work with Zebra search unit tests. This will not change Debian package installs, but it will fix any git-based installs. TO TEST (you'll need access to an ARM-based CPU, and, preferably, an x86-64 machine): ON THE ARM-BASED MACHINE: a) start ktd, then ktd --shell b) in ktd, run: prove -v t/db_dependent/Search.t c) notice how the test now fails APPLY PATCH d) rerun step b e) notice how the test now passes ON THE X86-64-BASED MACHINE: f) run steps a-e g) notice how the test passes, both before and after applying the patch SIGN OFF Signed-off-by: Paul Derscheid <paul.derscheid@lmscloud.de> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Lucas Gass (lukeg) <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |25.11.00 released in| | Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #16 from Lucas Gass (lukeg) <lucas@bywatersolutions.com> --- Nice work everyone! Pushed to main for 25.11 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to main |Pushed to stable Version(s)|25.11.00 |25.11.00,25.05.02 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 --- Comment #17 from Paul Derscheid <paul.derscheid@lmscloud.de> --- Nice work everyone! Pushed to 25.05.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m Version(s)|25.11.00,25.05.02 |25.11.00,25.05.02,24.11.08 released in| | Status|Pushed to stable |Pushed to oldstable --- Comment #18 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to 24.11.x for 24.11.08 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=40304 Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldstable |Needs documenting --- Comment #19 from Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> --- Not backporting to 22.11.x as it wasn't backported to 24.05.x -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org