[Bug 38375] New: SIP2 syspref SIP2SortBinMapping is not working
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Bug ID: 38375 Summary: SIP2 syspref SIP2SortBinMapping is not working Change sponsored?: --- Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: SIP2 Assignee: koha-bugs@lists.koha-community.org Reporter: lmstrand@gmail.com QA Contact: testopia@bugs.koha-community.org Sort bin rules in system preference SIP2SortBinMapping are not being used by the SIP server. This functionality has broken between the 23.11 and 24.05 versions. We are adding the mappings directly to the SIP2 server configuration XML file as a workaround in 24.05 for now since sort bin mappings still work using the sip config file. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Ewa Gózd <ewa.gozd@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ewa.gozd@gmail.com Keywords| |regression -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Janne Seppänen <janne.seppanen@lahti.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |janne.seppanen@lahti.fi --- Comment #1 from Janne Seppänen <janne.seppanen@lahti.fi> --- I can confirm that the system preference SIP2SortBinMapping is currently not working in 24.05. We had built our entire sorting system around this feature in 23.11 and it worked fine, but after the 24.05 update the SIP messages that Koha sends do not include the CL field at all. As we make regular updates to sorting rules (e.g. with seasonal materials) it's not a sustainable way to do this manually on the server side on all self-service stations, one by one. This is an important feature to us, and we hope that the bug could be found and fixed soon. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |RM_priority Severity|normal |critical -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |CONFIRMED CC| |dcook@prosentient.com.au --- Comment #2 from David Cook <dcook@prosentient.com.au> --- Marking as CONFIRMED on behalf of Janne -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Jan Kissig <bibliothek@th-wildau.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bibliothek@th-wildau.de --- Comment #3 from Jan Kissig <bibliothek@th-wildau.de> --- I just tried to confirm the bug on my ktd setup (v24.05.05) to dig into it but in my case it worked: SIP2SortBinMapping: CPL:itype:eq:BK:5 kohadev-koha@kohadevbox:misc((v24.05.05))$ perl sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su term1 -sp term1 -l CPL -m checkout --item 39999000000498 --patron 42 Attempting socket connection to 127.0.0.1:6001...connected! Use of uninitialized value $hold_mode in string eq at sip_cli_emulator.pl line 123. SEND: 9300CNterm1|COterm1|CPCPL| READ: 941 Trying 'checkout' SEND: 11YN20250104 19020520250104 190205AOCPL|AA42|AB39999000000498|ACterm1|BIN| READ: 121NNY20250104 190205AOCPL|AA42|AB39999000000498|AJCGI programming with Perl /|AH20250109 235900| kohadev-koha@kohadevbox:misc((v24.05.05))$ perl sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su term1 -sp term1 -l CPL -m checkin --item 39999000000498 --patron 42 Attempting socket connection to 127.0.0.1:6001...connected! Use of uninitialized value $hold_mode in string eq at sip_cli_emulator.pl line 123. SEND: 9300CNterm1|COterm1|CPCPL| READ: 941 Trying 'checkin' SEND: 09N20250104 19020620250104 190206APCPL|AOCPL|AB39999000000498|ACterm1|BIN| READ: 101YNN20250104 190207AOCPL|AB39999000000498|AQCPL|AJCGI programming with Perl /|CL5|AA42|CRREF| --- In the checkin response the mentioned field CL is set to 5. Also tried on main (24.12.) and it worked. Can you give a little more info on your sort bin rules? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|CONFIRMED |In Discussion -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #4 from Janne Seppänen <janne.seppanen@lahti.fi> --- Hmm, that's strange. We don't have access to an emulator to check if would differ from a self-service station in our environment, but we have a similar result from two different libraries using stations from different manufacturers. Our rules are extensive, but we have also tried to leave only one row/rule to the Sip2SortBinMapping syspref without any impact to the behaviour. We also use comments for structuring the rules (there are around 400 rules in total for our library consortium) but they don't seem to be cause. Here are a few examples: # Varastoaineisto LAPK:location:eq:AIV:03 LAPK:location:eq:MUV:03 LAPK:location:eq:LAV:03 LAPK:location:eq:NUV:03 # Omat lehdet (jää palautusalueelle) LAPK:itype:eq:14VRKLEHTI:homebranch:eq:LAPK:00 LAPK:itype:eq:14VRKLEHLA:homebranch:eq:LAPK:00 We've tried to remove the leading zero to no effect. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #5 from Jan Kissig <bibliothek@th-wildau.de> --- (In reply to Janne Seppänen from comment #4)
Hmm, that's strange. We don't have access to an emulator to check if would differ from a self-service station in our environment, but we have a similar result from two different libraries using stations from different manufacturers.
Our rules are extensive, but we have also tried to leave only one row/rule to the Sip2SortBinMapping syspref without any impact to the behaviour. We also use comments for structuring the rules (there are around 400 rules in total for our library consortium) but they don't seem to be cause.
Here are a few examples:
# Varastoaineisto LAPK:location:eq:AIV:03 LAPK:location:eq:MUV:03 LAPK:location:eq:LAV:03 LAPK:location:eq:NUV:03
# Omat lehdet (jää palautusalueelle) LAPK:itype:eq:14VRKLEHTI:homebranch:eq:LAPK:00 LAPK:itype:eq:14VRKLEHLA:homebranch:eq:LAPK:00
We've tried to remove the leading zero to no effect.
You can find the emulator directly on your koha server: /usr/share/koha/bin/sip_cli_emulator.pl -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #6 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I am not sure if this needs to be done, but have you tried restarting the SIP server after changing the values? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #7 from Lari Strand <lmstrand@gmail.com> --- Our 09 checkin messages are in a form like this: 09N20250116 16144920250116 161449AP|AOORPK|ABOM0L154262|AC|AY5AZEFAB Maybe it has something to do with the message structure that breaks stuff. Could this be translated to the sip emulator for verifying the issue exists with this message structure? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #8 from Jan Kissig <bibliothek@th-wildau.de> --- In your checkin the AP is empty: currentLocation AP= institutionId AO=ORPK Maybe you can check via emulator when setting -l ORPK so that the whole request looks like: perl sip_cli_emulator.pl -a 127.0.0.1 -p 6001 -su USER -sp PASS -l ORPK -m checkin --item OM0L154262 if the response includes a CL field. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #9 from Lari Strand <lmstrand@gmail.com> --- It seems parsing the mappings break when there are more than one line of them defined in SIP2SortBinMapping syspref. This gets logged in case of more than one line of mappings: [2025/01/20 14:25:02] [3705123] [WARN] TESTSIP1@10.0.2.202: Malformed preference line found: 'LAPK:location:eq:MUV:02 LAPK:location:eq:AIV:03' at /home/koha/Koha/C4/SIP/ILS/Transaction/Checkin.pm line 282, <STDIN> line 2. If there is only one line, the line's mapping rule works and the CL field gets set accordingly. I tried to hunt down a commit that's breaking the parsing of the syspref but I only found bug 34737 that touches the sub _get_sort_bin and reverting the commits did not resolve the issue with multiple lines of sort mappings. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #10 from Lari Strand <lmstrand@gmail.com> --- So it seems the issue is that splitting the lines does not happen unless the syspref is written in windows with a carriage return + newline \r\n? We should also support UNIX LF. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #11 from Lari Strand <lmstrand@gmail.com> --- "Windows typically uses CRLF (\r\n), macOS uses CR (\r), and Linux/Unix systems use LF (\n)". We need to support macOS \r as well. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Magnus Enger <magnus@libriotech.no> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |magnus@libriotech.no -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #12 from Magnus Enger <magnus@libriotech.no> --- Sounds like this is the problem, then: # Get the mapping and split on newlines my $raw_map = C4::Context->preference('SIP2SortBinMapping'); return unless $raw_map; @lines = split /\r\n/, $raw_map; https://git.koha-community.org/Koha-community/Koha/src/commit/bb2e88a1e3ebbe... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #13 from Magnus Enger <magnus@libriotech.no> --- And it looks like "split /\R/" should do the trick? POC: #!/usr/bin/perl use Modern::Perl; foreach my $string ( "a\r\nb", "a\rb", "a\nb" ) { say "=========="; say $string; say "----------"; my ( $a, $b ) = split /\R/, $string; say "$a + $b"; } https://perldoc.perl.org/perlrebackslash#Misc -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #14 from Lari Strand <lmstrand@gmail.com> --- Created attachment 176853 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=176853&action=edit Bug 38375: Splitting SIP2SortBinMapping syspref does not support all operating systems' line endings -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #15 from Lari Strand <lmstrand@gmail.com> --- Maybe someone can help with a test plan if this needs one. I'll just post the patch here. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #16 from Magnus Enger <magnus@libriotech.no> --- It would have been nice to have the relevant code in a subroutine, so we could have tests for it. Not sure if we can test it with different line endings, without testing in three different OSes? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #17 from Lari Strand <lmstrand@gmail.com> --- Could we generate the rules like in the existing test, but with different line endings and rerun the same tests again? existing test (Transactions.t line 921): my $rules = <<"RULES"; $branch:homebranch:ne:\$holdingbranch:X\r $branch:effective_itemtype:eq:CD:0\r $branch:itemcallnumber:<:340:1\r $branch:itemcallnumber:<:370:2\r $branch:itemcallnumber:<:600:3\r $branch2:homebranch:ne:\$holdingbranch:X\r $branch2:effective_itemtype:eq:CD:4\r $branch2:itemcallnumber:>:600:5\r RULES new test for unix: $branch:homebranch:ne:\$holdingbranch:X\n $branch:effective_itemtype:eq:CD:0\n $branch:itemcallnumber:<:340:1\n $branch:itemcallnumber:<:370:2\n $branch:itemcallnumber:<:600:3\n $branch2:homebranch:ne:\$holdingbranch:X\n $branch2:effective_itemtype:eq:CD:4\n $branch2:itemcallnumber:>:600:5\n RULES -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #18 from Lari Strand <lmstrand@gmail.com> --- Oops I was watching the wrong Koha version. The test rules are found at /Koha/t/db_dependent/SIP/Transaction.t line 1108 forwards in main branch. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |clemens.tubach@kit.edu, | |michaela.sieber@kit.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |martin.renvoize@ptfs-europe | |.com --- Comment #19 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Yikes, this is nasty.. my apologies for not thinking about line-ending differences sooner. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #176853|0 |1 is obsolete| | --- Comment #20 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Created attachment 178317 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=178317&action=edit Bug 38375: Splitting SIP2SortBinMapping syspref does not support all operating systems' line endings Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #21 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Created attachment 178318 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=178318&action=edit Bug 38375: Use perl 5.10+ \R -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #22 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Created attachment 178319 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=178319&action=edit Bug 38375: Unit tests -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Passed QA QA Contact|testopia@bugs.koha-communit |martin.renvoize@ptfs-europe |y.org |.com -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |lmstrand@gmail.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |25.05.00 released in| | Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #23 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Pushed for 25.05! Well done everyone, thank you! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|regression, RM_priority |rel_24_11_candidate -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|25.05.00 |25.05.00,24.11.03 released in| | Status|Pushed to main |Pushed to stable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 --- Comment #24 from Paul Derscheid <paul.derscheid@lmscloud.de> --- Nice work everyone! Pushed to 24.11.x for 24.11.03 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |paul.derscheid@lmscloud.de Keywords|rel_24_11_candidate | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #25 from Fridolin Somers <fridolin.somers@biblibre.com> --- Needed for 24.05.x ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38375 Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Needs documenting CC| |wainuiwitikapark@catalyst.n | |et.nz --- Comment #26 from Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> --- Not backporting to 22.11 as it wasn't backported to 24.05, 23.11 or 23.05 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org