[Bug 33418] New: Allow setting overlay_framework for connexion imports
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Bug ID: 33418 Summary: Allow setting overlay_framework for connexion imports Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: MARC Bibliographic record staging/import Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org Bug 15869 added the ability to specify frameworks when overlaying while importing, we should extend this to allow updating the framework for records imported from OCLC connexion -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 --- Comment #1 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 149203 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=149203&action=edit Bug 33419: POC untested This will require testing, and better instructions on how to match etc. To test: 1 - Supply a value for 'framework' and 'overlay_framework' in the connxion.conf file 2 - Setup connexion, there is some help here on testing values: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15720 3 - Test importing bibs that match, and that don't match 4 - Confirm the framework value is used for new records 5 - Confirm if framework is blank the default framework is used for new records 6 - Confirm if framework is not supplied the default framework is used for new records 7 - Confirm a matching record that overlays obeys the overlay_framework param 8 - Confirm overlayed record uses default framework if the param is blank 9 - Confirm the original records framework is retained if the param is not passed -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Jessie Zairo <jzairo@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jzairo@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- TO DO: cronjobs/import_webservice_batch.pl -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Patch complexity|--- |Trivial patch -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #149203|0 |1 is obsolete| | --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 162338 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162338&action=edit Bug 33418: Add overlay_framework option to connexion scripts This patch adds the option to specify a framework to be used when overlaying records from webservices/connexion To test: 1 - vim /etc/koha/sites/kohadev/connexion.cnf 2 - Set content: host: port: 8888 koha:http://localhost:8081 log:/var/log/koha/kohadev/connexion.log match:ISBN user:kohauser password:kohapass overlay_action:replace nomatch_action:create_new item_action:always_add import_mode:direct framework:BKS overlay_framework: debug:1 3 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach) 4 - On the command line: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 5 - In another terminal: cat bug_33418.test | nc -v localhost 8888 6 - It should report success and a biblionumber 7 - In Koha: Cataloguing->Manage staged MARC record 8 - View the most recent batch with file name (webservice) 9 - Confirm it was imported, no match 10 - Click 'View' under the 'Record' column 11 - Confirm record loads correctly and 'MARC framework' detail is 'Books, Booklets, Workbooks' 12 - On the terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 13 - It should succed 14 - View the new batch, confirm the record matched this time 15 - View the record details, confirm framework is now 'default' 16 - On the first terminal hit Ctrl+C to stop the daemon 16 - Edit connexion.cnf and change: import_mode:stage framework:ACQ overlay_framework:IR 17 - Restart daemon: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 18 - Delete the record created above 19 - On the second terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 20 - Confirm the batch is created, but not imported 21 - In terminal: perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 22 - Confirm batch imported, and record in ACQ framework 23 - In terminal: cat bug_33418.test | nc -v localhost 8888 perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 24 - Confirm batch added, record matched, record imported, and record now in Books framework 25 - Stop the deamon, edit connexion.cnf: import_mode:direct 26 - Start the daemon, and on other terminal repeat: cat bug_33418.test | nc -v localhost 8888 27 - Confirm record in Binders framework 28 - Set record framework to Books 29 - Stop daemon, edit cnf and remove 'overlay_framework' setting 30 - Start daemon and cat the file again 31 - Confirm the record remains in Books framework -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 --- Comment #4 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 162339 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162339&action=edit Test records for Connexion import -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Brendan Lawlor <blawlor@clamsnet.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Brendan Lawlor <blawlor@clamsnet.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162338|0 |1 is obsolete| | --- Comment #5 from Brendan Lawlor <blawlor@clamsnet.org> --- Created attachment 162471 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162471&action=edit Bug 33418: Add overlay_framework option to connexion scripts This patch adds the option to specify a framework to be used when overlaying records from webservices/connexion To test: 1 - vim /etc/koha/sites/kohadev/connexion.cnf 2 - Set content: host: port: 8888 koha:http://localhost:8081 log:/var/log/koha/kohadev/connexion.log match:ISBN user:kohauser password:kohapass overlay_action:replace nomatch_action:create_new item_action:always_add import_mode:direct framework:BKS overlay_framework: debug:1 3 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach) 4 - On the command line: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 5 - In another terminal: cat bug_33418.test | nc -v localhost 8888 6 - It should report success and a biblionumber 7 - In Koha: Cataloguing->Manage staged MARC record 8 - View the most recent batch with file name (webservice) 9 - Confirm it was imported, no match 10 - Click 'View' under the 'Record' column 11 - Confirm record loads correctly and 'MARC framework' detail is 'Books, Booklets, Workbooks' 12 - On the terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 13 - It should succed 14 - View the new batch, confirm the record matched this time 15 - View the record details, confirm framework is now 'default' 16 - On the first terminal hit Ctrl+C to stop the daemon 16 - Edit connexion.cnf and change: import_mode:stage framework:ACQ overlay_framework:IR 17 - Restart daemon: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 18 - Delete the record created above 19 - On the second terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 20 - Confirm the batch is created, but not imported 21 - In terminal: perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 22 - Confirm batch imported, and record in ACQ framework 23 - In terminal: cat bug_33418.test | nc -v localhost 8888 perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 24 - Confirm batch added, record matched, record imported, and record now in Books framework 25 - Stop the deamon, edit connexion.cnf: import_mode:direct 26 - Start the daemon, and on other terminal repeat: cat bug_33418.test | nc -v localhost 8888 27 - Confirm record in Binders framework 28 - Set record framework to Books 29 - Stop daemon, edit cnf and remove 'overlay_framework' setting 30 - Start daemon and cat the file again 31 - Confirm the record remains in Books framework Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Brendan Lawlor <blawlor@clamsnet.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |blawlor@clamsnet.org --- Comment #6 from Brendan Lawlor <blawlor@clamsnet.org> --- Thanks to your detailed instructions it was fun testing this bug. Test notes: Update system prefs to allow weak passwords Set up kohauser with Cataloging permissions, thanks for link to bug Follow step 18 to delete the record to avoid confusion\ Sometimes when trying to restart the daemon I would get an error like perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf Couldn't be a tcp server on port 8888: Address already in use IO::Socket::INET: Address already in use at misc/bin/connexion_import_daemon.pl line 209. but I found you can resolve this issue with sudo fuser 8888/tcp -k When I first tried the last step it overlaid the record with the default framework, but I realized that was because I did not remove the whole line overlay_framework. It took me a couple tries to realize I needed to remove the whole line, but I eventually got it. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162471|0 |1 is obsolete| | --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 166518 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=166518&action=edit Bug 33418: Add overlay_framework option to connexion scripts This patch adds the option to specify a framework to be used when overlaying records from webservices/connexion To test: 1 - vim /etc/koha/sites/kohadev/connexion.cnf 2 - Set content: host: port: 8888 koha:http://localhost:8081 log:/var/log/koha/kohadev/connexion.log match:ISBN user:kohauser password:kohapass overlay_action:replace nomatch_action:create_new item_action:always_add import_mode:direct framework:BKS overlay_framework: debug:1 3 - Save the sample file from this bug into your kohaclone (or copy and paste into a file your koha test site can reach) 4 - On the command line: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 5 - In another terminal: cat bug_33418.test | nc -v localhost 8888 6 - It should report success and a biblionumber 7 - In Koha: Cataloguing->Manage staged MARC record 8 - View the most recent batch with file name (webservice) 9 - Confirm it was imported, no match 10 - Click 'View' under the 'Record' column 11 - Confirm record loads correctly and 'MARC framework' detail is 'Books, Booklets, Workbooks' 12 - On the terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 13 - It should succed 14 - View the new batch, confirm the record matched this time 15 - View the record details, confirm framework is now 'default' 16 - On the first terminal hit Ctrl+C to stop the daemon 16 - Edit connexion.cnf and change: import_mode:stage framework:ACQ overlay_framework:IR 17 - Restart daemon: perl misc/bin/connexion_import_daemon.pl -c /etc/koha/sites/kohadev/connexion.cnf 18 - Delete the record created above 19 - On the second terminal repeat the command: cat bug_33418.test | nc -v localhost 8888 20 - Confirm the batch is created, but not imported 21 - In terminal: perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 22 - Confirm batch imported, and record in ACQ framework 23 - In terminal: cat bug_33418.test | nc -v localhost 8888 perl misc/cronjobs/import_webservice_batch.pl --framework=ACQ --overlay_framework=BKS 24 - Confirm batch added, record matched, record imported, and record now in Books framework 25 - Stop the deamon, edit connexion.cnf: import_mode:direct 26 - Start the daemon, and on other terminal repeat: cat bug_33418.test | nc -v localhost 8888 27 - Confirm record in Binders framework 28 - Set record framework to Books 29 - Stop daemon, edit cnf and remove 'overlay_framework' setting 30 - Start daemon and cat the file again 31 - Confirm the record remains in Books framework Signed-off-by: Brendan Lawlor <blawlor@clamsnet.org> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Essentially only adding the overlay framework parameter. Not tested but patch looks good to me. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |release-notes-needed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to main Version(s)| |24.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 --- Comment #9 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Pushed for 24.05! Well done everyone, thank you! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Michelle Spinney <mspinney@clamsnet.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mspinney@clamsnet.org -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m Status|Pushed to main |Needs documenting --- Comment #10 from Fridolin Somers <fridolin.somers@biblibre.com> --- Not backported to 23.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=37543 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33418 Lisette Scheer <lisette@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|release-notes-needed | -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org