[Bug 41087] New: EDI Retry: We should add the option to retry EDI processing for errors
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Bug ID: 41087 Summary: EDI Retry: We should add the option to retry EDI processing for errors Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Acquisitions Assignee: koha-bugs@lists.koha-community.org Reporter: martin.renvoize@openfifth.co.uk QA Contact: testopia@bugs.koha-community.org Depends on: 39215, 39291 Blocks: 3728 Now that we're catching and reporting EDI processing errors more, we should add the option to retry and revert a message Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=3728 [Bug 3728] select mandatory fields broken https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39215 [Bug 39215] EDIFACT quote files should skip orders that cannot be processed rather than failing to complete https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39291 [Bug 39291] Edifact should record an error and move on if there's a problem with transport filename change -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|3728 |37248 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=3728 [Bug 3728] select mandatory fields broken https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37248 [Bug 37248] [Omnibus] Power to the user -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |martin.renvoize@openfifth.c |ity.org |o.uk -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=40616 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #1 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- *** Bug 40616 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #2 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Sometimes an edifact transport may throw processing errors which are now reported in the UI. It would be nice to have the option of re-running the processing steps for such message if you believe you have corrected the issues (often simple mapping issues for example). The different message types would need slightly different processes to reverse the processing flow to allow for re-processing. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |41825 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41825 [Bug 41825] Holds cannot be cancelled after a checkout on the patron checkout page -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|41825 |41852 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41825 [Bug 41825] Holds cannot be cancelled after a checkout on the patron checkout page https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41852 [Bug 41852] Modernize EDI support in Koha -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|EDI Retry: We should add |EDI Retry: Add Rollback and |the option to retry EDI |Reprocess actions for |processing for errors |EDIFACT messages Depends on| |43790, 42001 See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=41528, | |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=38195, | |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=41852, | |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=16773, | |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=39867, | |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=39904 --- Comment #3 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Scoping this bug to: a Rollback action that undoes what processing a QUOTE or INVOICE message created (refused up front, with a reason, when unsafe, e.g. closed invoice or closed basket), a new 'rolledback' message status that Import accepts (so edi_cron never silently re-imports it), and a Reprocess shortcut that does rollback + import in one transaction. Backing a message out while waiting for a corrected vendor message is simply Rollback. The staff-facing 'hide/dismiss' request for errors is tracked separately in bug 43081. Built on top of bug 43790 (Import dispatches by message type), and on 39215 and 42001 which touch the same process_quote code. Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42001 [Bug 42001] Update duplicate purchase order number detection on EDI quote import to optionally send notifications to staff and vendors https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43790 [Bug 43790] EDIFACT messages Import button always runs the invoice importer, even for quotes -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on|39215, 42001 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39215 [Bug 39215] EDIFACT quote files should skip orders that cannot be processed rather than failing to complete https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=42001 [Bug 42001] Update duplicate purchase order number detection on EDI quote import to optionally send notifications to staff and vendors -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #4 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 208008 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208008&action=edit Bug 41087: Add resolved columns to edifact_errors Rolling an EDIFACT message back keeps its processing errors so it stays visible why the message was rolled back: they are marked resolved instead of being deleted. This adds four nullable columns to edifact_errors: - resolved_on: when the error was resolved, NULL while it is open - resolved_by_id: the staff member who resolved it (FK to borrowers) - resolution: how it was resolved (rolledback, or dismissed later) - resolution_note: the optional reason given Test plan: 1. Apply the patch and run updatedatabase.pl 2. Check the new columns and the emfk_resolved_by foreign key exist: SHOW CREATE TABLE edifact_errors; 3. Run updatedatabase.pl again and confirm it does not fail Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #5 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 208009 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208009&action=edit Bug 41087: Automated Schema Update Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #6 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 208010 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208010&action=edit Bug 41087: Add resolve and filter_by_unresolved to EDIFACT file errors Koha::Edifact::File::Error gains resolve() and is_resolved(), and the API maps resolved_by_id to resolved_by. Koha::Edifact::File::Errors gains filter_by_unresolved(). Test plan: 1. Apply the patch 2. prove t/db_dependent/Koha/Edifact/File/Errors.t Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #7 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 208011 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208011&action=edit Bug 41087: Allow Import for rolled back EDIFACT messages The Import button and the cud-import operation only accepted messages in status 'new'. Add Koha::Edifact::File->can_import, true for inbound QUOTE, INVOICE and ORDRSP messages that are 'new' or 'rolledback', and use it in both places. 'rolledback' is the status the following patches give a rolled back message so that it can be imported again. The cleanup_database.pl help now says that 'rolledback' messages are purged by age like other non-new messages. Test plan: 1. Apply the patch 2. prove t/db_dependent/Koha/Edifact/File.t 3. Insert a 'rolledback' INVOICE message in edifact_messages and open Acquisitions > EDIFACT messages: the Import button is shown Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #8 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 208012 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208012&action=edit Bug 41087: Add Koha::Edifact::File->rollback core Add Koha::Edifact::File->rollback_check and ->rollback. rollback_check is a read-only pre-flight returning whether the message can be rolled back, the reasons if not, and a summary of the impact. Only QUOTE and INVOICE messages in status 'received' or 'error' can be rolled back. rollback re-runs the check inside a transaction with the message row locked, so two staff members cannot undo the same message twice. It then marks the message's open errors resolved (they are kept, not deleted), sets the status to 'rolledback' and writes EDI_ROLLBACK to the acquisitions log. If the check refuses it throws Koha::Exceptions::EDI::RollbackNotPossible with the reasons and changes nothing. Undoing what an invoice or a quote created is added by the next two patches. Test plan: 1. Apply the patch 2. prove t/db_dependent/Koha/Edifact/File/Rollback.t Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #9 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 208013 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208013&action=edit Bug 41087: Roll back EDIFACT invoice messages Rolling back an INVOICE message un-receives the orders on the invoices created from it (part receipts are merged back into their parent order by CancelReceipt) and deletes those invoices. The result of CancelReceipt is verified because it only warns on failure. The rollback is refused, with a reason and a link, when: - an invoice is closed - a part receipt's parent order has since been received in full - an invoice has adjustments that no plugin has claimed - the EDI account uses a plugin that processes invoices itself Test plan: 1. Apply the patch 2. prove t/db_dependent/Koha/Edifact/File/Rollback.t Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #10 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 208014 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208014&action=edit Bug 41087: Roll back EDIFACT quote messages Rolling back a QUOTE message removes the baskets created from it, including those of split multi-message quotes, with their orders, items and the biblios the quote created. Biblios that other items, orders or subscriptions still use are kept. Baskets closed by auto_orders can be rolled back while their ORDERS message is still Pending; that message is deleted in the same transaction so edi_cron.pl cannot send it. The parent message is detached from its basket first because the basketno foreign key cascades. The rollback is refused when a basket is in a basket group, is closed without a pending EDI order, has an order that was received or changed, or when its ORDERS message has already been sent. Test plan: 1. Apply the patch 2. prove t/db_dependent/Koha/Edifact/File/Rollback.t Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #11 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 208015 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208015&action=edit Bug 41087: Add before_edifact_action plugin hook for rollbacks Customer plugins can add data to an EDI invoice (for example invoice adjustments) or record it as exported. Add a before_edifact_action plugin hook so they can take part in a rollback. The hook is called with action 'rollback_check' during the read-only pre-flight: a plugin returns blockers, which refuse the rollback, and owned_adjustments, the invoice adjustments it will remove. Core never deletes an adjustment it does not own. It is then called with action 'rollback' inside the rollback transaction so the plugin can delete what it claimed. Koha::Plugins->call swallows plugin exceptions, so core fails closed: a plugin that does not answer the check refuses the rollback, and after the rollback call no adjustment may remain on the invoice or the rollback is aborted. after_edifact_action (Bug 43804) is fired with action 'rollback' once a message has been rolled back. Test plan: 1. Apply the patch 2. prove t/db_dependent/Koha/Plugins/Edifact_rollback_hooks.t Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #12 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 208016 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208016&action=edit Bug 41087: Add reprocess for EDIFACT messages Add Koha::Edifact::File->reprocess, which rolls a message back and processes it again in one transaction. If processing fails the rollback is reverted too and the message is left exactly as it was. Plugins are told about the rollback (after_edifact_action) before the new run is processed. Test plan: 1. Apply the patch 2. prove t/db_dependent/Koha/Edifact/File/Rollback.t (it includes a run against the real INVOICE.CEI test file: process, roll back, import again and reprocess) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 --- Comment #13 from Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> --- Created attachment 208017 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=208017&action=edit Bug 41087: Add Roll back and Reprocess to the EDIFACT messages page Add Roll back and Reprocess buttons to Acquisitions > EDIFACT messages for received and error QUOTE and INVOICE messages. The buttons open a confirmation modal that first asks the server for the pre-flight result (a read-only rollback-check JSON operation), so the staff member sees why a message cannot be rolled back, or what will be undone, before confirming. An optional reason is kept with the resolved errors. Rolling back and reprocessing are POST operations (cud-rollback, cud-reprocess) protected by the CSRF token. Test plan: 1. Apply the patch and run restart_all 2. Process an EDIFACT INVOICE that produces errors (for example t/edi_testfiles/INVOICE.CEI) and open Acquisitions > EDIFACT messages 3. Click 'Roll back' on it: the modal lists what will be undone. Confirm with a reason 4. The message is now 'rolledback' and has an Import button 5. Process another invoice, close its Koha invoice and try to roll it back: the reason is shown and Confirm stays disabled 6. npx cypress run --spec t/cypress/integration/Acquisitions/EdifactRollback_spec.ts Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |43804 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43804 [Bug 43804] Add after_edifact_action plugin hook for processed EDIFACT messages -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |43081 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=43081 [Bug 43081] Hide option/button for edifact error messages in Acquisitions -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41087 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks|41852 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41852 [Bug 41852] Modernize EDI support in Koha -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org