[Bug 33972] New: Possible deadlock in C4::ImportBatch::SetimportBatchStatus from BatchCommitRecords
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Bug ID: 33972 Summary: Possible deadlock in C4::ImportBatch::SetimportBatchStatus from BatchCommitRecords Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: blocker Priority: P5 - low Component: Tools Assignee: koha-bugs@lists.koha-community.org Reporter: nick@bywatersolutions.com QA Contact: testopia@bugs.koha-community.org In BatchCommitRecords we currently Start a transaction ... SetImportBatchStatus($batch_id, 'importing'); ... SetImportBatchStatus($batch_id, 'finished'); ... Finish the transaction In certain settings the second call fails as it is waiting for the first to commit it seems: [Fri Jun 9 09:06:29 2023] background_jobs_worker.pl: C4::ImportBatch::SetImportBatchStatus(): DBI Exception: DBD::mysql::db do failed: Lock wait timeout exceeded; try restarting transaction at /usr/share/koha/lib/C4/ImportBatch.pm line 717 Setting the status twice inside the transaction doesn't make sense - the DB will never show the 'importing' status - as it will either be rolled back when the transaction fails, or set to 'finished' when it completes -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=32990 CC| |katrin.fischer@bsz-bw.de, | |m.de.rooy@rijksmuseum.nl, | |martin.renvoize@ptfs-europe | |.com -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com --- Comment #1 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Is it a recent regression? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #2 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 152249 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152249&action=edit Bug 33972: Set importing status to batch outside of transaction This patch moves the setting of the 'importing' status for the batch outside fo the transaction. This prevents a scenario seen on some setups where a deadlock is created: background_jobs_worker.pl: C4::ImportBatch::SetImportBatchStatus(): DBI Exception: DBD::mysql::db do failed: Lock wait timeout exceeded; try restarting transaction at /usr/share/koha/lib/C4/ImportBatch.pm line 717 To test: 1 - Apply patch 2 - Stage and import some files 3 - Confirm the imports successfully complete -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |27421 --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Jonathan Druart from comment #1)
Is it a recent regression?
Yes, only seen in 22.11, suspect it is from the move of the transaction on 27421 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27421 [Bug 27421] Porting tools/stage-marc-import.pl to BackgroundJob -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=33987 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #4 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Nick Clemens from comment #0)
In BatchCommitRecords we currently
Start a transaction ... SetImportBatchStatus($batch_id, 'importing'); ... SetImportBatchStatus($batch_id, 'finished'); ... Finish the transaction
In certain settings the second call fails as it is waiting for the first to commit it seems: [Fri Jun 9 09:06:29 2023] background_jobs_worker.pl: C4::ImportBatch::SetImportBatchStatus(): DBI Exception: DBD::mysql::db do failed: Lock wait timeout exceeded; try restarting transaction at /usr/share/koha/lib/C4/ImportBatch.pm line 717
Setting the status twice inside the transaction doesn't make sense - the DB will never show the 'importing' status - as it will either be rolled back when the transaction fails, or set to 'finished' when it completes
Normally, you should be able to update the same record within the same transaction. And only another transaction trying to get/update this same record in import_batches should be locked. So, are you sure that there was no second process involved? The other thing, though, is that SetImportBatchStatus asks for another db handle via Context/Koha::Database. Since you have a connection, you would not expect a problem. But is it possible that somehting there somehow goes wrong? Would it be an idea to move the second SetImportBatchStatus call at the end also after the transaction? Hard to verify if we dont know the exact cause and cannot reproduce.. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #5 from Nick Clemens <nick@bywatersolutions.com> --- Also see 33987 - similar deadlock when accessing the same import record row multiple times I can say, I can't reproduce in KTD, but I can replicate every time on our production servers. In terms of setting the status, doing so twice in a transaction doesn't make sense. It will revert to the original if it fails, or use the final if it succeeds. Moving the first one made sense because then the status will be left at 'importing' - the job should be 'failed' so we know there is a problem? Here and on 33987 I think the code makes sense on its own, even if the problem doesn't always occur -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #6 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Nick Clemens from comment #5)
Also see 33987 - similar deadlock when accessing the same import record row multiple times
Will have a look too
I can say, I can't reproduce in KTD, but I can replicate every time on our production servers.
What are the specs? Koha version, SQL version ? Do you also confirm herewith that you can no longer reproduce with your fix applied? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #7 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Are you using --max-processes in the worker? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #8 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Jonathan Druart from comment #7)
Are you using --max-processes in the worker?
If another process is involved, we should look for a statement that locks the whole table. Since normally these processes only work on a single batch record and the associated import records ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #152249|0 |1 is obsolete| | --- Comment #9 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 152400 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152400&action=edit Bug 33972: Set importing status to batch outside of transaction This patch moves the setting of the 'importing' status for the batch outside fo the transaction. This prevents a scenario seen on some setups where a deadlock is created: background_jobs_worker.pl: C4::ImportBatch::SetImportBatchStatus(): DBI Exception: DBD::mysql::db do failed: Lock wait timeout exceeded; try restarting transaction at /usr/share/koha/lib/C4/ImportBatch.pm line 717 To test: 1 - Apply patch 2 - Stage and import some files 3 - Confirm the imports successfully complete Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #10 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 152401 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152401&action=edit Bug 33972: (follow-up) Remove status change to importing It actually is not used, checked, etc. It makes more sense to me to remove it than to put it before the transaction. If the transaction succeeds, we end up with imported. It it fails, we will still be on staged (which is better than importing). The job will be marked as failed in that case too. Note: I am not convinced that this is the definitive solution. This feels like a workaround. We might still need to dig further. Are we simply doing too much in this transaction for large staged imports? Or shouldnt we call Context->dbh constantly in several subroutines? Is this just a Mariadb bug? No answers yet. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #11 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Katrin and I tested a few larger imports and could not reproduce the lock with and without the patches. A file with 700 records went in with/without patches. Another file had data issues, too bad. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #12 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- I also tested a script updating a record in import_batches constantly, getting new db handles, few updates, fetches and that 100.000 times. Unfortuantely no deadlocks :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #13 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- In conclusion, please add more details about Koha/SQL version. And please test again. Can you still reproduce with these patches ? Note that these patches are fine with me. But not the solution yet? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #14 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Marcel de Rooy from comment #13)
In conclusion, please add more details about Koha/SQL version. And please test again. Can you still reproduce with these patches ? Note that these patches are fine with me. But not the solution yet?
Server version: 10.6.12-MariaDB-log MariaDB Server Koha version 22.11 - with some ByWater patches: https://github.com/bywatersolutions/bywater-koha/commits/bywater-v22.11.06-0... Fixed in branch with this patch: https://github.com/bywatersolutions/bywater-koha/commits/bywater-v22.11.06-0... (In reply to Marcel de Rooy from comment #6)
Do you also confirm herewith that you can no longer reproduce with your fix applied?
I do, after this we do not see this deadlock occur The deadlock happens consistently, with imports of any size, we are asking our DBA to take a deeper look -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #15 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Could it be 22.11.x specific? I have tested on master today and while I bumped into 34029, things went mostly ok. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #152400|0 |1 is obsolete| | --- Comment #16 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 152409 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152409&action=edit Bug 33972: Set importing status to batch outside of transaction This patch moves the setting of the 'importing' status for the batch outside fo the transaction. This prevents a scenario seen on some setups where a deadlock is created: background_jobs_worker.pl: C4::ImportBatch::SetImportBatchStatus(): DBI Exception: DBD::mysql::db do failed: Lock wait timeout exceeded; try restarting transaction at /usr/share/koha/lib/C4/ImportBatch.pm line 717 To test: 1 - Apply patch 2 - Stage and import some files 3 - Confirm the imports successfully complete Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #152401|0 |1 is obsolete| | --- Comment #17 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 152410 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152410&action=edit Bug 33972: (follow-up) Remove status change to importing It actually is not used, checked, etc. It makes more sense to me to remove it than to put it before the transaction. If the transaction succeeds, we end up with imported. It it fails, we will still be on staged (which is better than importing). The job will be marked as failed in that case too. Note: I am not convinced that this is the definitive solution. This feels like a workaround. We might still need to dig further. Are we simply doing too much in this transaction for large staged imports? Or shouldnt we call Context->dbh constantly in several subroutines? Is this just a Mariadb bug? No answers yet. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #18 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Nick Clemens from comment #14)
Server version: 10.6.12-MariaDB-log MariaDB Server
I tested with 10.5.17. Katrin also had a 10.5.X
Do you also confirm herewith that you can no longer reproduce with your fix applied? I do, after this we do not see this deadlock occur
Great, but still weird.
The deadlock happens consistently, with imports of any size, we are asking our DBA to take a deeper look
That sounds great too. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #19 from David Cook <dcook@prosentient.com.au> --- Side note: We assume that "Koha::Database->schema->storage->dbh" always returns the same database handle, but that's not necessarily true. According to the documentation and source code at https://metacpan.org/pod/DBIx::Class::Storage::DBI#dbh it seems that it tries to fetch a cached database handle but if for whatever reason that cached database handle doesn't seem connected, it'll create a new handle and connect. Probably not related but just noticed that while looking at something else... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #20 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to David Cook from comment #19)
Side note: We assume that "Koha::Database->schema->storage->dbh" always returns the same database handle, but that's not necessarily true.
According to the documentation and source code at https://metacpan.org/pod/DBIx::Class::Storage::DBI#dbh it seems that it tries to fetch a cached database handle but if for whatever reason that cached database handle doesn't seem connected, it'll create a new handle and connect.
Probably not related but just noticed that while looking at something else...
wait_timeout defaults to 28800 (seconds) == 8 hours. The worker is a continuous process. It does db reads itself for the job details. At some point probably a _resultset call in Koha objects or so triggers a reconnect and we get a new handle. But this problem seems to be reproduced every time. The reconnect should logically not occur when we already read job details and continue in the child? So there must be something else going on too.. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #21 from David Cook <dcook@prosentient.com.au> --- (In reply to Marcel de Rooy from comment #20)
wait_timeout defaults to 28800 (seconds) == 8 hours. The worker is a continuous process. It does db reads itself for the job details. At some point probably a _resultset call in Koha objects or so triggers a reconnect and we get a new handle. But this problem seems to be reproduced every time. The reconnect should logically not occur when we already read job details and continue in the child? So there must be something else going on too..
I'm thinking more along the lines of that dbh's ping not coming back as expected and it spawning a new connection, but it's too tough to know arm's length. Just trying to think of any angle that could be used to troubleshoot it. Without being able to reproduce it outside of Bywater prod, it's a tough one :/. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #22 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Nick, what did your DBA find ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #23 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Marcel de Rooy from comment #22)
Nick, what did your DBA find ?
We are still investigating - we have now seen it both in our mariadb cluster, and on a local mariadb on a self hosted server In both cases the problem is now occurring in _batchCommitItems -> AddItemFromMarc -> Koha::Item->store -> Koha::Object->store The INSERT call on items is hanging and then failing: DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Lock wait timeout exceeded; try restarting transaction at /usr/share/koha/lib/Koha/Object.pm line 170 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #24 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- It looks like I forgot to add my SO lines here earlier (see comment#16) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #152409|0 |1 is obsolete| | --- Comment #25 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 152654 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152654&action=edit Bug 33972: Set importing status to batch outside of transaction This patch moves the setting of the 'importing' status for the batch outside fo the transaction. This prevents a scenario seen on some setups where a deadlock is created: background_jobs_worker.pl: C4::ImportBatch::SetImportBatchStatus(): DBI Exception: DBD::mysql::db do failed: Lock wait timeout exceeded; try restarting transaction at /usr/share/koha/lib/C4/ImportBatch.pm line 717 To test: 1 - Apply patch 2 - Stage and import some files 3 - Confirm the imports successfully complete Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #152410|0 |1 is obsolete| | --- Comment #26 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 152655 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152655&action=edit Bug 33972: (follow-up) Remove status change to importing It actually is not used, checked, etc. It makes more sense to me to remove it than to put it before the transaction. If the transaction succeeds, we end up with imported. It it fails, we will still be on staged (which is better than importing). The job will be marked as failed in that case too. Note: I am not convinced that this is the definitive solution. This feels like a workaround. We might still need to dig further. Are we simply doing too much in this transaction for large staged imports? Or shouldnt we call Context->dbh constantly in several subroutines? Is this just a Mariadb bug? No answers yet. Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #27 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Nick Clemens from comment #23)
(In reply to Marcel de Rooy from comment #22)
Nick, what did your DBA find ?
We are still investigating - we have now seen it both in our mariadb cluster, and on a local mariadb on a self hosted server
In both cases the problem is now occurring in _batchCommitItems -> AddItemFromMarc -> Koha::Item->store -> Koha::Object->store
The INSERT call on items is hanging and then failing: DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Lock wait timeout exceeded; try restarting transaction at /usr/share/koha/lib/Koha/Object.pm line 170
Thats interesting. Will try to reproduce this first with two processes inserting items. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #28 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- I came across this remark: https://stackoverflow.com/questions/64789956/mariadb-innodb-deadlock-while-d... An InnoDB table with both a PRIMARY KEY and a secondary UNIQUE KEY has a large chance of causing a deadlock. Example of such a case is described clearly in this bug report: https://bugs.mysql.com/bug.php?id=86812 Note that items has PK on itemnumber and UNIQUE key on barcode. When you temporarily remove the unique constraint, can you still reproduce? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #29 from Nick Clemens <nick@bywatersolutions.com> --- We found the issue, it is related to plugins that create their own objects (Curbside is known offender) The plugins use a trick: # register the additional schema classes Koha::Schema->register_class(CurbsidePickup => 'Koha::Schema::Result::CurbsidePickup'); Koha::Schema->register_class(CurbsidePickupPolicy => 'Koha::Schema::Result::CurbsidePickupPolicy'); Koha::Schema->register_class(CurbsidePickupIssue => 'Koha::Schema::Result::CurbsidePickupIssue'); # ... and force a refresh of the database handle so that it includes # the new classes Koha::Database->schema({ new => 1 }); } That forces a new DB connection - which means the transaction doesn't commit (the savepoint is released, but it is on a different connection, so moot) The call: _after_biblio_action_hooks({ action => 'create', biblio_id => $biblionumber }); calls: get_enabled_plugins which calls: my $plugin = eval { $plugin_class->new() }; which triggers the new schema/connection when get_enabled_plugins is cached we call $plugin->can($method) which also instantiates the plugin/creates new connection I think the patches on this report still make sense, they remove an unecessary status change. We are fixing the plugin, but we probably should update: https://koha-community.gitlab.io/KohaAdvent/2020-12-07-dbic/ and I think get_enabled_plugins should be using the plugin_methods table to avoid loading plugins that don't have the method we need? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=34121 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #30 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Nick Clemens from comment #29)
We found the issue, it is related to plugins that create their own objects (Curbside is known offender)
Great, you catched it. It is always something else ;) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|blocker |minor -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #31 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Nick Clemens from comment #29)
# register the additional schema classes Koha::Schema->register_class(CurbsidePickup => 'Koha::Schema::Result::CurbsidePickup');
[...]
Koha::Database->schema({ new => 1 }); }
If you use these plugins/tables, why not add these classes to the standard schema?
I think the patches on this report still make sense, they remove an unecessary status change.
We dont really need them anymore. No strong opinion on that.
and I think get_enabled_plugins should be using the plugin_methods table to avoid loading plugins that don't have the method we need?
Or just use normal connection or make a different connection later on ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #32 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Marcel de Rooy from comment #31)
(In reply to Nick Clemens from comment #29)
I think the patches on this report still make sense, they remove an unecessary status change.
We dont really need them anymore. No strong opinion on that.
This got enough attention now to push it into the RM queue. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |BLOCKED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #33 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- It makes no sense anymore with bug 33987 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|BLOCKED |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|RESOLVED |REOPENED Resolution|FIXED |--- --- Comment #34 from Nick Clemens <nick@bywatersolutions.com> --- That one deals with import_records updates - this one deals with import_batches - it removes a status change that will never be committed -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|REOPENED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #35 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- What happened with: - SetImportBatchStatus($batch_id, 'importing'); -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #36 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- This needs attention ! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #152654|0 |1 is obsolete| | Attachment #152655|0 |1 is obsolete| | --- Comment #37 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 152738 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152738&action=edit Bug 33972: Remove settings of batch status to imorting This change was done in a transaction - it would either be set as imported on success, or rolled back to staged on failure There is no need for the intermediate status which is never committed Signed-off-by: Nick Clemens <nick@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Possible deadlock in |Remove unnecessary batch |C4::ImportBatch::SetimportB |status change in |atchStatus from |C4::ImportBatch |BatchCommitRecords | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|Small patch |Trivial patch Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #38 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Created attachment 152744 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=152744&action=edit Bug 33972: Remove settings of batch status to importing This change was done in a transaction - it would either be set as imported on success, or rolled back to staged on failure There is no need for the intermediate status which is never committed Signed-off-by: Nick Clemens <nick@bywatersolutions.com> Signed-off-by: Marcel de Rooy <m.de.rooy@rijksmuseum.nl> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |m.de.rooy@rijksmuseum.nl |y.org | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #152738|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |23.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #39 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 23.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|23.11.00 |23.11.00,23.05.02 released in| | Status|Pushed to master |Pushed to stable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #40 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Thanks for all the hard work! Pushed to 23.05.x for the next release -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 Pedro Amorim <pedro.amorim@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|23.11.00,23.05.02 |23.11.00,23.05.02,22.11.08 released in| | Status|Pushed to stable |Pushed to oldstable -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33972 --- Comment #41 from Pedro Amorim <pedro.amorim@ptfs-europe.com> --- Nice work everyone! Pushed to 22.11.x for next release -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org