[Bug 26024] New: Purge undone of zebraqueue in cleanup_database.pl
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 Bug ID: 26024 Summary: Purge undone of zebraqueue in cleanup_database.pl Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Searching - Zebra Assignee: koha-bugs@lists.koha-community.org Reporter: fridolin.somers@biblibre.com Purge script misc/cronjobs/cleanup_database.pl with --zebraqueue DAYS will delete entries from zebraqueue table with done=1 We now start to use Elasticsearch only so entries of zebraqueue with done=0 stay in database. I propose we purge all entries in cleanup_database.pl. Even with zebra, if indexing does not occur after several days (30 by default), there is a problem and one need a full rebuild. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED Assignee|koha-bugs@lists.koha-commun |fridolin.somers@biblibre.co |ity.org |m -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Trivial patch Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 --- Comment #1 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Created attachment 107083 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=107083&action=edit Bug 26024: Purge undone of zebraqueue in cleanup_database.pl Purge script misc/cronjobs/cleanup_database.pl with --zebraqueue DAYS will delete entries from zebraqueue table with done=1 We now start to use Elasticsearch only so entries of zebraqueue with done=0 stay in database. I propose we purge all entries in cleanup_database.pl. Even with zebra, if indexing does not occur after several days (30 by default), there is a problem and one need a full rebuild. Test plan: 1) Use an database with entries in zebraqueue older than 7 days 2) Stop zebra indexing 3) Count entries older than 7 days : mysql > select count(*),done from zebraqueue where time < date_sub(curdate(), INTERVAL 7 DAY) group by done; 4) Simulate done=0 : mysql > update zebraqueue set done=0; 5) Run misc/cronjobs/cleanup_database.pl --zebraqueue 7 -v 6) Re run 3) you have no results -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |In Discussion --- Comment #2 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I am sorry, Frido, but I don't feel this is the right solution. 1) If there is a problem with indexing, I want to know about it, this will hide the issue by deleting the evidence. 2) You can do a partial rebuilt now but simply having the queue pick up again where the indexer stopped - so there is sense in keeping the old entries. Depending on how big and busy your library is you might well want to avoid a full reindex. 3) Wouldn't the solution be that if your library uses Elasticsearch only, we should not write to the zebraqueue at all? As of now, your patch affects all libraries and doesn't allow for any choice. For example, we delete done entries every night, but I'd want the undone to be a separate option at least. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 --- Comment #3 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- (In reply to Katrin Fischer from comment #2)
3) Wouldn't the solution be that if your library uses Elasticsearch only, we should not write to the zebraqueue at all? Of course its even better.
If one must switch back to Zebra just rebuild full. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=25710 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=21820 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 --- Comment #4 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Ok thats Bug 21820 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #107083|0 |1 is obsolete| | --- Comment #5 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- Comment on attachment 107083 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=107083 Bug 26024: Purge undone of zebraqueue in cleanup_database.pl
From fd02420ab8d40053906af75634104b5bb45f2dfa Mon Sep 17 00:00:00 2001 From: Fridolin Somers <fridolin.somers@biblibre.com> Date: Mon, 20 Jul 2020 11:48:24 +0200 Subject: [PATCH] Bug 26024: Purge undone of zebraqueue in cleanup_database.pl
Purge script misc/cronjobs/cleanup_database.pl with --zebraqueue DAYS will delete entries from zebraqueue table with done=1
We now start to use Elasticsearch only so entries of zebraqueue with done=0 stay in database.
I propose we purge all entries in cleanup_database.pl. Even with zebra, if indexing does not occur after several days (30 by default), there is a problem and one need a full rebuild.
Test plan: 1) Use an database with entries in zebraqueue older than 7 days 2) Stop zebra indexing 3) Count entries older than 7 days : mysql > select count(*),done from zebraqueue where time < date_sub(curdate(), INTERVAL 7 DAY) group by done; 4) Simulate done=0 : mysql > update zebraqueue set done=0; 5) Run misc/cronjobs/cleanup_database.pl --zebraqueue 7 -v 6) Re run 3) you have no results --- misc/cronjobs/cleanup_database.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/misc/cronjobs/cleanup_database.pl b/misc/cronjobs/cleanup_database.pl index a1a9cae442..8a6224ae55 100755 --- a/misc/cronjobs/cleanup_database.pl +++ b/misc/cronjobs/cleanup_database.pl @@ -236,7 +236,7 @@ if ($zebraqueue_days) { q{ SELECT id,biblio_auth_number,server,time FROM zebraqueue - WHERE done=1 AND time < date_sub(curdate(), INTERVAL ? DAY) + WHERE time < date_sub(curdate(), INTERVAL ? DAY) } ); $sth->execute($zebraqueue_days) or die $dbh->errstr; -- 2.27.0
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 Peter Vashchuk <stalkernoid@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 --- Comment #6 from Peter Vashchuk <stalkernoid@gmail.com> --- Created attachment 172779 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=172779&action=edit Bug 26024: Purge option for undone zebraq entries in cleanup_database.pl Added --zebraqueue-undone DAYS option to cleanup_database.pl to purge uncompleted (done=0) zebraqueue entries older than DAYS days (90 days default). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #7 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #2)
3) Wouldn't the solution be that if your library uses Elasticsearch only, we should not write to the zebraqueue at all? As of now, your patch affects all libraries and doesn't allow for any choice.
Yeah, that's probably the way. I plan to do that at some point... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com --- Comment #8 from David Nind <david@davidnind.com> --- Not sure from the comments if this is ready for sign off, or if more work is required. The current patch still applies. Should the status be "In discussion" or "Failed QA"? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |In Discussion --- Comment #9 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I think this might be "In Discussion" even tho I don't like the status. I feel like it fixes the "wrong" thing. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26024 --- Comment #10 from David Nind <david@davidnind.com> --- (In reply to Katrin Fischer from comment #9)
I think this might be "In Discussion" even tho I don't like the status. I feel like it fixes the "wrong" thing.
Thanks Katrin! -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org