Should we be cleaning zebraqueue and action_logs tables periodically?
Please forgive me for pulling this bit from the patches list. I think the relevant discussion should be on this list. On Sat, 2009-04-25 at 13:29 -0400, Ryan Higgins wrote:
If you've never truncated your zebraqueue table, you might want to update your index and truncate it before applying this patch and running updatedatabase.
Truncating the zebraqueue table, as noted by Ryan with this patch, is probably something Koha itself should do periodically. Last time I looked at the cron example I didn't see anything to this end. The data in the table is quite small, but there are still a lot of rows there. This could also be applied to the action_logs table, which holds more data, and has more rows in my installation. Problem is troubleshooting or data retention requirements. I suppose there should be some config option to set a threshold on how old of data should be kept here. Or maybe a system preference could do that job. I for one would like to see Koha clearing these table itself, so there is one less thing for me to take care of. This is especially important to me as I didn't realize before today that these tables would be as large as they are ( as far as number of rows is concerned ). I would say it's safe to assume other system admins also aren't thinking about the size of these tables. Comments please. -- Michael Hafen Systems Analyst and Programmer Washington County School District Utah, USA for Koha checkout http://development.washk12.org/gitweb/ or git://development.washk12.org/koha
Hi, On Mon, Apr 27, 2009 at 2:54 PM, Michael Hafen <mdhafen@tech.washk12.org> wrote:
Truncating the zebraqueue table, as noted by Ryan with this patch, is probably something Koha itself should do periodically. Last time I looked at the cron example I didn't see anything to this end. The data in the table is quite small, but there are still a lot of rows there.
The data in zebraqueue has very little use once a record has been successfully reindexed, so periodically purging completed entries is a good idea.
This could also be applied to the action_logs table, which holds more data, and has more rows in my installation.
Problem is troubleshooting or data retention requirements. I suppose there should be some config option to set a threshold on how old of data should be kept here. Or maybe a system preference could do that job.
There's is more need to retain data from action_logs, as for some types of transactions it is the only available source of audit and statistical information. IMO, a cronjob that cleans old action_log entries needs the following: * fine-grained configuration of which types of entries to purge and what the age threshold for each type should be * the ability to dump purged entries to a CSV or XML file. Regards, Galen -- Galen Charlton VP, Research & Development, LibLime galen.charlton@liblime.com p: 1-888-564-2457 x709 skype: gmcharlt
Yeah, this is an overall maintenance problem. Take a look at sessions, statistics, accountlines and probably others too, like import_*. A general purpose fix here would be good, but the implementation probably should involve crontab. -- Joe Atzberger LibLime - Open Source Library Solutions On Mon, Apr 27, 2009 at 2:54 PM, Michael Hafen <mdhafen@tech.washk12.org>wrote:
Please forgive me for pulling this bit from the patches list. I think the relevant discussion should be on this list.
On Sat, 2009-04-25 at 13:29 -0400, Ryan Higgins wrote:
If you've never truncated your zebraqueue table, you might want to update your index and truncate it before applying this patch and running updatedatabase.
Truncating the zebraqueue table, as noted by Ryan with this patch, is probably something Koha itself should do periodically. Last time I looked at the cron example I didn't see anything to this end. The data in the table is quite small, but there are still a lot of rows there.
This could also be applied to the action_logs table, which holds more data, and has more rows in my installation.
Problem is troubleshooting or data retention requirements. I suppose there should be some config option to set a threshold on how old of data should be kept here. Or maybe a system preference could do that job.
I for one would like to see Koha clearing these table itself, so there is one less thing for me to take care of. This is especially important to me as I didn't realize before today that these tables would be as large as they are ( as far as number of rows is concerned ). I would say it's safe to assume other system admins also aren't thinking about the size of these tables.
Comments please. -- Michael Hafen
participants (3)
-
Galen Charlton -
Joe Atzberger -
Michael Hafen