Koha Command and Control
Hi all, For a while, I've been thinking that it might be good to have an open source "Koha Command and Control" for managing Koha instances. While managing 1 Koha is not too much work, it's considerably more difficult to manage 10, 100, or 1000 Koha instances, especially across different servers, geographic locations, timezones, etc. I am sure that most vendors have mechanisms in place for checking the system preference values for all their instances (I have scripts for this sort of thing), what about tracking Zebra queries that contain syntax errors (bug 27139)? I know that we have HEA for sharing a lot of data, but there is some data (like search queries) that I don't think most libraries would want to share with the community. But Koha administrators, especially vendors with a large number of Koha instances, want to know when problems arise. I suppose that a person could just implement log file scanning for Zebra, but I was thinking about something more structured, which could be analyzed to provide data quality control. My thought is that we'd configure Koha to push events to the "Koha Command and Control" (which would be configured by the Koha system administrator - not librarian level). If Zebra logs a 'ZOOM error 10014 "CCL parsing error"', we probably want to tell the Koha user that we found no search results (nice user experience), but we want to flag with a Koha administrator that there is a problem. If Koha administrators (e.g. Koha vendors) start receiving a lot of these errors via the Koha CnC (Command and Control), they're better placed to raise Bugzilla reports/write patches. Anyway, it's just an idea. Probably too ambitious. But I think that it would be wise for us to start thinking more about the *data* generated by Koha and how we can use that data to improve Koha. I don't think we need machine learning algorithms to do the data analysis (yet), but I think having automated data collection, organisation, and reporting would be useful. I have to run but I'll keep thinking about it. I know I have more grand ideas than I have time, but I could put some thought into what a system might look like, and maybe throw something together locally first before suggesting anyone else get on board. David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595
This popped into my email ... Above my pay grade, but maybe of relevance? https://newrelic.com/ All best, Cab Vinton, Director Plaistow Public Library On Tue, Dec 15, 2020 at 6:04 PM <dcook@prosentient.com.au> wrote:
Hi all,
For a while, I’ve been thinking that it might be good to have an open source “Koha Command and Control” for managing Koha instances.
While managing 1 Koha is not too much work, it’s considerably more difficult to manage 10, 100, or 1000 Koha instances, especially across different servers, geographic locations, timezones, etc.
I am sure that most vendors have mechanisms in place for checking the system preference values for all their instances (I have scripts for this sort of thing), what about tracking Zebra queries that contain syntax errors (bug 27139)?
I know that we have HEA for sharing a lot of data, but there is some data (like search queries) that I don’t think most libraries would want to share with the community. But Koha administrators, especially vendors with a large number of Koha instances, want to know when problems arise.
I suppose that a person could just implement log file scanning for Zebra, but I was thinking about something more structured, which could be analyzed to provide data quality control. My thought is that we’d configure Koha to push events to the “Koha Command and Control” (which would be configured by the Koha system administrator – not librarian level). If Zebra logs a ‘ZOOM error 10014 "CCL parsing error"’, we probably want to tell the Koha user that we found no search results (nice user experience), but we want to flag with a Koha administrator that there is a problem. If Koha administrators (e.g. Koha vendors) start receiving a lot of these errors via the Koha CnC (Command and Control), they’re better placed to raise Bugzilla reports/write patches.
Anyway, it’s just an idea. Probably too ambitious. But I think that it would be wise for us to start thinking more about the *data* generated by Koha and how we can use that data to improve Koha. I don’t think we need machine learning algorithms to do the data analysis (yet), but I think having automated data collection, organisation, and reporting would be useful.
I have to run but I’ll keep thinking about it. I know I have more grand ideas than I have time, but I could put some thought into what a system might look like, and maybe throw something together locally first before suggesting anyone else get on board…
David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Hi there, for this kind of purpose I would recommend using Ansible scripts to ship commands to several Koha's at once. Using Ansible only needs an ssh connexion to each of your Koha servers, no specific software or agent to install on the targets, then commands are simple well-known bash. It is also possible to group targets (manually) based on OS, web-server software or other tweaks to deploy specific sets of commands to each of your groups, this is very handy. I personally don't use it because our environment is very heterogeneous, making grouping difficult and prone to error (and then we would have a lot of groups of a single instance...) Best, Arthur Suzuki BibLibre Koha Support On 16/12/2020 16:01, Cab Vinton wrote:
This popped into my email ... Above my pay grade, but maybe of relevance?
All best,
Cab Vinton, Director Plaistow Public Library
On Tue, Dec 15, 2020 at 6:04 PM <dcook@prosentient.com.au> wrote:
Hi all,
For a while, I’ve been thinking that it might be good to have an open source “Koha Command and Control” for managing Koha instances.
While managing 1 Koha is not too much work, it’s considerably more difficult to manage 10, 100, or 1000 Koha instances, especially across different servers, geographic locations, timezones, etc.
I am sure that most vendors have mechanisms in place for checking the system preference values for all their instances (I have scripts for this sort of thing), what about tracking Zebra queries that contain syntax errors (bug 27139)?
I know that we have HEA for sharing a lot of data, but there is some data (like search queries) that I don’t think most libraries would want to share with the community. But Koha administrators, especially vendors with a large number of Koha instances, want to know when problems arise.
I suppose that a person could just implement log file scanning for Zebra, but I was thinking about something more structured, which could be analyzed to provide data quality control. My thought is that we’d configure Koha to push events to the “Koha Command and Control” (which would be configured by the Koha system administrator – not librarian level). If Zebra logs a ‘ZOOM error 10014 "CCL parsing error"’, we probably want to tell the Koha user that we found no search results (nice user experience), but we want to flag with a Koha administrator that there is a problem. If Koha administrators (e.g. Koha vendors) start receiving a lot of these errors via the Koha CnC (Command and Control), they’re better placed to raise Bugzilla reports/write patches.
Anyway, it’s just an idea. Probably too ambitious. But I think that it would be wise for us to start thinking more about the *data* generated by Koha and how we can use that data to improve Koha. I don’t think we need machine learning algorithms to do the data analysis (yet), but I think having automated data collection, organisation, and reporting would be useful.
I have to run but I’ll keep thinking about it. I know I have more grand ideas than I have time, but I could put some thought into what a system might look like, and maybe throw something together locally first before suggesting anyone else get on board…
David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Hi Arthur, Thanks for your email. I use Ansible quite a bit already, so I see where you're coming from, but I was thinking of something more elegant. I was also thinking about something that would allow for more private data collection than Hea. Although when I put it that way... it could be interesting to update C4::UsageStats::ReportToCommunity() to actually take a list of URLs, and people could run up their own private Hea (https://gitlab.com/koha-community/hea-app). David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595 -----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Arthur Sent: Thursday, 17 December 2020 2:15 AM To: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] Koha Command and Control Hi there, for this kind of purpose I would recommend using Ansible scripts to ship commands to several Koha's at once. Using Ansible only needs an ssh connexion to each of your Koha servers, no specific software or agent to install on the targets, then commands are simple well-known bash. It is also possible to group targets (manually) based on OS, web-server software or other tweaks to deploy specific sets of commands to each of your groups, this is very handy. I personally don't use it because our environment is very heterogeneous, making grouping difficult and prone to error (and then we would have a lot of groups of a single instance...) Best, Arthur Suzuki BibLibre Koha Support On 16/12/2020 16:01, Cab Vinton wrote:
This popped into my email ... Above my pay grade, but maybe of relevance?
All best,
Cab Vinton, Director Plaistow Public Library
On Tue, Dec 15, 2020 at 6:04 PM <dcook@prosentient.com.au> wrote:
Hi all,
For a while, I’ve been thinking that it might be good to have an open source “Koha Command and Control” for managing Koha instances.
While managing 1 Koha is not too much work, it’s considerably more difficult to manage 10, 100, or 1000 Koha instances, especially across different servers, geographic locations, timezones, etc.
I am sure that most vendors have mechanisms in place for checking the system preference values for all their instances (I have scripts for this sort of thing), what about tracking Zebra queries that contain syntax errors (bug 27139)?
I know that we have HEA for sharing a lot of data, but there is some data (like search queries) that I don’t think most libraries would want to share with the community. But Koha administrators, especially vendors with a large number of Koha instances, want to know when problems arise.
I suppose that a person could just implement log file scanning for Zebra, but I was thinking about something more structured, which could be analyzed to provide data quality control. My thought is that we’d configure Koha to push events to the “Koha Command and Control” (which would be configured by the Koha system administrator – not librarian level). If Zebra logs a ‘ZOOM error 10014 "CCL parsing error"’, we probably want to tell the Koha user that we found no search results (nice user experience), but we want to flag with a Koha administrator that there is a problem. If Koha administrators (e.g. Koha vendors) start receiving a lot of these errors via the Koha CnC (Command and Control), they’re better placed to raise Bugzilla reports/write patches.
Anyway, it’s just an idea. Probably too ambitious. But I think that it would be wise for us to start thinking more about the *data* generated by Koha and how we can use that data to improve Koha. I don’t think we need machine learning algorithms to do the data analysis (yet), but I think having automated data collection, organisation, and reporting would be useful.
I have to run but I’ll keep thinking about it. I know I have more grand ideas than I have time, but I could put some thought into what a system might look like, and maybe throw something together locally first before suggesting anyone else get on board…
David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Hi, Indeed good idea. Maybe we could add a Koha plugin hook to add datas to this HEA. So poeple can add specific datas that may be private for the general HEA. "One HEA to rule them all" ^^ Le 16/12/2020 à 23:31, dcook@prosentient.com.au a écrit :
Hi Arthur,
Thanks for your email.
I use Ansible quite a bit already, so I see where you're coming from, but I was thinking of something more elegant. I was also thinking about something that would allow for more private data collection than Hea.
Although when I put it that way... it could be interesting to update C4::UsageStats::ReportToCommunity() to actually take a list of URLs, and people could run up their own private Hea (https://gitlab.com/koha-community/hea-app).
David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia
Office: 02 9212 0899 Online: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Arthur Sent: Thursday, 17 December 2020 2:15 AM To: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] Koha Command and Control
Hi there,
for this kind of purpose I would recommend using Ansible scripts to ship commands to several Koha's at once.
Using Ansible only needs an ssh connexion to each of your Koha servers, no specific software or agent to install on the targets, then commands are simple well-known bash.
It is also possible to group targets (manually) based on OS, web-server software or other tweaks to deploy specific sets of commands to each of your groups, this is very handy.
I personally don't use it because our environment is very heterogeneous, making grouping difficult and prone to error (and then we would have a lot of groups of a single instance...)
Best,
Arthur Suzuki
BibLibre Koha Support
On 16/12/2020 16:01, Cab Vinton wrote:
This popped into my email ... Above my pay grade, but maybe of relevance?
All best,
Cab Vinton, Director Plaistow Public Library
On Tue, Dec 15, 2020 at 6:04 PM <dcook@prosentient.com.au> wrote:
Hi all,
For a while, I’ve been thinking that it might be good to have an open source “Koha Command and Control” for managing Koha instances.
While managing 1 Koha is not too much work, it’s considerably more difficult to manage 10, 100, or 1000 Koha instances, especially across different servers, geographic locations, timezones, etc.
I am sure that most vendors have mechanisms in place for checking the system preference values for all their instances (I have scripts for this sort of thing), what about tracking Zebra queries that contain syntax errors (bug 27139)?
I know that we have HEA for sharing a lot of data, but there is some data (like search queries) that I don’t think most libraries would want to share with the community. But Koha administrators, especially vendors with a large number of Koha instances, want to know when problems arise.
I suppose that a person could just implement log file scanning for Zebra, but I was thinking about something more structured, which could be analyzed to provide data quality control. My thought is that we’d configure Koha to push events to the “Koha Command and Control” (which would be configured by the Koha system administrator – not librarian level). If Zebra logs a ‘ZOOM error 10014 "CCL parsing error"’, we probably want to tell the Koha user that we found no search results (nice user experience), but we want to flag with a Koha administrator that there is a problem. If Koha administrators (e.g. Koha vendors) start receiving a lot of these errors via the Koha CnC (Command and Control), they’re better placed to raise Bugzilla reports/write patches.
Anyway, it’s just an idea. Probably too ambitious. But I think that it would be wise for us to start thinking more about the *data* generated by Koha and how we can use that data to improve Koha. I don’t think we need machine learning algorithms to do the data analysis (yet), but I think having automated data collection, organisation, and reporting would be useful.
I have to run but I’ll keep thinking about it. I know I have more grand ideas than I have time, but I could put some thought into what a system might look like, and maybe throw something together locally first before suggesting anyone else get on board…
David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Fridolin SOMERS <fridolin.somers@biblibre.com> Software and system maintainer 🦄 BibLibre, France
Hi everyone, if Koha is aiming to replace zebra with elastic search, I think it would be best to use ELK Stack for log parsing and sending notifications. What do you think? On Thu, Dec 17, 2020 at 5:48 AM Fridolin SOMERS < fridolin.somers@biblibre.com> wrote:
Hi,
Indeed good idea.
Maybe we could add a Koha plugin hook to add datas to this HEA. So poeple can add specific datas that may be private for the general HEA.
"One HEA to rule them all" ^^
Hi Arthur,
Thanks for your email.
I use Ansible quite a bit already, so I see where you're coming from, but I was thinking of something more elegant. I was also thinking about something that would allow for more private data collection than Hea.
Although when I put it that way... it could be interesting to update C4::UsageStats::ReportToCommunity() to actually take a list of URLs, and
Le 16/12/2020 à 23:31, dcook@prosentient.com.au a écrit : people could run up their own private Hea ( https://gitlab.com/koha-community/hea-app).
David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia
Office: 02 9212 0899 Online: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On
Sent: Thursday, 17 December 2020 2:15 AM To: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] Koha Command and Control
Hi there,
for this kind of purpose I would recommend using Ansible scripts to ship commands to several Koha's at once.
Using Ansible only needs an ssh connexion to each of your Koha servers, no specific software or agent to install on the targets, then commands are simple well-known bash.
It is also possible to group targets (manually) based on OS, web-server software or other tweaks to deploy specific sets of commands to each of your groups, this is very handy.
I personally don't use it because our environment is very heterogeneous, making grouping difficult and prone to error (and then we would have a lot of groups of a single instance...)
Best,
Arthur Suzuki
BibLibre Koha Support
On 16/12/2020 16:01, Cab Vinton wrote:
This popped into my email ... Above my pay grade, but maybe of relevance?
All best,
Cab Vinton, Director Plaistow Public Library
On Tue, Dec 15, 2020 at 6:04 PM <dcook@prosentient.com.au> wrote:
Hi all,
For a while, I’ve been thinking that it might be good to have an open
Behalf Of Arthur source “Koha Command and Control” for managing Koha instances.
While managing 1 Koha is not too much work, it’s considerably more
difficult to manage 10, 100, or 1000 Koha instances, especially across different servers, geographic locations, timezones, etc.
I am sure that most vendors have mechanisms in place for checking the
system preference values for all their instances (I have scripts for this sort of thing), what about tracking Zebra queries that contain syntax errors (bug 27139)?
I know that we have HEA for sharing a lot of data, but there is some
data (like search queries) that I don’t think most libraries would want to share with the community. But Koha administrators, especially vendors with a large number of Koha instances, want to know when problems arise.
I suppose that a person could just implement log file scanning for
Zebra, but I was thinking about something more structured, which could be analyzed to provide data quality control. My thought is that we’d configure Koha to push events to the “Koha Command and Control” (which would be configured by the Koha system administrator – not librarian level). If Zebra logs a ‘ZOOM error 10014 "CCL parsing error"’, we probably want to tell the Koha user that we found no search results (nice user experience), but we want to flag with a Koha administrator that there is a problem. If Koha administrators (e.g. Koha vendors) start receiving a lot of these errors via the Koha CnC (Command and Control), they’re better placed to raise Bugzilla reports/write patches.
Anyway, it’s just an idea. Probably too ambitious. But I think that it
would be wise for us to start thinking more about the *data* generated by Koha and how we can use that data to improve Koha. I don’t think we need machine learning algorithms to do the data analysis (yet), but I think having automated data collection, organisation, and reporting would be useful.
I have to run but I’ll keep thinking about it. I know I have more grand ideas than I have time, but I could put some thought into what a system might look like, and maybe throw something together locally first before suggesting anyone else get on board…
David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Fridolin SOMERS <fridolin.somers@biblibre.com> Software and system maintainer 🦄 BibLibre, France _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
I have been wondering that as well. We aren’t using Elasticsearch yet, but if we were it would certainly be an interesting option. David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595 From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Agustin Moyano Sent: Friday, 18 December 2020 1:09 AM To: Fridolin SOMERS <fridolin.somers@biblibre.com> Cc: Koha Devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Koha Command and Control Hi everyone, if Koha is aiming to replace zebra with elastic search, I think it would be best to use ELK Stack for log parsing and sending notifications. What do you think? On Thu, Dec 17, 2020 at 5:48 AM Fridolin SOMERS <fridolin.somers@biblibre.com <mailto:fridolin.somers@biblibre.com> > wrote: Hi, Indeed good idea. Maybe we could add a Koha plugin hook to add datas to this HEA. So poeple can add specific datas that may be private for the general HEA. "One HEA to rule them all" ^^ Le 16/12/2020 à 23:31, dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> a écrit :
Hi Arthur,
Thanks for your email.
I use Ansible quite a bit already, so I see where you're coming from, but I was thinking of something more elegant. I was also thinking about something that would allow for more private data collection than Hea.
Although when I put it that way... it could be interesting to update C4::UsageStats::ReportToCommunity() to actually take a list of URLs, and people could run up their own private Hea (https://gitlab.com/koha-community/hea-app).
David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia
Office: 02 9212 0899 Online: 02 8005 0595
-----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of Arthur Sent: Thursday, 17 December 2020 2:15 AM To: koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Koha Command and Control
Hi there,
for this kind of purpose I would recommend using Ansible scripts to ship commands to several Koha's at once.
Using Ansible only needs an ssh connexion to each of your Koha servers, no specific software or agent to install on the targets, then commands are simple well-known bash.
It is also possible to group targets (manually) based on OS, web-server software or other tweaks to deploy specific sets of commands to each of your groups, this is very handy.
I personally don't use it because our environment is very heterogeneous, making grouping difficult and prone to error (and then we would have a lot of groups of a single instance...)
Best,
Arthur Suzuki
BibLibre Koha Support
On 16/12/2020 16:01, Cab Vinton wrote:
This popped into my email ... Above my pay grade, but maybe of relevance?
All best,
Cab Vinton, Director Plaistow Public Library
On Tue, Dec 15, 2020 at 6:04 PM <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote:
Hi all,
For a while, I’ve been thinking that it might be good to have an open source “Koha Command and Control” for managing Koha instances.
While managing 1 Koha is not too much work, it’s considerably more difficult to manage 10, 100, or 1000 Koha instances, especially across different servers, geographic locations, timezones, etc.
I am sure that most vendors have mechanisms in place for checking the system preference values for all their instances (I have scripts for this sort of thing), what about tracking Zebra queries that contain syntax errors (bug 27139)?
I know that we have HEA for sharing a lot of data, but there is some data (like search queries) that I don’t think most libraries would want to share with the community. But Koha administrators, especially vendors with a large number of Koha instances, want to know when problems arise.
I suppose that a person could just implement log file scanning for Zebra, but I was thinking about something more structured, which could be analyzed to provide data quality control. My thought is that we’d configure Koha to push events to the “Koha Command and Control” (which would be configured by the Koha system administrator – not librarian level). If Zebra logs a ‘ZOOM error 10014 "CCL parsing error"’, we probably want to tell the Koha user that we found no search results (nice user experience), but we want to flag with a Koha administrator that there is a problem. If Koha administrators (e.g. Koha vendors) start receiving a lot of these errors via the Koha CnC (Command and Control), they’re better placed to raise Bugzilla reports/write patches.
Anyway, it’s just an idea. Probably too ambitious. But I think that it would be wise for us to start thinking more about the *data* generated by Koha and how we can use that data to improve Koha. I don’t think we need machine learning algorithms to do the data analysis (yet), but I think having automated data collection, organisation, and reporting would be useful.
I have to run but I’ll keep thinking about it. I know I have more grand ideas than I have time, but I could put some thought into what a system might look like, and maybe throw something together locally first before suggesting anyone else get on board…
David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Fridolin SOMERS <fridolin.somers@biblibre.com <mailto:fridolin.somers@biblibre.com> > Software and system maintainer 🦄 BibLibre, France _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Hi Cab, I was thinking a bit about how there is no point re-inventing the wheel. I've never heard of New Relic, but that reminds me about Prometheus (https://en.wikipedia.org/wiki/Prometheus_(software)). I could write a Koha plugin to create an "exporter" for Prometheus. Or even use something like https://github.com/fstab/grok_exporter to read Zebra logs. I think that would handle the data collection aspect in a more standardized way than I was originally thinking. I know many people would say that monitoring isn't part of the application, but a lot of applications these days have Prometheus exporters out of the box. Hmm food for thought. As for the command aspect, an Admin API is the way to go with that. And if something can't be handled by the API, perhaps we should re-think how it's done. (Containers help with this. For instance, instead of updating a configuration file and restarting Apache, you would actually commit a change, use a deployment pipeline, and the container(s) would ultimately be replaced via automation.) Anyway, thanks for indulging me heh. David Cook Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595 -----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Cab Vinton Sent: Thursday, 17 December 2020 2:02 AM To: Koha Devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Koha Command and Control This popped into my email ... Above my pay grade, but maybe of relevance? https://newrelic.com/ All best, Cab Vinton, Director Plaistow Public Library On Tue, Dec 15, 2020 at 6:04 PM <dcook@prosentient.com.au> wrote:
Hi all,
For a while, I’ve been thinking that it might be good to have an open source “Koha Command and Control” for managing Koha instances.
While managing 1 Koha is not too much work, it’s considerably more difficult to manage 10, 100, or 1000 Koha instances, especially across different servers, geographic locations, timezones, etc.
I am sure that most vendors have mechanisms in place for checking the system preference values for all their instances (I have scripts for this sort of thing), what about tracking Zebra queries that contain syntax errors (bug 27139)?
I know that we have HEA for sharing a lot of data, but there is some data (like search queries) that I don’t think most libraries would want to share with the community. But Koha administrators, especially vendors with a large number of Koha instances, want to know when problems arise.
I suppose that a person could just implement log file scanning for Zebra, but I was thinking about something more structured, which could be analyzed to provide data quality control. My thought is that we’d configure Koha to push events to the “Koha Command and Control” (which would be configured by the Koha system administrator – not librarian level). If Zebra logs a ‘ZOOM error 10014 "CCL parsing error"’, we probably want to tell the Koha user that we found no search results (nice user experience), but we want to flag with a Koha administrator that there is a problem. If Koha administrators (e.g. Koha vendors) start receiving a lot of these errors via the Koha CnC (Command and Control), they’re better placed to raise Bugzilla reports/write patches.
Anyway, it’s just an idea. Probably too ambitious. But I think that it would be wise for us to start thinking more about the *data* generated by Koha and how we can use that data to improve Koha. I don’t think we need machine learning algorithms to do the data analysis (yet), but I think having automated data collection, organisation, and reporting would be useful.
I have to run but I’ll keep thinking about it. I know I have more grand ideas than I have time, but I could put some thought into what a system might look like, and maybe throw something together locally first before suggesting anyone else get on board…
David Cook
Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
participants (5)
-
Agustin Moyano -
Arthur -
Cab Vinton -
dcook＠prosentient.com.au -
Fridolin SOMERS