<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Segoe UI Symbol";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-AU link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Thanks, Agustin.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I haven’t played with Helm yet, but I recognize the Kubernetes manifests in there. It looks interesting. When will these 2 projects be ready for testing?<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>With koha-docker, have you thought about using “secrets” rather than environmental variables for things like passwords? Reference <a href="https://docs.docker.com/compose/compose-file/#secrets">https://docs.docker.com/compose/compose-file/#secrets</a>. Secrets could be used for the Koha images too following a model like <a href="https://github.com/docker-library/mysql/blob/b0f81a33748561ae4e35a09895b2ad112ff89ba6/8.0/docker-entrypoint.sh">https://github.com/docker-library/mysql/blob/b0f81a33748561ae4e35a09895b2ad112ff89ba6/8.0/docker-entrypoint.sh</a>. That could be useful if someone wanted to use Amazon ECS or something instead of docker-compose too actually.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>David Cook<o:p></o:p></p><p class=MsoNormal>Software Engineer<o:p></o:p></p><p class=MsoNormal>Prosentient Systems<o:p></o:p></p><p class=MsoNormal>72/330 Wattle St<o:p></o:p></p><p class=MsoNormal>Ultimo, NSW 2007<o:p></o:p></p><p class=MsoNormal>Australia<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Office: 02 9212 0899<o:p></o:p></p><p class=MsoNormal>Online: 02 8005 0595<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US>From:</span></b><span lang=EN-US> Agustin Moyano <agustinmoyano@theke.io> <br><b>Sent:</b> Wednesday, 19 August 2020 1:51 AM<br><b>To:</b> dcook@prosentient.com.au<br><b>Cc:</b> Tomás <tomas.cohen@gmail.com>; koha-devel <koha-devel@lists.koha-community.org><br><b>Subject:</b> Re: [Koha-devel] Deploying Koha at scale<o:p></o:p></span></p></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>Hi David, we've got two separate projects.<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>One is koha-docker (<a href="https://gitlab.com/thekesolutions/koha-docker">https://gitlab.com/thekesolutions/koha-docker</a>) that has a "docker-compose" approach, and the second one is koha-helm-chart (<a href="https://gitlab.com/thekesolutions/charts/koha-helm-chart">https://gitlab.com/thekesolutions/charts/koha-helm-chart</a>) where we will implement the kubernetes cronjob approach.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Please take a look at that one too.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Regards<o:p></o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>On Mon, Aug 17, 2020 at 9:51 PM <<a href="mailto:dcook@prosentient.com.au">dcook@prosentient.com.au</a>> wrote:<o:p></o:p></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hey Tom<span lang=EN-US>á</span>s,<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I remembered your email and I thought that I would check in. How’s work on koha-docker going? It looks interesting.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>It’s too bad the “koha-common” package name is already used, as it would be interesting to package just the core libraries and scripts to create a really minimal Koha image to use for the Z39.50, SIP, Indexer, etc. <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I look forward to a time when we don’t need Apache to do CGI scripts and just function as a reverse proxy and front end web server of static assets (or just the a web server of static assets and Traefik or something could be used as the reverse proxy). <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I think that I’m most intrigued with the cron service*, as I think that’s probably the hardest thing to containerize and scale well. <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I’ll have a look at running this up at some point!<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>*(Kubernetes seems to have a neat approach to cronjobs <a href="https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/" target="_blank">https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/</a>. Julia Evans talks about it at <a href="https://stripe.com/blog/operating-kubernetes" target="_blank">https://stripe.com/blog/operating-kubernetes</a>. However, that would require a Kubernetes-first design, which makes no sense for Koha.)<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>David Cook<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Software Engineer<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Prosentient Systems<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>72/330 Wattle St<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Ultimo, NSW 2007<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Australia<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Office: 02 9212 0899<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Online: 02 8005 0595<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span lang=EN-US>From:</span></b><span lang=EN-US> Koha-devel <<a href="mailto:koha-devel-bounces@lists.koha-community.org" target="_blank">koha-devel-bounces@lists.koha-community.org</a>> <b>On Behalf Of </b>Tomás<br><b>Sent:</b> Friday, 12 June 2020 11:57 AM<br><b>To:</b> koha-devel <<a href="mailto:koha-devel@lists.koha-community.org" target="_blank">koha-devel@lists.koha-community.org</a>><br><b>Subject:</b> [Koha-devel] Deploying Koha at scale</span><o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi folks.<o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Theke Solutions has started a project some time ago, sponsored by ByWater Solutions to generate the tools and pipelines to provide a reliable Docker implementation for deploying Koha in production environments. We have just finished the housekeeping a few days ago so we are not embarrassed by our commit messages and leftovers :-D and want to propose it for community adoption anytime soon.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>The project lives here for now: <a href="https://gitlab.com/thekesolutions/koha-docker" target="_blank">https://gitlab.com/thekesolutions/koha-docker</a><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>What you will find there is a Docker image that can be used to run any of the different Koha services/pieces on separate containers, or all inside a single one. We aimed to target different deployment scenarios, from a small library running it on a desktop computer to a multi-cluster K8s setup. A helm chart is part of the roadmap.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>You will notice there are a couple issues to be addressed [1]. For instance, the shipped docker-compose.yml is outdated regarding the MySQL/MariaDB server and we definitely need more on the README.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thanks to ByWater for supporting this effort, and now that people are talking about this, let's start the conversation \o/<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>[1] <a href="https://gitlab.com/thekesolutions/koha-docker/-/issues" target="_blank">https://gitlab.com/thekesolutions/koha-docker/-/issues</a><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>El jue., 11 jun. 2020 a las 4:52, <<a href="mailto:dcook@prosentient.com.au" target="_blank">dcook@prosentient.com.au</a>> escribió:<o:p></o:p></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt'><p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'>Hi all,<br><br>For a long time, I've been thinking about how Koha could scale better, and<br>I'm wondering what other people are thinking about this topic. I think Mengu<br>is probably the best expert on this topic that we have on Earth, so<br>especially curious about his input. <br><br>It's relatively easy to add more Apache web servers, more MySQL nodes, and<br>more Elasticsearch nodes. but the same might not be said for other parts of<br>Koha like the following:<br><br>1. Zebra<br> a. In theory, you could load balance queries to a cluster of Zebra<br>listeners<br> b. But indexing updates to many Zebra databases could be tricky (I<br>don't know if Zebra's database can support many readers). I suppose you<br>could alter <a href="http://rebuild_zebra.pl" target="_blank">rebuild_zebra.pl</a> to update multiple databases in a variety of<br>different ways. I imagine Mengu could speak to this, as I recall him talking<br>once about needing to do work to make Zebra scale?<br><br>2. I'm not familiar enough with the SIP server to say<br> a. Off the top of my head, I think a TCP load balancer would be good<br>enough here<br><br>3. Z3950 Responder? <br> a. I'm not familiar with this daemon<br><br>4. What about cronjobs? Only 1 instance should execute the cronjobs.<br> a. I suppose a person could disable cronjobs on all Koha nodes<br>except for 1? It wouldn't be very robust in terms of availability but it<br>would technically work most of the time. <br><br>Any other thoughts?<br><br>David Cook<br>Systems Librarian<br>Prosentient Systems<br>72/330 Wattle St<br>Ultimo, NSW 2007<br>Australia<br><br>Office: 02 9212 0899<br>Online: 02 8005 0595<o:p></o:p></p></blockquote></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><br clear=all><o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>-- <o:p></o:p></p><div><div><div><div><div><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Tomás Cohen Arazi<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Theke Solutions (<a href="http://theke.io" target="_blank">http://theke.io</a>)<br><span style='font-family:"Segoe UI Symbol",sans-serif'>✆</span> +54 9351 3513384<br><span style='font-size:9.5pt'>GPG: B2F3C15F</span><o:p></o:p></p></div></div></div></div></div></div></div></div></div></div></div></div><p class=MsoNormal>_______________________________________________<br>Koha-devel mailing list<br><a href="mailto:Koha-devel@lists.koha-community.org" target="_blank">Koha-devel@lists.koha-community.org</a><br><a href="https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel" target="_blank">https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel</a><br>website : <a href="http://www.koha-community.org/" target="_blank">http://www.koha-community.org/</a><br>git : <a href="http://git.koha-community.org/" target="_blank">http://git.koha-community.org/</a><br>bugs : <a href="http://bugs.koha-community.org/" target="_blank">http://bugs.koha-community.org/</a><o:p></o:p></p></blockquote></div></div></body></html>