[Koha-devel] Dedicated Zebra Server

[Kivilahti Olli-Antti]

Hi!

We install Koha in a cluster of LXC containers:

One haproxy for dns redirection and ssl offloading
One container for MariaDB
One for OPAC
One for staff client
One for Zebra
One for misc stuff, like the SIP2-server
One for ElasticSearch
One for Memcached
+containers for other support services

With haproxy it is rather straight forward to start load balancing multiple "web bricks" if you choose to. We don't need to do it since we chose to buy a lot of hardware. Hardware is less expensive than work where we live in.
This clusterization is simply to increase robustness and security.
And actually has saved our bacon few times during the past years, when we get unidentifiable hard-drive freezes with the SIP2-container,
or some cronjob has went awol and completely hogged all system resources available to it.
Without containerization we would have had downtime.
Also helps with simple DOS-attacks which can happen by accident.

I recommend this type of setup.

Unfortunately the debian packages add a lot of unnecesary cruft to each container so we use dev-install instead.

Getting Zebra to work in a separate container:
Zebra is really really really tightly wound into the make-toolchain Koha uses and there is no convenient way of separating it. Maybe you could just bind mount the zebradb-directory between containers. This might be enough to run Zebra separate of the Koha's source code.

Since mass storage is very cheap, we just run two identical Koha instances from the same source code repo. We make several identical Koha-installations via the dev-install.
The Zebra-one only has Zebra and Koha sources (no Apache2, MariaDB, etc packages needed by a standalone Koha).
You can configure Zebra to listen on tcp socket via the $KOHA_CONF. You can conveniently centrally configure the same $KOHA_CONF to all the Koha containers, so they all find the same Zebra by the ip and the same MariaDB. This is best done with Ansible (since community has already started working towards Ansible)


Separating Zebra to a isolated (and easily monitorable) environment is entirely possible.



Here is a snippet from our Ansible infrastructure definitions where Koha is set up with capability: zebra

These should be most of the Ansible steps needed together with Koha-source to get Zebra running standalone:



ansible at hephaestus:~/KSAnsible$ cat roles/koha/tasks/capabilities/zebra.yml

---

- name: Install zebrasrv and zebraidx
  apt:
    name: idzebra-2.0
  become: yes

- name: Create the zebra socket-dir and permissions
  file:
    path: "{{zebra_run_dir}}"
    owner: koha
    group: koha
    state: directory
  become: yes

- name: Create the zebra lock-dir and permissions
  file:
    path: "{{zebra_lock_dir}}"
    owner: koha
    group: koha
    state: directory
  become: yes

- name: Create the zebra data-dir and permissions
  file:
    path: "{{zebra_data_dir}}"
    owner: koha
    group: koha
    state: directory
  become: yes


- name: Configure Zebra stripes
  lineinfile:
    path:   "{{item.path}}"
    regexp: "{{item.regexp}}"
    line:   "{{item.line}}"
  with_items:
    - path:   "{{koha_dev_path}}/etc/zebradb/zebra-biblios-dom.cfg"
      regexp: "^memMax:"
      line:    "memMax: {{zebra_index_mem}}"
    - path:   "{{koha_dev_path}}/etc/zebradb/zebra-biblios-dom.cfg"
      regexp: "^register:"
      line:    "register: {{zebra_data_dir}}/biblios/register:{{zebra_index_size}}"
    - path:   "{{koha_dev_path}}/etc/zebradb/zebra-biblios-dom.cfg"
      regexp: "^shadow:"
      line:    "shadow: {{zebra_data_dir}}/biblios/shadow:{{zebra_index_size}}"
    - path:   "{{koha_dev_path}}/etc/zebradb/zebra-biblios-dom.cfg"
      regexp: "^sortmax:"
      line:    "sortmax: {{zebra_sort_max}}"
    - path:   "{{koha_dev_path}}/etc/zebradb/zebra-biblios-dom.cfg"
      regexp: "^facetNumRecs:"
      line:    "facetNumRecs:{{zebra_facet_num_records}}"
      owner: koha
  become: yes
  notify: Restart Zebra

- name: Zebra - Link Zebra-service
  file:
    src:   "{{koha_dev_path}}/bin/koha-zebra-ctl.sh"
    dest:  "/etc/init.d/koha-zebra-daemon"
    owner: root
    group: root
    state: "link"
  become: yes

- name: Zebra - Enable Zebra-service
  systemd:
    name: koha-zebra-daemon
    enabled: yes
    daemon_reload: yes
    state: started
  become: yes

- name: Zebra-index - Link Zebra-index-service
  file:
    src:   "{{koha_dev_path}}/bin/koha-index-daemon-ctl.sh"
    dest:  "/etc/init.d/koha-index-daemon"
    owner: root
    group: root
    state: "link"
  become: yes

- name: Zebra-index - Enable Zebra-index-service
  systemd:
    name: koha-index-daemon
    enabled: yes
    daemon_reload: yes
    state: started
  become: yes

- name: Deploy Zebra cronjobs
  template:
    owner: "{{koha_user}}"
    src:   "etc_cron.d_koha.j2"
    dest:  "/etc/cron.d/koha-zebra"
  vars:
    cronjobs:
      rbza:
        comment: '#Trigger full Zebra reindexing daily'
        prefix:  ' '
        timing:  '46 10 * * *'
        user:    'koha'
        command: 'migration_tools/rebuild_zebra.pl -b -a -r -v -x'
  become: yes




________________________________
From: koha-devel-bounces at lists.koha-community.org [koha-devel-bounces at lists.koha-community.org] on behalf of Mansur Ali [mansuralih at gmail.com]
Sent: Tuesday, July 18, 2017 3:11 PM
To: Tajoli Zeno
Cc: koha-devel at lists.koha-community.org
Subject: Re: [Koha-devel] Dedicated Zebra Server

Thank you for your valuable information.



On Tue, Jul 18, 2017 at 2:59 PM, Tajoli Zeno <z.tajoli at cineca.it<mailto:z.tajoli at cineca.it>> wrote:
Hi,

Il 18/07/2017 13:28, Mansur Ali ha scritto:
Hello,

I am using KOHA ILS. Can i able to use dedicated Zebra Server which is installed in another server? Could you please advice if possible?

with present standard installation, the answer is no, it is not possible.

In theory if:
1)You setup to use PazPar instead of Zebra [an old option not tested from years] AND
2)You rewrite the updating scripts and deamon

you can try to do it.

I suggest you to do a try only if you are very skilled on Zebra and others Indexdata's tools.

Bye
Zeno Tajoli

--
Zeno Tajoli
/SVILUPPO PRODOTTI CINECA/ - Automazione Biblioteche
Email: z.tajoli at cineca.it<mailto:z.tajoli at cineca.it> Fax: 051/6132198
*CINECA* Consorzio Interuniversitario - Sede operativa di Segrate (MI)



--
Mansur Ali

+966568978721 | +919061635954 | mansuralih at gmail.com<mailto:mansuralih at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20170718/fe522ba1/attachment-0001.html>