[Koha-devel] Dedicated Zebra Server
Mansur Ali
mansuralih at gmail.com
Fri Jul 21 09:43:39 CEST 2017
Thank you for your tremendous support.
I have successfully integrated dedicated zebra server over TCP connection.
In my existing system I am getting an error (*** Error in `zebraidx':
corrupted double-linked list: 0x00000000012c6a30 ***) while indexing large
database. Because of that I was looking the workaround and finally get
done.
I am glad to help somebody want to do this integration.
On Wed, Jul 19, 2017 at 5:09 AM, David Cook <dcook at prosentient.com.au>
wrote:
> A sizeable portion of koha-conf.xml is actually Zebra configuration. An
> interesting task would be to separate out all the Zebra-specific stuff and
> put it in its own file, so that it was less intimidating and perhaps easier
> to manage.
>
>
>
> For searching, Koha is just using a Z39.50 connection, so Zebra could be
> accessed over TCP on a different server.
>
>
>
> For indexing, the indexing needs to happen on the Zebra server, and in
> order to index you need to have records from the Koha database. There’s a
> number of ways you could handle that I suppose… but yeah it currently is
> closely tied to the Koha application monolith. In theory, if your MySQL
> database was on yet another server, you’d just need an indexing script and
> the credentials to access the MySQL database.
>
>
>
> I suppose there are some system preferences that get taken into account as
> well… but that’s still coming from the database.
>
>
>
> In theory, I would think the “koha-common” package would contain the
> libraries needed for anything interfacing with Koha and then a person could
> use Ansible to keep the remote servers all consistently configured.
>
>
>
> Anyway, all that would require some work, which I know I’m not planning to
> do. At this point, it’s probably more worthwhile to focus on Elasticsearch
> development, although I think the plan is to keep Zebra around anyway, so
> decoupling it a bit from the Koha monolith wouldn’t necessarily be a bad
> idea…
>
>
>
> David Cook
>
> Systems Librarian
>
> Prosentient Systems
>
> 72/330 Wattle St
>
> Ultimo, NSW 2007
>
> Australiarpm
>
>
>
> Office: 02 9212 0899
>
> Direct: 02 8005 0595
>
>
>
> *From:* koha-devel-bounces at lists.koha-community.org [mailto:
> koha-devel-bounces at lists.koha-community.org] *On Behalf Of *Michael Hafen
> *Sent:* Wednesday, 19 July 2017 2:45 AM
> *To:* Kivilahti Olli-Antti <olli-antti.kivilahti at jns.fi>
>
> *Cc:* koha-devel at lists.koha-community.org
> *Subject:* Re: [Koha-devel] Dedicated Zebra Server
>
>
>
> I'm thinking of doing something like this too. I'd really like to see the
> zebra var's from the koha_conf.xml too, if possible. Thanks.
>
>
>
> On Tue, Jul 18, 2017 at 8:37 AM, Kivilahti Olli-Antti <
> olli-antti.kivilahti at jns.fi> wrote:
>
> Hi!
>
> We install Koha in a cluster of LXC containers:
>
> One haproxy for dns redirection and ssl offloading
> One container for MariaDB
> One for OPAC
> One for staff client
> One for Zebra
> One for misc stuff, like the SIP2-server
> One for ElasticSearch
> One for Memcached
> +containers for other support services
>
> With haproxy it is rather straight forward to start load balancing
> multiple "web bricks" if you choose to. We don't need to do it since we
> chose to buy a lot of hardware. Hardware is less expensive than work where
> we live in.
> This clusterization is simply to increase robustness and security.
> And actually has saved our bacon few times during the past years, when we
> get unidentifiable hard-drive freezes with the SIP2-container,
> or some cronjob has went awol and completely hogged all system resources
> available to it.
> Without containerization we would have had downtime.
> Also helps with simple DOS-attacks which can happen by accident.
>
> I recommend this type of setup.
>
> Unfortunately the debian packages add a lot of unnecesary cruft to each
> container so we use dev-install instead.
>
> Getting Zebra to work in a separate container:
> Zebra is really really really tightly wound into the make-toolchain Koha
> uses and there is no convenient way of separating it. Maybe you could just
> bind mount the zebradb-directory between containers. This might be enough
> to run Zebra separate of the Koha's source code.
>
> Since mass storage is very cheap, we just run two identical Koha instances
> from the same source code repo. We make several identical
> Koha-installations via the dev-install.
> The Zebra-one only has Zebra and Koha sources (no Apache2, MariaDB, etc
> packages needed by a standalone Koha).
> You can configure Zebra to listen on tcp socket via the $KOHA_CONF. You
> can conveniently centrally configure the same $KOHA_CONF to all the Koha
> containers, so they all find the same Zebra by the ip and the same MariaDB.
> This is best done with Ansible (since community has already started working
> towards Ansible)
>
>
> Separating Zebra to a isolated (and easily monitorable) environment is
> entirely possible.
>
>
>
> Here is a snippet from our Ansible infrastructure definitions where Koha
> is set up with capability: zebra
>
> These should be most of the Ansible steps needed together with Koha-source
> to get Zebra running standalone:
>
>
>
> ansible at hephaestus:~/KSAnsible$ cat roles/koha/tasks/capabilities/
> zebra.yml
>
> ---
>
> - name: Install zebrasrv and zebraidx
> apt:
> name: idzebra-2.0
> become: yes
>
> - name: Create the zebra socket-dir and permissions
> file:
> path: "{{zebra_run_dir}}"
> owner: koha
> group: koha
> state: directory
> become: yes
>
> - name: Create the zebra lock-dir and permissions
> file:
> path: "{{zebra_lock_dir}}"
> owner: koha
> group: koha
> state: directory
> become: yes
>
> - name: Create the zebra data-dir and permissions
> file:
> path: "{{zebra_data_dir}}"
> owner: koha
> group: koha
> state: directory
> become: yes
>
>
> - name: Configure Zebra stripes
> lineinfile:
> path: "{{item.path}}"
> regexp: "{{item.regexp}}"
> line: "{{item.line}}"
> with_items:
> - path: "{{koha_dev_path}}/etc/zebradb/zebra-biblios-dom.cfg"
> regexp: "^memMax:"
> line: "memMax: {{zebra_index_mem}}"
> - path: "{{koha_dev_path}}/etc/zebradb/zebra-biblios-dom.cfg"
> regexp: "^register:"
> line: "register: {{zebra_data_dir}}/biblios/
> register:{{zebra_index_size}}"
> - path: "{{koha_dev_path}}/etc/zebradb/zebra-biblios-dom.cfg"
> regexp: "^shadow:"
> line: "shadow: {{zebra_data_dir}}/biblios/
> shadow:{{zebra_index_size}}"
> - path: "{{koha_dev_path}}/etc/zebradb/zebra-biblios-dom.cfg"
> regexp: "^sortmax:"
> line: "sortmax: {{zebra_sort_max}}"
> - path: "{{koha_dev_path}}/etc/zebradb/zebra-biblios-dom.cfg"
> regexp: "^facetNumRecs:"
> line: "facetNumRecs:{{zebra_facet_num_records}}"
> owner: koha
> become: yes
> notify: Restart Zebra
>
> - name: Zebra - Link Zebra-service
> file:
> src: "{{koha_dev_path}}/bin/koha-zebra-ctl.sh"
> dest: "/etc/init.d/koha-zebra-daemon"
> owner: root
> group: root
> state: "link"
> become: yes
>
> - name: Zebra - Enable Zebra-service
> systemd:
> name: koha-zebra-daemon
> enabled: yes
> daemon_reload: yes
> state: started
> become: yes
>
> - name: Zebra-index - Link Zebra-index-service
> file:
> src: "{{koha_dev_path}}/bin/koha-index-daemon-ctl.sh"
> dest: "/etc/init.d/koha-index-daemon"
> owner: root
> group: root
> state: "link"
> become: yes
>
> - name: Zebra-index - Enable Zebra-index-service
> systemd:
> name: koha-index-daemon
> enabled: yes
> daemon_reload: yes
> state: started
> become: yes
>
> - name: Deploy Zebra cronjobs
> template:
> owner: "{{koha_user}}"
> src: "etc_cron.d_koha.j2"
> dest: "/etc/cron.d/koha-zebra"
> vars:
> cronjobs:
> rbza:
> comment: '#Trigger full Zebra reindexing daily'
> prefix: ' '
> timing: '46 10 * * *'
> user: 'koha'
> command: 'migration_tools/rebuild_zebra.pl -b -a -r -v -x'
> become: yes
>
>
>
> ------------------------------
>
> *From:* koha-devel-bounces at lists.koha-community.org [
> koha-devel-bounces at lists.koha-community.org] on behalf of Mansur Ali [
> mansuralih at gmail.com]
> *Sent:* Tuesday, July 18, 2017 3:11 PM
> *To:* Tajoli Zeno
> *Cc:* koha-devel at lists.koha-community.org
> *Subject:* Re: [Koha-devel] Dedicated Zebra Server
>
> Thank you for your valuable information.
>
>
>
> On Tue, Jul 18, 2017 at 2:59 PM, Tajoli Zeno <z.tajoli at cineca.it> wrote:
>
> Hi,
>
> Il 18/07/2017 13:28, Mansur Ali ha scritto:
>
> Hello,
>
> I am using KOHA ILS. Can i able to use dedicated Zebra Server which is
> installed in another server? Could you please advice if possible?
>
>
> with present standard installation, the answer is no, it is not possible.
>
> In theory if:
> 1)You setup to use PazPar instead of Zebra [an old option not tested from
> years] AND
> 2)You rewrite the updating scripts and deamon
>
> you can try to do it.
>
> I suggest you to do a try only if you are very skilled on Zebra and others
> Indexdata's tools.
>
> Bye
> Zeno Tajoli
>
> --
> Zeno Tajoli
> /SVILUPPO PRODOTTI CINECA/ - Automazione Biblioteche
> Email: z.tajoli at cineca.it Fax: 051/6132198
> *CINECA* Consorzio Interuniversitario - Sede operativa di Segrate (MI)
>
>
>
>
> --
>
> Mansur Ali
>
> +966568978721 <+966%2056%20897%208721> | +919061635954 |
> mansuralih at gmail.com
>
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
>
>
>
>
> --
>
> Michael Hafen
>
> Washington County School District Technology Department
>
> Systems Analyst
>
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : http://www.koha-community.org/
> git : http://git.koha-community.org/
> bugs : http://bugs.koha-community.org/
>
--
Mansur Ali
+966568978721 | +919061635954 | mansuralih at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20170721/2b45e93c/attachment-0001.html>
More information about the Koha-devel
mailing list