[Koha-devel] 16.05, zebra and jessie

[Paul A]

At 03:57 PM 8/23/2016 +0100, Jonathan Druart wrote:
>2016-08-23 15:43 GMT+01:00 Paul A <paul.a at navalmarinearchive.com>:
> > At 09:21 AM 8/23/2016 -0400, Barton Chittenden wrote:
> >
> > Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to
> > disk (~16G/query on large libraries). Bug 13665 mentions that searches
> > could be taking upwards of 40 seconds to complete -- I think that we've 
> seen
> > searches time out and return no results at about 1 minute.
> >
> >
> > When I looked into "speed" some time ago
> > <http://navalmarinearchive.com/z_koha/>, it was the fact that multi-core
> > processing was not possible (Intel 8-core I7 machine.)  Speed is the only
> > reason that we decided to stay with 3.8.24 rather than upgrade.
>
>Could you stop propagating this argument and have a look at what has
>been done recently on the performance side?
>You should reconsider your choice and have a look at 16.05, or at
>least maintained releases.
>Unless you backport the security / bug fixes by yourself?
>
>Did you compare with 16.05.x with memcached + plack? That's what you
>should do, instead of comparing 3.8 vs 3.18

Yes, I did. Please see my email to this list Tue, 28 Jun 2016 20:46:04 
-0400. (8 core Intel I7, 16 Gigs RAM, SSD) -- and it's still between three 
and four times slower. Slight difference, it was on Ubuntu 16.04, but with 
MySQL re-installed to the "older" standard.

I only "propagated" the 3.18 comparison because (a) I wrote it up, fully 
annotated, and it has all the NYTProf analyses, and (b) it makes the point 
about (not) multi-threading. I did not do such a full analysis with 16.05.

And I do backport (or otherwise) fix security issues if and when I become 
aware of them...

Best -- Paul



> > Best -- Paul
> >
> >
> > Is it possible to tune Zebra's space/time optimizations in any way so that
> > it doesn't write such large files to disk?
> >
> > On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart
> > <jonathan.druart at bugs.koha-community.org> wrote:
> > See bug 13665 - Retrieve facets from zebra is slow
> > To understand why and when use_zebra_facet=1 is slow
> >
> > 2016-08-22 21:31 GMT+01:00 Barton Chittenden <barton at bywatersolutions.com>:
> >> I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I
> >> have run into this, when using ICU-Chains:
> >>
> >> Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1
> >> URLÂ  Â  Â  Â :
> >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581
> >> Priority  : P5 - low
> >> Urgency   : enhancement
> >> Status    : NEW
> >>
> >> I also know that when use_zebra_facets was first introduced, it was *very*
> >> slow -- I can't find any bugs about that though. It's possible that it got
> >> so slow under idzebra-2.0 2.61 that the searches are timing out.
> >>
> >> It should be possible to set the logging for zebra so that you can see the
> >> PQF queries:
> >>
> >> See
> >>
> >> Bug 15714 : Remove zebra.log from debian scripts and add optional log
> >> levels
> >> URLÂ  Â  Â  Â :
> >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714
> >> Priority  : P5 - low
> >> Urgency   : enhancement
> >> Status    : RESOLVED
> >>
> >> For setting the log levels
> >>
> >> And
> >> http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html
> >>
> >> For a general discussion of how to use them.
> >>
> >> ... This should give you some idea of what's failing, both in terms of the
> >> dashes in 2.0.59 and the non-functional zebra facets under 2.0.61.
> >>
> >> My general feeling is that 2.0.59 is irredeemably broken by bug 16581, and
> >> we need at least 2.0.60, but I don't have any experience with zebra
> >> facets.
> >>
> >> --Barton
> >>
> >>
> >>
> >> On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett <mtompset at hotmail.com>
> >> wrote:
> >>>
> >>> Greetings,
> >>>
> >>> Similar problem. I hope someone has a better solution than setting it to
> >>> 0.
> >>>
> >>> GPML,
> >>> Mark Tompsett
> >>>
> >>> -----Original Message-----
> >>> From: Philippe Blouin
> >>> Sent: Monday, August 22, 2016 2:40 PM
> >>> To: koha-devel at lists.koha-community.org
> >>> Subject: [Koha-devel] 16.05, zebra and jessie
> >>>
> >>> Hello!
> >>>
> >>> We're trying to find the correction combination.  We're new on Jessie,
> >>> so we still have some tweaking needed...
> >>>
> >>> - By default, we get zebra 2.00.59 installed on Jessie through the
> >>> packages.
> >>> - On 16.05, we get some very bad results in the search when the itemtype
> >>> contains an hyphen (-), like 'A-DOC'.
> >>> - So we installed zebra 2.00.62.  This fixes the search...
> >>> - But now we do not have facets.
> >>> - So we set <use_zebra_facets>0</use_zebra_facets>
> >>> - And now we have facets.  But this feels... wrong?
> >
> >>>
> >>> My dummy question: what is the supposedly correct version of Zebra on
> >>> Jessie ?
> >>> And we're we correct in setting the config to 0 ?
> >>>
> >>> Thanks
> >>> Blou
> >>> _______________________________________________
> >>> Koha-devel mailing list
> >>> Koha-devel at lists.koha-community.org
> >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> >>> website : http://www.koha-community.org/
> >>> git : http://git.koha-community.org/
> >>> bugs : http://bugs.koha-community.org/
> >>>
> >>> _______________________________________________
> >>> Koha-devel mailing list
> >>> Koha-devel at lists.koha-community.org
> >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> >>> website : http://www.koha-community.org/
> >>> git : http://git.koha-community.org/
> >>> bugs : http://bugs.koha-community.org/
> >>
> >>
> >>
> >> _______________________________________________
> >> Koha-devel mailing list
> >> Koha-devel at lists.koha-community.org
> >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> >> website : http://www.koha-community.org/
> >> git : http://git.koha-community.org/
> >> bugs : http://bugs.koha-community.org/
> >
> >
> > _______________________________________________
> > Koha-devel mailing list
> > Koha-devel at lists.koha-community.org
> > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> > website : http://www.koha-community.org/
> > git : http://git.koha-community.org/
> > bugs : http://bugs.koha-community.org/
> >
> > ---
> > Maritime heritage and history, preservation and conservation,
> > research and education through the written word and the arts.
> > <http://NavalMarineArchive.com> and <http://UltraMarine.ca>
> >
> >
> > _______________________________________________
> > Koha-devel mailing list
> > Koha-devel at lists.koha-community.org
> > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> > website : http://www.koha-community.org/
> > git : http://git.koha-community.org/
> > bugs : http://bugs.koha-community.org/
>_______________________________________________
>Koha-devel mailing list
>Koha-devel at lists.koha-community.org
>http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>website : http://www.koha-community.org/
>git : http://git.koha-community.org/
>bugs : http://bugs.koha-community.org/

---
Maritime heritage and history, preservation and conservation,
research and education through the written word and the arts.
<http://NavalMarineArchive.com> and <http://UltraMarine.ca>