<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI Symbol";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-AU link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><a name="_MailEndCompose"><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'>If I had more time, I’d patch Zebra…<o:p></o:p></span></a></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'>It’s been a while since I looked at this. Is the problem that no facets are returned if one of them would return zero, or that no facets are returned if one of the fields specified in the facet browse is an empty index/un-indexed field in Zebra? If it’s the latter, I wonder if using the Zebra specific search extension “14” in the main query would help… (see “Specifies whether un-indexed fields should be ignored. A zero value (default) throws a diagnostic when an un-indexed field is specified. A non-zero value makes it return 0 hits.” </span><a href="http://www.indexdata.com/zebra/doc/zebra.pdf"><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'>http://www.indexdata.com/zebra/doc/zebra.pdf</span></a><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'>). <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'>I suppose Adam at IndexData has been busy with the FOLIO project, so I doubt he has time to work on Zebra these days, even if we did have a patch available. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'>Is ElasticSearch usable with Koha at this point? I heard a lot in 2015, but after Robin left I haven’t heard a word other than rumours that the patches had been pushed?<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>David Cook<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Systems Librarian<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Prosentient Systems<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>72/330 Wattle St<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Ultimo, NSW 2007<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Australia<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Office: 02 9212 0899<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Direct: 02 8005 0595<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:EN-US'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri",sans-serif'>From:</span></b><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri",sans-serif'> koha-devel-bounces@lists.koha-community.org [mailto:koha-devel-bounces@lists.koha-community.org] <b>On Behalf Of </b>Tomas Cohen Arazi<br><b>Sent:</b> Tuesday, 23 August 2016 11:42 PM<br><b>To:</b> Barton Chittenden <barton@bywatersolutions.com>; Jonathan Druart <jonathan.druart@bugs.koha-community.org><br><b>Cc:</b> koha-devel@lists.koha-community.org<br><b>Subject:</b> Re: [Koha-devel] 16.05, zebra and jessie<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>I have seen use_zebra_facets=1 cause no facets rendered when GRS-1 configuration files are kept during upgrades up to where GRS-1 got deprecated (3.20?). Is it the case? What does the About > System information page says about your config?<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>The slowliness is not in zebra per se, but in the way we retrieve the facets from it (so Koha/Perl side). We retrieve each facet at a time instead of fetching them all in one call. And they come in XML format, so need to be parsed. So, if anyone is willing to improve it, just need to optimize this function (read the TODO):<o:p></o:p></p></div><div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>sub _get_facet_from_result_set {<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> my $facet_idx = shift;<o:p></o:p></p></div><div><p class=MsoNormal> my $rs = shift;<o:p></o:p></p></div><div><p class=MsoNormal> my $sep = shift;<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> my $internal_sep = '<*>';<o:p></o:p></p></div><div><p class=MsoNormal> my $facetMaxCount = C4::Context->preference('FacetMaxCount') // 20;<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> return if ( ! defined $facet_idx || ! defined $rs );<o:p></o:p></p></div><div><p class=MsoNormal> # zebra's facet element, untokenized index<o:p></o:p></p></div><div><p class=MsoNormal> my $facet_element = 'zebra::facet::' . $facet_idx . ':0:' . $facetMaxCount;<o:p></o:p></p></div><div><p class=MsoNormal> # configure zebra results for retrieving the desired facet<o:p></o:p></p></div><div><p class=MsoNormal> $rs->option( elementSetName => $facet_element );<o:p></o:p></p></div><div><p class=MsoNormal> # get the facet record from result set<o:p></o:p></p></div><div><p class=MsoNormal> my $facet = $rs->record( 0 )->raw;<o:p></o:p></p></div><div><p class=MsoNormal> # if the facet has no restuls...<o:p></o:p></p></div><div><p class=MsoNormal> return if !defined $facet;<o:p></o:p></p></div><div><p class=MsoNormal> # TODO: benchmark DOM vs. SAX performance<o:p></o:p></p></div><div><p class=MsoNormal> my $facet_dom = XML::LibXML->load_xml(<o:p></o:p></p></div><div><p class=MsoNormal> string => ($facet)<o:p></o:p></p></div><div><p class=MsoNormal> );<o:p></o:p></p></div><div><p class=MsoNormal> my @terms = $facet_dom->getElementsByTagName('term');<o:p></o:p></p></div><div><p class=MsoNormal> return if ! @terms;<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> my $facets = {};<o:p></o:p></p></div><div><p class=MsoNormal> foreach my $term ( @terms ) {<o:p></o:p></p></div><div><p class=MsoNormal> my $facet_value = $term->textContent;<o:p></o:p></p></div><div><p class=MsoNormal> $facet_value =~ s/\Q$internal_sep\E/$sep/ if defined $sep;<o:p></o:p></p></div><div><p class=MsoNormal> $facets->{ $facet_value } = $term->getAttribute( 'occur' );<o:p></o:p></p></div><div><p class=MsoNormal> }<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> return $facets;<o:p></o:p></p></div><div><p class=MsoNormal>}<o:p></o:p></p></div></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Another option would be to make _get_facets_from_zebra build the element set containing all facets so they are read in one call (comma-separate all elements). The problem is that Zebra returns zero if one of the elements is empty. Jared proposed to create a ghost record with all facet fields. I didn't manage to make it work. Another option is to patch Zebra. I started that, but abandoned once the ES code got pushed.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>So, if use_zebra_facets=0 is good enough, maybe it should be recommended. Problem is it is not a real facet, but the sole extraction of the fields from the first x records.<o:p></o:p></p></div><div><p class=MsoNormal>As I said, it could be good enough anyway.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Regards<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>El mar., 23 ago. 2016 a las 10:21, Barton Chittenden (<<a href="mailto:barton@bywatersolutions.com">barton@bywatersolutions.com</a>>) escribió:<o:p></o:p></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><div><p class=MsoNormal>Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to disk (~16G/query on large libraries). Bug 13665 mentions that searches could be taking upwards of 40 seconds to complete -- I think that we've seen searches time out and return no results at about 1 minute.<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Is it possible to tune Zebra's space/time optimizations in any way so that it doesn't write such large files to disk?<o:p></o:p></p></div></div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart <<a href="mailto:jonathan.druart@bugs.koha-community.org" target="_blank">jonathan.druart@bugs.koha-community.org</a>> wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm'><p class=MsoNormal>See bug 13665 - Retrieve facets from zebra is slow<br>To understand why and when use_zebra_facet=1 is slow<o:p></o:p></p><div><div><p class=MsoNormal><br>2016-08-22 21:31 GMT+01:00 Barton Chittenden <<a href="mailto:barton@bywatersolutions.com" target="_blank">barton@bywatersolutions.com</a>>:<br>> I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I<br>> have run into this, when using ICU-Chains:<br>><br>> Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1<br>> URL : <a href="https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581" target="_blank">https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581</a><br>> Priority : P5 - low<br>> Urgency : enhancement<br>> Status : NEW<br>><br>> I also know that when use_zebra_facets was first introduced, it was *very*<br>> slow -- I can't find any bugs about that though. It's possible that it got<br>> so slow under idzebra-2.0 2.61 that the searches are timing out.<br>><br>> It should be possible to set the logging for zebra so that you can see the<br>> PQF queries:<br>><br>> See<br>><br>> Bug 15714 : Remove zebra.log from debian scripts and add optional log levels<br>> URL : <a href="https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714" target="_blank">https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714</a><br>> Priority : P5 - low<br>> Urgency : enhancement<br>> Status : RESOLVED<br>><br>> For setting the log levels<br>><br>> And <a href="http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html" target="_blank">http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html</a><br>><br>> For a general discussion of how to use them.<br>><br>> ... This should give you some idea of what's failing, both in terms of the<br>> dashes in 2.0.59 and the non-functional zebra facets under 2.0.61.<br>><br>> My general feeling is that 2.0.59 is irredeemably broken by bug 16581, and<br>> we need at least 2.0.60, but I don't have any experience with zebra facets.<br>><br>> --Barton<br>><br>><br>><br>> On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett <<a href="mailto:mtompset@hotmail.com" target="_blank">mtompset@hotmail.com</a>> wrote:<br>>><br>>> Greetings,<br>>><br>>> Similar problem. I hope someone has a better solution than setting it to<br>>> 0.<br>>><br>>> GPML,<br>>> Mark Tompsett<br>>><br>>> -----Original Message-----<br>>> From: Philippe Blouin<br>>> Sent: Monday, August 22, 2016 2:40 PM<br>>> To: <a href="mailto:koha-devel@lists.koha-community.org" target="_blank">koha-devel@lists.koha-community.org</a><br>>> Subject: [Koha-devel] 16.05, zebra and jessie<br>>><br>>> Hello!<br>>><br>>> We're trying to find the correction combination. We're new on Jessie,<br>>> so we still have some tweaking needed...<br>>><br>>> - By default, we get zebra 2.00.59 installed on Jessie through the<br>>> packages.<br>>> - On 16.05, we get some very bad results in the search when the itemtype<br>>> contains an hyphen (-), like 'A-DOC'.<br>>> - So we installed zebra 2.00.62. This fixes the search...<br>>> - But now we do not have facets.<br>>> - So we set <use_zebra_facets>0</use_zebra_facets><br>>> - And now we have facets. But this feels... wrong?<br>>><br>>> My dummy question: what is the supposedly correct version of Zebra on<br>>> Jessie ?<br>>> And we're we correct in setting the config to 0 ?<br>>><br>>> Thanks<br>>> Blou<br>>> _______________________________________________<br>>> Koha-devel mailing list<br>>> <a href="mailto:Koha-devel@lists.koha-community.org" target="_blank">Koha-devel@lists.koha-community.org</a><br>>> <a href="http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel" target="_blank">http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel</a><br>>> website : <a href="http://www.koha-community.org/" target="_blank">http://www.koha-community.org/</a><br>>> git : <a href="http://git.koha-community.org/" target="_blank">http://git.koha-community.org/</a><br>>> bugs : <a href="http://bugs.koha-community.org/" target="_blank">http://bugs.koha-community.org/</a><br>>><br>>> _______________________________________________<br>>> Koha-devel mailing list<br>>> <a href="mailto:Koha-devel@lists.koha-community.org" target="_blank">Koha-devel@lists.koha-community.org</a><br>>> <a href="http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel" target="_blank">http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel</a><br>>> website : <a href="http://www.koha-community.org/" target="_blank">http://www.koha-community.org/</a><br>>> git : <a href="http://git.koha-community.org/" target="_blank">http://git.koha-community.org/</a><br>>> bugs : <a href="http://bugs.koha-community.org/" target="_blank">http://bugs.koha-community.org/</a><br>><br>><br>><br>> _______________________________________________<br>> Koha-devel mailing list<br>> <a href="mailto:Koha-devel@lists.koha-community.org" target="_blank">Koha-devel@lists.koha-community.org</a><br>> <a href="http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel" target="_blank">http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel</a><br>> website : <a href="http://www.koha-community.org/" target="_blank">http://www.koha-community.org/</a><br>> git : <a href="http://git.koha-community.org/" target="_blank">http://git.koha-community.org/</a><br>> bugs : <a href="http://bugs.koha-community.org/" target="_blank">http://bugs.koha-community.org/</a><o:p></o:p></p></div></div></blockquote></div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal>_______________________________________________<br>Koha-devel mailing list<br><a href="mailto:Koha-devel@lists.koha-community.org" target="_blank">Koha-devel@lists.koha-community.org</a><br><a href="http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel" target="_blank">http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel</a><br>website : <a href="http://www.koha-community.org/" target="_blank">http://www.koha-community.org/</a><br>git : <a href="http://git.koha-community.org/" target="_blank">http://git.koha-community.org/</a><br>bugs : <a href="http://bugs.koha-community.org/" target="_blank">http://bugs.koha-community.org/</a><o:p></o:p></p></blockquote></div><div><p class=MsoNormal>-- <o:p></o:p></p></div><div><div><div><p class=MsoNormal><span style='font-size:9.5pt;font-family:"Helvetica",sans-serif;color:#757575'>Tomás Cohen Arazi<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:9.5pt;font-family:"Helvetica",sans-serif;color:#757575'>Theke Solutions (<a href="http://theke.io/">https://theke.io</a>)<br></span><span style='font-size:9.5pt;font-family:"Segoe UI Symbol",sans-serif;color:#757575'>✆</span><span style='font-size:9.5pt;font-family:"Helvetica",sans-serif;color:#757575'> +54 9351 3513384<br>GPG: B2F3C15F<o:p></o:p></span></p></div></div></div></div></div></body></html>