February 2015 - Koha-devel - lists.koha-community.org

repeated entries in biblioimages (record covers)
by Giuseppe Angilella 10 Feb '15

10 Feb '15

Hi, as far as I understand, the image files of additional record covers are saved in binary format under the mysql fields biblioimages.imagefile and biblioimages.thumbnail, which are of type mediumblob. I would now like to associate several (hundreds) records to the same cover image. (These are books belonging to an old collection, for which no ISBN exists and therefore no image is available on amazon, all sharing the same sort of cover jacket.) Should I enter the same cover image for each record separately, this would result in an unnecessary increase in the size of the biblioimages table, and therefore of the whole database. I wonder whether some sort of "aliasing" would be possible, in this case. A related question is also whether adding a cover image to a record could be performed routinely through an external script, rather than through the Koha staff browser, to expedite the procedure in the case of several records. Many thanks in advance and best regards. Giuseppe.

3 2

3.18 performance issues - benchmarks
by Jonathan Druart 10 Feb '15

10 Feb '15

Hi devs, I have opened a bug report (bug 13690) to show you the results of benchmarks I did between 3.16 and 3.18. I also submitted 3 patches to try to fix the performance problem in 3.18. Hope to get some feedbacks and see you involved in this real issue. Cheers, Jonathan http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13690

1 0

More info about Zebra hyphen truncating search (and announcing Zebra 2.0.60)
by David Cook 09 Feb '15

09 Feb '15

Hi all: Just letting everyone know that there is definitely a bug in Zebra 2.0.59 (and a certain number of unknown versions after 2.0.44 which is the version in Debian and Ubuntu prior to Jessie/Vivid which both use 2.0.59 as of this email), which causes problems with tokenization when using ICU and having a hyphen in a search term. Basically, the hyphen triggers tokenization, but only the first token is used. So a search for "Mont-Royal" will actually just be a search for "Mont". Or a search for "up-to-date" will just be a search for "up". This is the case even when trying to use ICU transformation/transliteration rules to remove the hyphen before tokenizing. However, I reported the issue to IndexData on February 4th, and they wrote a fix on February 7th (http://git.indexdata.com/?p=idzebra.git;a=commitdiff;h=704fd190292cb771df94 553b0ed6f9f4b71660a6). They've released that fix as part of Zebra 2.0.60, which is now available via the IndexData repositories. I just tested it on a Debian Squeeze install (using the IndexData apt repository), and it works great. I'm going to ask IndexData if they can provide more information, as Robin has volunteered to report a bug to Debian. It might be too late to get Zebra 2.0.60 shipped with Jessie, but maybe they can backport the patch or at least be aware of the issue... Cheers, David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007

1 0

Koha Community Newsletter: January 2015
by kohanews 06 Feb '15

06 Feb '15

Fellow Koha users: The Koha Community Newsletter for January 2015 is here: http://koha-community.org/koha-community-newsletter-january-2015/ Many thanks to the folks who submitted articles and news to this month's newsletter. Please feel free to email us with any corrections or suggestions. Thanks! Chad Roseburg Joanne Dillon Editors, Koha Community Newsletter

1 0

Re: [Koha-devel] Search speed
by Paul A 06 Feb '15

06 Feb '15

At 07:16 PM 2/4/2015 -0300, Tomas Cohen Arazi wrote: >For a fair comparison you should set facetNumRecs = 20 in >zebradb/zebra-biblios-dom.xml. Because otherwise use_zebra_facets=0 is >only processing the first 20 records, against 1000 from use_zebra_facets=1. Thanks for the suggestion. All tests stop/start memcached and zebrasrv. NYTProf gives: *** Both set to 20: Profile of ./opac-search.pl for 21.1s (of 21.3s), executing 1018674 statements and 252207 subroutine calls in 590 source files and 132 string evals. with 19.3s spent in Net::Z3950::ZOOM::resultset_record (xsub) and the CPU (from 'top') is stuck at 100% of a single core for all that time *** Set 20 (sysprefs) 1000 zebra-biblios-dom.xml: Profile of ./opac-search.pl for 20.9s (of 21.1s), executing 1018683 statements and 252793 subroutine calls in 590 source files and 133 string evals. with 19.1s spent in Net::Z3950::ZOOM::resultset_record (xsub) and the CPU at 100% *** Set 20 (sysprefs) 30000 zebra-biblios-dom.xml: Profile of ./opac-search.pl for 20.9s (of 21.2s), executing 1018669 statements and 252616 subroutine calls in 590 source files and 134 string evals. with 19.2s spent in Net::Z3950::ZOOM::resultset_record (xsub) and the CPU at 100% But (catastrophe), if I set both for 100 facets (maybe not unreasonable): Profile of ./opac-search.pl for 67.9s (of 68.2s), executing 1023091 statements and 255995 subroutine calls in 590 source files and 132 string evals, 66.2 secs in resultset_record with CPU 100% for that time. Conclusion: It's the syspref 'FacetMaxCount' that is the controlling value. >Regarding the amount of statements/function calls, I don't think they mean >too much. Some things got pulled from a +150 line piece of code into >several helper functions. I don't think it introduces much overhead. Maybe not, but several hundred thousand operations, even at a low median of 240 microsecs, adds up to 3.08 responding in 0.6 - 0.8 secs, and 3.18 (use_zebra_facets false) needing 1.3 to 1.6 And there are other aspects of "speed" that I'm looking at -- e.g. apache 2.4 with mod_deflate needs some fine tuning; otherwise images, css, js (maybe more) don't necessarily get compressed and add considerably to the number of packets, syns, acks, etc and web latency can kick in. But that's for another day... ;=) Again thanks for the suggestion, best -- Paul >REgards >Tomas > >PS. if you play with facetNumRecs, remember to restart zebra each time you >change it. > >On Wed, Feb 4, 2015 at 6:22 PM, Paul A <paul.a(a)navalmarinearchive.com> wrote: >At 02:02 PM 2/2/2015 +0000, Tomas Cohen Arazi wrote: >Paul, while you are running your tests, could you please compare this both >scenarios: >- <use_zebra_facets>1</use_zebra_facets> >- <use_zebra_facets>0</use_zebra_facets> >Thanks >PS: I have found that searchResults is too slow too. > > >I have posted notes, results and some NYTProf examples at ><http://navalmarinearchive.com/z_koha/> -- the results are repeatable (I >must have run NYTProf nearly 200 times.) > >Provisional conclusions are that <use_zebra_facets> = 1 (true) is not >usable in production for a search that finds > 20k results. For smaller >results, it might be acceptable, but is still appreciably slower than >3.08. Under the best circumstances (0-5 results) 3.08 responds in < 0.7 >secs, 3.18 in 1.3 secs.Â At the other extreme (25k results), 3.08 >requires 1.07 secs, 3.18 21+ secs. > >I have defined an "optimal" 3.18 combination (see green line, Graph 2) >which runs at about half the speed of 3.08, but does not give 'Series', >'Places' and 'Titles' in the facets. > >One point that jumped out is that a best case 3.08 search executes 431207 >statements and 76556 subroutine calls in 297 source files and 81 string >evals whereas a worst case 3.18 executes 1238167 statements and 309905 >subroutine calls in 591 source files and 132 string evals. (Even the worst >case 308 appears much more efficient than the best case 3.18.) > >The use of -T to start Zebra is non-conclusive -- it vaguely slows down >smaller result sets and vaguely speeds up larger result sets (testing is >repeatable.)Â Robin was of course quite correct in his recent email, but >I cannot explain even the minor variations experienced. > >I'm not at all certain how to file this as a bug, so will await any comments. > >Best -- Paul

1 0

A report for finding records with multiple items?
by andrea.furin＠slacky.it 06 Feb '15

06 Feb '15

Hi. I'd need to write a report for finding records with multiple items in them: any suggestions? TIA Andrea Furin

2 1

Dev meeting part 2 missed
by Tomas Cohen Arazi 06 Feb '15

06 Feb '15

Hi, I got home late and it seems there hasn't been part 2 of the dev meeting. People that didn't attend part 1, please propose a date/time. Important votes regarding stuff removal are being held. -- Tomás Cohen Arazi Prosecretaría de Informática Universidad Nacional de Córdoba ✆ +54 351 5353750 ext 13168 GPG: B76C 6E7C 2D80 551A C765 E225 0A27 2EA1 B2F3 C15F

2 1

Marc in spreadsheet
by andrea.furin＠slacky.it 05 Feb '15

05 Feb '15

Hi. I need a way to transform my marc file from koha in a spreadsheet file (excel or libreoffice): there is a way? Thanks in advance Andrea Furin

1 0

Re: [Koha-devel] Search speed
by Paul A 05 Feb '15

05 Feb '15

At 10:03 AM 2/5/2015 +0100, Jonathan Druart wrote: >I have filled bug 13665 yesterday >http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13665 Thanks. I had come to that same conclusion concerning resultset_record (see my email 9 Jan to this list.) For anyone wanting to start chasing this down, I noted: ZOOM modules: We have Net::Z3950::ZOOM (v1.26) Ubuntu libnet-z3950-zoom-perl (latest.) Includes libidzebra-2.0-mod-alvis, but there are two different files named ZOOM.pm me@hood:/$ cat -n /usr/lib/perl5/ZOOM.pm 1110 lines me@hood:/$ cat -n /usr/lib/perl5/Net/Z3950/ZOOM.pm 194 lines Debug: "book mark" 'f ZOOM' both choose the short one: 'b 152' therefore force 'f /usr/lib/perl5/ZOOM.pm' 'b 739' 'f /usr/share/koha/lib/C4/Auth.pm' 'b 442' return to 'f opac-search.pl' But from then on, chasing individual lines/functions (s, n, r) became frustrating, and a dump (T) was unmanageable. Again, thanks -- Paul >2015-02-04 22:22 GMT+01:00 Paul A <paul.a(a)navalmarinearchive.com>: > > At 02:02 PM 2/2/2015 +0000, Tomas Cohen Arazi wrote: > >> > >> Paul, while you are running your tests, could you please compare this both > >> scenarios: > >> - <use_zebra_facets>1</use_zebra_facets> > >> - <use_zebra_facets>0</use_zebra_facets> > >> Thanks > >> PS: I have found that searchResults is too slow too. > > > > > > I have posted notes, results and some NYTProf examples at > > <http://navalmarinearchive.com/z_koha/> -- the results are repeatable (I > > must have run NYTProf nearly 200 times.) > > > > Provisional conclusions are that <use_zebra_facets> = 1 (true) is not > usable > > in production for a search that finds > 20k results. For smaller > results, it > > might be acceptable, but is still appreciably slower than 3.08. Under the > > best circumstances (0-5 results) 3.08 responds in < 0.7 secs, 3.18 in 1.3 > > secs. At the other extreme (25k results), 3.08 requires 1.07 secs, > 3.18 21+ > > secs. > > > > I have defined an "optimal" 3.18 combination (see green line, Graph 2) > which > > runs at about half the speed of 3.08, but does not give 'Series', 'Places' > > and 'Titles' in the facets. > > > > One point that jumped out is that a best case 3.08 search executes 431207 > > statements and 76556 subroutine calls in 297 source files and 81 string > > evals whereas a worst case 3.18 executes 1238167 statements and 309905 > > subroutine calls in 591 source files and 132 string evals. (Even the worst > > case 308 appears much more efficient than the best case 3.18.) > > > > The use of -T to start Zebra is non-conclusive -- it vaguely slows down > > smaller result sets and vaguely speeds up larger result sets (testing is > > repeatable.) Robin was of course quite correct in his recent email, but I > > cannot explain even the minor variations experienced. > > > > I'm not at all certain how to file this as a bug, so will await any > > comments. > > > > Best -- Paul > > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel(a)lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ --- Maritime heritage and history, preservation and conservation, research and education through the written word and the arts. <http://NavalMarineArchive.com> and <http://UltraMarine.ca>

1 0

Re: [Koha-devel] Search speed
by Paul A 05 Feb '15

05 Feb '15

At 02:02 PM 2/2/2015 +0000, Tomas Cohen Arazi wrote: >Paul, while you are running your tests, could you please compare this both >scenarios: >- <use_zebra_facets>1</use_zebra_facets> >- <use_zebra_facets>0</use_zebra_facets> >Thanks >PS: I have found that searchResults is too slow too. I have posted notes, results and some NYTProf examples at <http://navalmarinearchive.com/z_koha/> -- the results are repeatable (I must have run NYTProf nearly 200 times.) Provisional conclusions are that <use_zebra_facets> = 1 (true) is not usable in production for a search that finds > 20k results. For smaller results, it might be acceptable, but is still appreciably slower than 3.08. Under the best circumstances (0-5 results) 3.08 responds in < 0.7 secs, 3.18 in 1.3 secs. At the other extreme (25k results), 3.08 requires 1.07 secs, 3.18 21+ secs. I have defined an "optimal" 3.18 combination (see green line, Graph 2) which runs at about half the speed of 3.08, but does not give 'Series', 'Places' and 'Titles' in the facets. One point that jumped out is that a best case 3.08 search executes 431207 statements and 76556 subroutine calls in 297 source files and 81 string evals whereas a worst case 3.18 executes 1238167 statements and 309905 subroutine calls in 591 source files and 132 string evals. (Even the worst case 308 appears much more efficient than the best case 3.18.) The use of -T to start Zebra is non-conclusive -- it vaguely slows down smaller result sets and vaguely speeds up larger result sets (testing is repeatable.) Robin was of course quite correct in his recent email, but I cannot explain even the minor variations experienced. I'm not at all certain how to file this as a bug, so will await any comments. Best -- Paul

3 2