- Koha-devel - lists.koha-community.org

Fix on DOM inedxing. We still work on biblio-koha-indexdefs.xml or directly on biblio-zebra-indexdefs.xsl
by Tajoli Zeno 13 Feb '15

13 Feb '15

Hi to all, I find a bug on MARC21/Normarc DOM indexing. Bug 12948: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=12948 Now that grs1 is deprecated, do we still use biblio-koha-indexdefs.xml and generate biblio-zebra-indexdefs.xsl ? Or now do we prefer edit biblio-zebra-indexdefs.xsl directly Bye Zeno Tajoli -- Dr. Zeno Tajoli Servizi Innovativi -- Automazione Biblioteche z.tajoli(a)cineca.it fax +39 02 2135520 CINECA - Sede operativa di Segrate

2 1

Re: [Koha-devel] [Koha] Koha development
by Paul A 12 Feb '15

12 Feb '15

Marshall, Another "numbers source" that might be helpful to you is <http://git.koha-community.org/stats/koha-master/index.html> Best regards -- Paul At 11:48 AM 2/12/2015 -0300, Tomas Cohen Arazi wrote: >Marshall, I don't think we can provide you with that information in that >time frame. We could try, but not sure. > >On the other hand, have you seen the project's profile on OpenHub? [1]. It >provides lots of metrics about the projects activity, even developer-wise. >You could take a look and see if it helps. One thing that might be missing >is the mapping <individual> -> <company/institution>. We could help with >that. > >[1] https://www.openhub.net/p/koha > >On Thu, Feb 12, 2015 at 11:07 AM, Marshall Breeding < >marshall.breeding(a)librarytechnology.org> wrote: > > > To the Koha development community, > > > > > > > > I am in the process of writing a news story about the recent grant from > > EBSCO Information Services to support some major development initiatives > > for Koha. This seems to be an important opportunity for the Koha community > > to accelerate development that might have otherwise taken longer to > > accomplish. > > > > > > > > As part of the context for this article, and for other reports and > > presentations that I make that include Koha, I would like to be able to > > provide a quantitative measure of the total amount of personnel resources > > contributed globally towards the development of Koha. I understand that > > several firms dedicate specific amounts of personnel to Koha development > > and that there are dozens of individuals that make commits to the code, > > write documentation, or perform quality assurance. > > > > > > > > I would appreciate it members of this list would help me compile a current > > figure on the number of FTE involved in Koha development. > > > > To end up with a consistent quantifiable measure, the following > > definitions would apply: > > > > 1 FTE (full time equivalent) = 1 person working full time for 1 year. > > > > Full time = 40 hours per week 50 weeks/year. > > > > > > > > An individual that contributes 20 hours per week on Koha for the whole > > year= > > > > (20 hours per week * 50 weeks per year) / 2000 hours per year = .5 FTE > > > > > > > > An individual that contributes 20 hours total for the year: > > > > (20 / 2000) * 100 = 0.01 FTE > > > > > > > > Could each of the support firms or individuals involved in Koha > > development please either post to this list or send to me individually the > > FTE that you or your organization contributes for each of the major > > categories of activity? > > > > > > > > For example: > > > > Company XYZ: 5 FTE development, 1 FTE Documentation, 1.5 FTE Quality > > Assurance. > > > > > > > > Alternatively, the number of hours per year contributed could be given > > instead of FTE and Iâll perform the calculations. > > > > > > > > If there is a better way to quantify the resources devoted to Koha, I > > would be interested in your suggestions. > > > > > > > > I believe that documenting the development resources involved in Koha will > > help those outside the community have a better understanding of Kohaâs > > potential to continue to be enhanced to meet the needs of libraries. > > > > > > > > I need to get this article to the publisher (ALA TechSource) in the next > > day or two, it would be great to hear from you soon. > > > > > > > > Thanks very much for your help with this request. > > > > > > > > -marshall > > > > > > > > > > > > Marshall Breeding > > > > marshall.breeding(a)librarytechnology.org > > > > www.librarytechnology.org/ > > > > twitter.com/mbreeding > > > > http://www.linkedin.com/in/breeding > > > > http://scholar.google.com/citations?user=NnvfJ5cAAAAJ > > > > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel(a)lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ > > > > > >-- >TomÃ¡s Cohen Arazi >ProsecretarÃa de InformÃ¡tica >Universidad Nacional de CÃ³rdoba >â +54 351 5353750 ext 13168 >GPG: B76C 6E7C 2D80 551A C765 E225 0A27 2EA1 B2F3 C15F >_______________________________________________ >Koha mailing list http://koha-community.org >Koha(a)lists.katipo.co.nz >http://lists.katipo.co.nz/mailman/listinfo/koha --- Maritime heritage and history, preservation and conservation, research and education through the written word and the arts. <http://NavalMarineArchive.com> and <http://UltraMarine.ca>

1 0

repeated entries in biblioimages (record covers)
by Giuseppe Angilella 10 Feb '15

10 Feb '15

Hi, as far as I understand, the image files of additional record covers are saved in binary format under the mysql fields biblioimages.imagefile and biblioimages.thumbnail, which are of type mediumblob. I would now like to associate several (hundreds) records to the same cover image. (These are books belonging to an old collection, for which no ISBN exists and therefore no image is available on amazon, all sharing the same sort of cover jacket.) Should I enter the same cover image for each record separately, this would result in an unnecessary increase in the size of the biblioimages table, and therefore of the whole database. I wonder whether some sort of "aliasing" would be possible, in this case. A related question is also whether adding a cover image to a record could be performed routinely through an external script, rather than through the Koha staff browser, to expedite the procedure in the case of several records. Many thanks in advance and best regards. Giuseppe.

3 2

3.18 performance issues - benchmarks
by Jonathan Druart 10 Feb '15

10 Feb '15

Hi devs, I have opened a bug report (bug 13690) to show you the results of benchmarks I did between 3.16 and 3.18. I also submitted 3 patches to try to fix the performance problem in 3.18. Hope to get some feedbacks and see you involved in this real issue. Cheers, Jonathan http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13690

1 0

More info about Zebra hyphen truncating search (and announcing Zebra 2.0.60)
by David Cook 09 Feb '15

09 Feb '15

Hi all: Just letting everyone know that there is definitely a bug in Zebra 2.0.59 (and a certain number of unknown versions after 2.0.44 which is the version in Debian and Ubuntu prior to Jessie/Vivid which both use 2.0.59 as of this email), which causes problems with tokenization when using ICU and having a hyphen in a search term. Basically, the hyphen triggers tokenization, but only the first token is used. So a search for "Mont-Royal" will actually just be a search for "Mont". Or a search for "up-to-date" will just be a search for "up". This is the case even when trying to use ICU transformation/transliteration rules to remove the hyphen before tokenizing. However, I reported the issue to IndexData on February 4th, and they wrote a fix on February 7th (http://git.indexdata.com/?p=idzebra.git;a=commitdiff;h=704fd190292cb771df94 553b0ed6f9f4b71660a6). They've released that fix as part of Zebra 2.0.60, which is now available via the IndexData repositories. I just tested it on a Debian Squeeze install (using the IndexData apt repository), and it works great. I'm going to ask IndexData if they can provide more information, as Robin has volunteered to report a bug to Debian. It might be too late to get Zebra 2.0.60 shipped with Jessie, but maybe they can backport the patch or at least be aware of the issue... Cheers, David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007

1 0

Koha Community Newsletter: January 2015
by kohanews 06 Feb '15

06 Feb '15

Fellow Koha users: The Koha Community Newsletter for January 2015 is here: http://koha-community.org/koha-community-newsletter-january-2015/ Many thanks to the folks who submitted articles and news to this month's newsletter. Please feel free to email us with any corrections or suggestions. Thanks! Chad Roseburg Joanne Dillon Editors, Koha Community Newsletter

1 0

Re: [Koha-devel] Search speed
by Paul A 06 Feb '15

06 Feb '15

At 07:16 PM 2/4/2015 -0300, Tomas Cohen Arazi wrote: >For a fair comparison you should set facetNumRecs = 20 in >zebradb/zebra-biblios-dom.xml. Because otherwise use_zebra_facets=0 is >only processing the first 20 records, against 1000 from use_zebra_facets=1. Thanks for the suggestion. All tests stop/start memcached and zebrasrv. NYTProf gives: *** Both set to 20: Profile of ./opac-search.pl for 21.1s (of 21.3s), executing 1018674 statements and 252207 subroutine calls in 590 source files and 132 string evals. with 19.3s spent in Net::Z3950::ZOOM::resultset_record (xsub) and the CPU (from 'top') is stuck at 100% of a single core for all that time *** Set 20 (sysprefs) 1000 zebra-biblios-dom.xml: Profile of ./opac-search.pl for 20.9s (of 21.1s), executing 1018683 statements and 252793 subroutine calls in 590 source files and 133 string evals. with 19.1s spent in Net::Z3950::ZOOM::resultset_record (xsub) and the CPU at 100% *** Set 20 (sysprefs) 30000 zebra-biblios-dom.xml: Profile of ./opac-search.pl for 20.9s (of 21.2s), executing 1018669 statements and 252616 subroutine calls in 590 source files and 134 string evals. with 19.2s spent in Net::Z3950::ZOOM::resultset_record (xsub) and the CPU at 100% But (catastrophe), if I set both for 100 facets (maybe not unreasonable): Profile of ./opac-search.pl for 67.9s (of 68.2s), executing 1023091 statements and 255995 subroutine calls in 590 source files and 132 string evals, 66.2 secs in resultset_record with CPU 100% for that time. Conclusion: It's the syspref 'FacetMaxCount' that is the controlling value. >Regarding the amount of statements/function calls, I don't think they mean >too much. Some things got pulled from a +150 line piece of code into >several helper functions. I don't think it introduces much overhead. Maybe not, but several hundred thousand operations, even at a low median of 240 microsecs, adds up to 3.08 responding in 0.6 - 0.8 secs, and 3.18 (use_zebra_facets false) needing 1.3 to 1.6 And there are other aspects of "speed" that I'm looking at -- e.g. apache 2.4 with mod_deflate needs some fine tuning; otherwise images, css, js (maybe more) don't necessarily get compressed and add considerably to the number of packets, syns, acks, etc and web latency can kick in. But that's for another day... ;=) Again thanks for the suggestion, best -- Paul >REgards >Tomas > >PS. if you play with facetNumRecs, remember to restart zebra each time you >change it. > >On Wed, Feb 4, 2015 at 6:22 PM, Paul A <paul.a(a)navalmarinearchive.com> wrote: >At 02:02 PM 2/2/2015 +0000, Tomas Cohen Arazi wrote: >Paul, while you are running your tests, could you please compare this both >scenarios: >- <use_zebra_facets>1</use_zebra_facets> >- <use_zebra_facets>0</use_zebra_facets> >Thanks >PS: I have found that searchResults is too slow too. > > >I have posted notes, results and some NYTProf examples at ><http://navalmarinearchive.com/z_koha/> -- the results are repeatable (I >must have run NYTProf nearly 200 times.) > >Provisional conclusions are that <use_zebra_facets> = 1 (true) is not >usable in production for a search that finds > 20k results. For smaller >results, it might be acceptable, but is still appreciably slower than >3.08. Under the best circumstances (0-5 results) 3.08 responds in < 0.7 >secs, 3.18 in 1.3 secs.Â At the other extreme (25k results), 3.08 >requires 1.07 secs, 3.18 21+ secs. > >I have defined an "optimal" 3.18 combination (see green line, Graph 2) >which runs at about half the speed of 3.08, but does not give 'Series', >'Places' and 'Titles' in the facets. > >One point that jumped out is that a best case 3.08 search executes 431207 >statements and 76556 subroutine calls in 297 source files and 81 string >evals whereas a worst case 3.18 executes 1238167 statements and 309905 >subroutine calls in 591 source files and 132 string evals. (Even the worst >case 308 appears much more efficient than the best case 3.18.) > >The use of -T to start Zebra is non-conclusive -- it vaguely slows down >smaller result sets and vaguely speeds up larger result sets (testing is >repeatable.)Â Robin was of course quite correct in his recent email, but >I cannot explain even the minor variations experienced. > >I'm not at all certain how to file this as a bug, so will await any comments. > >Best -- Paul

1 0

A report for finding records with multiple items?
by andrea.furin＠slacky.it 06 Feb '15

06 Feb '15

Hi. I'd need to write a report for finding records with multiple items in them: any suggestions? TIA Andrea Furin

2 1

Dev meeting part 2 missed
by Tomas Cohen Arazi 06 Feb '15

06 Feb '15

Hi, I got home late and it seems there hasn't been part 2 of the dev meeting. People that didn't attend part 1, please propose a date/time. Important votes regarding stuff removal are being held. -- Tomás Cohen Arazi Prosecretaría de Informática Universidad Nacional de Córdoba ✆ +54 351 5353750 ext 13168 GPG: B76C 6E7C 2D80 551A C765 E225 0A27 2EA1 B2F3 C15F

2 1

Marc in spreadsheet
by andrea.furin＠slacky.it 05 Feb '15

05 Feb '15

Hi. I need a way to transform my marc file from koha in a spreadsheet file (excel or libreoffice): there is a way? Thanks in advance Andrea Furin

1 0