October 2012 - Koha-devel - lists.koha-community.org

Re: [Koha-devel] DOM, id:x, reindexing (bug8665?)
by Archives and Collections Society 18 Oct '12

18 Oct '12

Dear Jared, Many thanks for the reply. Comments|questions interspersed below: At 04:22 PM 10/17/2012 -0400, Jared Camins-Esakov wrote: >Our cataloguers are doing a sterling job ridding our db of duplicate 001 >fields, but every time they do a simple edit of *only* the 001, both OPAC >and staff-admin page find TWO absolutely identical (biblio, barcode, etc) >records. > >Zebra incremental (-z) counts the *edit* as a completely new record. > >I can get rid of it by doing a full zebra reindex (-a and -b). This >"error" is totally reprodicible in our db. Can anyone elso confirm it >please? (edit a 001 tag, save, wait for the CRON -z, and search on title.) > >This is the way Zebra works. I understand that -- however (purely based on personal experience after only two years of use) the "concept" of Koha is that the CRON takes care of updating the search engine after cataloguers' edits (and I don't think that even a reboot does a "complete" re-cataloguing job.) For example, if I edit the date of publication in a biblio, zebra finds the edit, makes the change within sixty seconds (our CRON starts '*/1') and does not create a second biblio. But if I edit (again in a biblio) the 001, we get an entirely identical, but supplemental and erroneous record that "sticks." I've got more than enough computing power to CRON a complete reindex every minute, and if that was all that was necessary to fix everything I can do it very easily -- but should this go into the users' manual? I fear that other users may not have an 8-core I7 with raided ssd's and 16Gb ram. Nor do I think it fixes all the problems. Continued below... >A serious addition to this, is that we seem to have totally lost search >capability on serials. Â Example: we have a series of 40 "Warship >Profiles" which a mysql query (the one I gave under bug 8665) gives me: > >biblionumbers: >17472, 17473, 17474, 17475, 17476, 17477, 17478, 17479, 17480, 17482, >17483, 17484, 17485, 17486, 17487, 17488, 17489, 17493, 17494, 17495, >17496, 17497, 17498, 17499, 17500, 17501, 17502, 17503, 17504, 17505, >17506, 17508, 17509, 17511, 17513, 17514, 17516, 17517, 17518, 17520 > >id (Control Number 001): Â 01032053 > >but NONE of the records can be found -- not by biblionumber, nor by >control-number, nor by barcode, nor by title, author, etc ... Â The >records are there (other mysql queries give me the barcodes, values, >shelving locations, titles, authors, etc) but as far as staff and our >patrons are concerned, they plain don't exist. Only for a mysql query. > > >Isn't this the problem you were trying to solve to begin with? You can >access those records directly, as an aside: just go to >/cgi-bin/koha/catalogue/<http://detail.pl?biblionumber=XXXX>detail.pl?biblionumber=XXXX > Yes -- it is one of the "problem[s] you were trying to solve to begin with" ... a "normal" search: http://koha-admin/cgi-bin/koha/catalogue/search.pl?q=biblionumber=17472 comes up with "No results match your search for 'biblionumber=17472' in NMA Catalog." >(easiest is to just view another record, then change the biblionumber in >the URL). Many thanks - from a quick try that works - is there a way in staff-admin to directly use 'detail.pl'? (how I explain that to our lady cataloguers downstairs is going to be challenging, and it remains to be seen if changing biblionumber or 001 will get these records back into the OPAC.) >>[snip] There must have been a good reason, hence our efforts to normalize >>001 to be a unique identifier. >No, this was a bug. There is a patch to restore biblionumber as the z:id. Please (and I'm not trying to be obtuse), how do I install the patch to our production box? That's exactly my next question: >Chris C. has a comment in Bug 8665 "Use a user-specified field for z:id." >Â I am no expert with zebra (and so far haven't worked out the role or the >wheareabouts of id:x), but: Â If z:id is currently set to '001', how do I >set it to biblionumber in 3.8.5? > >Perhaps this will help our non-existent "authorities"? > >There's another bug, also patched, related to this. It has to do with >extra spaces in a configuration file. Again, desperately trying to be diplomatic and not annoy you, how do I get the patch into production. I made a huge mistake (me, personally, and I take all the blame) when I upgraded from 3.6.1 to 3.8.x without enough testing at our end. Now I'm between the devil and the deep blue sea; go back to 3.6.1 and discard two months of cataloguing (maybe try to export these - now over a thousand - biblios from 3.8.5 and re-import them in 3.6.1?) or patch 3.8.5? I remain relatively confident that the mysql is retaining the data and that it's 'only' a zebra problem. So it's patching 3.8.5 that I'd prefer. As I said below, I'll put time and effort into it, and share the results with this community. Best regards, Paul >All suggestions would be truly appreciated -- I'll be more than happy to >try and follow any suggested cures, and will obviously share the outcome here.

1 0

DOM, id:x, reindexing (bug8665?)
by Paul 17 Oct '12

17 Oct '12

Would one of you please be kind enough to save my sanity (if I have any left)? Our cataloguers are doing a sterling job ridding our db of duplicate 001 fields, but every time they do a simple edit of *only* the 001, both OPAC and staff-admin page find TWO absolutely identical (biblio, barcode, etc) records. Zebra incremental (-z) counts the *edit* as a completely new record. I can get rid of it by doing a full zebra reindex (-a and -b). This "error" is totally reprodicible in our db. Can anyone elso confirm it please? (edit a 001 tag, save, wait for the CRON -z, and search on title.) A serious addition to this, is that we seem to have totally lost search capability on serials. Example: we have a series of 40 "Warship Profiles" which a mysql query (the one I gave under bug 8665) gives me: biblionumbers: 17472, 17473, 17474, 17475, 17476, 17477, 17478, 17479, 17480, 17482, 17483, 17484, 17485, 17486, 17487, 17488, 17489, 17493, 17494, 17495, 17496, 17497, 17498, 17499, 17500, 17501, 17502, 17503, 17504, 17505, 17506, 17508, 17509, 17511, 17513, 17514, 17516, 17517, 17518, 17520 id (Control Number 001): 01032053 but NONE of the records can be found -- not by biblionumber, nor by control-number, nor by barcode, nor by title, author, etc ... The records are there (other mysql queries give me the barcodes, values, shelving locations, titles, authors, etc) but as far as staff and our patrons are concerned, they plain don't exist. Only for a mysql query. I have a feeling that all this is "associated with" Bug 8665, but would appreciate a little more understanding (is there a full write up somewhere? I've searched but can't find one.) I believe that 3.6 (at least .1, but changed sometime before .7) used biblionumber as the primary reference; now field 001 has become "primary." Why? There must have been a good reason, hence our efforts to normalize 001 to be a unique identifier. Chris C. has a comment in Bug 8665 "Use a user-specified field for z:id." I am no expert with zebra (and so far haven't worked out the role or the wheareabouts of id:x), but: If z:id is currently set to '001', how do I set it to biblionumber in 3.8.5? Perhaps this will help our non-existent "authorities"? All suggestions would be truly appreciated -- I'll be more than happy to try and follow any suggested cures, and will obviously share the outcome here. Many thanks in advance - Paul

2 1

It's string freeze time again
by Chris Cormack 16 Oct '12

16 Oct '12

Hi all In preparation for the release of 3.8.6 next week, I'd like to call a string freeze now. Frederic as one of your last duties as translation manager can you please update the .po files (or maybe show Ruth how to). I'd like to take this opportunity to thank Frederic for all his hard work as translation manager, he has done a fantastic job. Chris

4 4

Memcached and Koha
by Pongtawat Chippimolchai 16 Oct '12

16 Oct '12

Hello all, I'm trying to speed up Koha 3.8 by enabling memcached, following the instructions in the wiki (http://wiki.koha-community.org/wiki/Performance) But I found that OPAC search performance is almost unchanged with memcached enabled. By looking at memcached stats and MySQL query log, it seems like memcached is only used to store Koha's config (koha-conf.xml). The query to MySQL is exactly the same with or with out memcached. I suppose that once memcached is enabled, all preferences should be read from memcached instead? Is there anything else I should do to get memcached fully working with Koha? FYI, I'm running Koha 3.8.5 on Ubuntu 12.04 using koha-common package. Thank you, Pongtawat

3 6

Next Global Bug Sqashing Day (GBSD) - 2012-09-14
by Fischer, Katrin 16 Oct '12

16 Oct '12

Dear community, as Paul pointed out in his monthly RM newsletter, the freeze dates for the upcoming version 3.10 are getting closer and closer (http://koha-community.org/koha-release-manager-newsletter-10-2012-08/) - and there are still lots of patches, features and bug fixes waiting to be tested, that could be part of this next release. So it’s about time we have another GBSD! > Friday, 14th September 2012 More information about GBSD and useful links can be found on the wiki: http://wiki.koha-community.org/wiki/2012-09-14_Global_bug_squashing_day And there is also our super cool dashboard to have a look at: http://dashboard.koha-community.org/ Best regards, Katrin

2 1

Tag 001 - search
by Paul 14 Oct '12

14 Oct '12

We're continuing to track/correct problems in field 001 (for whatever reasons our db has hundreds of totally erroneous duplicates) and have written a "report" for our cataloguers' use that lists the dups. Now I'm being asked for an easy "paste and click" tool to edit the records. Search for 'control-number:' or 'control-number=' functions well, but when I add an option to the search box, the phrase ',wrdl' gets added as in 'control-number,wrdl:' and the search fails (full details below.) I must be missing something simple, but googling for 'zebra and wrdl' has not come up trumps -- it has been mentioned several times on this and the users' list, but apparently not in this context; maybe (not on Koha) a hint of string/field length, but no suggestions as to how to add this to the search term. Could a kind soul point me in the right direction, please? Is there a flag that can be added? a ccl property to be modded? Thanks and best regards - Paul ccl.properties suggests that the 001 CONTROL NUMBER is either: Local-number 1=12 sn Local-number or Authority-Number 1=Koha-Auth-Number an Authority-Number Koha-Auth-Number Authority-Number A valid record can be found using normal "catalogue search" by entering 'control-number:7725723' and is returned as <http://koha-admin/cgi-bin/koha/catalogue/detail.pl?biblionumber=16553> Search for: Local-number:7725723 ==> No results match your search for 'Local-number:7725723' in NMA Catalog. sn=7725723 ==> No results match your search for 'sn=7725723' in NMA Catalog. Authority-Number:7725723 ==> No results match your search for 'Authority-Number:7725723' in NMA Catalog. an:7725723 ==> No results match your search for 'an:7725723' in NMA Catalog. Koha-Auth-Number:7725723 ==> No results match your search for 'Koha-Auth-Number:7725723' in NMA Catalog. control-number:7725723 ==> proper result control-number=7725723 ==> proper result so it appears that 'control-number:' or 'control-number=' is the only way of finding the biblio that corresponds to a 001 field. To make this easier for our cataloguers to edit possibly erroneous 001 fields, I tried adding: <option value="control-number">     Control Num. 001</option> to: search_indexes.inc which gives the "Option" in the "Advanced Search" drop downs, but: http://koha-admin/cgi-bin/koha/catalogue/search.pl?idx=control-number&q=772… ==> No results match your search for 'control-number,wrdl: 7725723' in NMA Catalog. It adds a ',wrdl' to the search term and fails. Also, it would appear that the search that works is: search.pl?q=control-number%3A+7725723 and the one that fails is: search.pl?idx=control-number&q=7725723&idx=kw&idx=kw&sort_by=relevance but I would appreciate help on the 'idx' usage compared to plain 'q' ... [NB - either : or = between search term and data appear to systematically give the same results.]

1 0

Bug 8665, files, git ...
by Paul 14 Oct '12

14 Oct '12

Good morning all, Probably a very naive question; I would like to try the proposed solution (Bug 8665) and can find the "diff", but when I try and navigate "git", the relevant section (if I have navigated successfully which is dubious as I'm new to "git") <https://github.com/Koha-Community/Koha/tree/master/etc/zebradb/marc_defs/ma…>, the master appears to stop at Bug 7475. Is there anywhere to find "ready edited" files, rather than copy/paste lines from the diff? Many thanks -- Paul

3 3

1st patch in bugzilla
by Mathieu Saby 13 Oct '12

13 Oct '12

Hi I've made my first bug report and my first patch in bugzilla. Even if you don't plan to test it, could you just take a look and tell me if there is something *formally *wrong with my bug and my patch ? I'm just a librairian, so I hope I've understood the process... http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8916 Should I write a mail with my patch to koha-patches(a)lists.koha-community.org now ? Kind regards, M. Saby Rennes 2 university -- Mathieu Saby Service d'Informatique Documentaire Service Commun de Documentation Université Rennes 2 Téléphone : 02 99 14 12 65 Courriel : mathieu.saby(a)univ-rennes2.fr

1 0

patron card PDF - without making a batch - v3.08?
by Waqar Azeem 12 Oct '12

12 Oct '12

Can koha admin generate patron-card without making a batch (Tools › Patron card creator)? I was looking for a feature that administrator can generate the patron-card directly from Patron detail page. During my code browsing I found two files 'print.pl' and 'create-pdf.pl' in 'patroncards' folder. Good thing was there was already a GET parameter 'borrower_number' (@borrower_numbers) that seem to be used to generate the PDF for a list of borrower. So, I think there is/was a feature from where i could chose a list of patrons without making a Batch. This code also seem to have a bug! I didn't find any online browse-able code therefore a snip-it from create-pdf.pl is here: grep { push(@{$items}, {*item_number* => $_}); } @borrower_numbers; 'item_number' itself is not used anywhere. 'browser_number' is used to retrieved this data. foreach my $item (@{$items}) { if ($item) { my $borrower_number = $item->{'*borrower_number*'}; When I replaced second with the first one, it stared to generate the pdf local URL to test: http://waqar/cgi-bin/koha/patroncards/print.pl?borrower_number=19 -- Thanks & Best Regards, Waqar Azeem

2 1

001 in 3.8.x
by Paul 12 Oct '12

12 Oct '12

We are - perhaps erroneously - coming to the conclusion that a part (major? minor?) of our "authorities" and possibly "other" problems in 3.8.4/5 is due to field 001. Error messages such as "Can't call method "data" on an undefined value at /usr/share/koha/lib/C4/AuthoritiesMarc.pm line 335" lead to 'my $authid=$authrecord->field('001')->data();' Philosophically - again perhaps erroneously - the definition of 001 does not require absolute uniqueness, but, taken in conjunction with 003 they should be [1]. We (unfortunately and for whatever reasons) have several hundred "duplicates" all with our 003 CaOPIACS attached. Our cataloguers are having a lengthy discussion about "our" responsibility, "original agency", "transcribing agency", "modifying agency" without any firm conclusion. So I'm left with the IT side of things ;={ Without major risk to the sql db, I am looking for a way to correct these, but will probably end up "editing" using the Koha interface -- but I am running into trouble and would appreciate some advice. We can't ask the cataloguers to "remember" the last number used, so in 3.6.1 I wrote a "plugin" called by field 001 from addbiblio based (extremely closely) on callnumber.pl that served us well. The only relevant changed line is: my $dbh = C4::Context->dbh; # If the textbox is empty, we return a simple incremented callnumber if ( $code eq "" ) { # my $sth = $dbh->prepare("SELECT MAX(CAST(itemcallnumber AS SIGNED)) FROM items"); ################# # replaced with # ################# my $sth = $dbh->prepare("SELECT MAX(CAST(TRIM(EXTRACTVALUE(marcxml,'//controlfield[\@tag=\"001\"]')) AS UNSIGNED)) from biblioitems where EXTRACTVALUE(marcxml,'//controlfield[\@tag=\"003\"]') = 'CaOPIACS'"); $sth->execute; if ( my $max = $sth->fetchrow ) { $template->param( return => $max+1, ); } It works well *if* the cataloguer (after clicking on the 003 to bring it to our framework default, books, fastadd, etc of 'CaOPIACS') remembers to delete the 001 and click on the icon. Could this be automated? i.e. IF {003 has changed} THEN {blank out 001} END IF (note I have made 001 mandatory, so records will not save with a blank entry.) I am just not savvy enough with the intracicies of perl to mysql to template, etc, coding. Or maybe a js guru could find something for addbiblio.js which seems to be empty...? Also (my dream to avoid manually editing hundreds of records), could any of you possibly come up with some mysql code to replace duplicates (I can define the starting number for replacement values). Or, maybe, (I'm sure there must have been a good reason not to use biblionumber for indexing zebra) just replace duplicate 001s with biblionumber (in my case, not necessarily true for anyone else, all out biblionumbers are less than 7 digits, and our 001s (and as far as I can see other major libraries via Z39.50) are at least 8 digits Many thanks and best regards, Paul Note [1]: from LoC <http://www.loc.gov/marc/bibliographic/bd001.html> 001 - Control Number (NR) MARC 21 Bibliographic - Full FIELD DEFINITION AND SCOPE Contains the control number assigned by the organization creating, using, or distributing the record. For interchange purposes, documentation of the structure of the control number and input conventions should be provided to exchange partners by the organization initiating the interchange. The MARC code identifying whose system control number is present in field 001 is contained in field 003 (Control Number Identifier).

2 1