- Koha-devel - lists.koha-community.org

Re: [Koha-devel] Help testing a bug (willing to swap sign offs)
by David Cook 19 Jan '16

19 Jan '16

Awesome! Cheers, Philippe. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007 From: Philippe Blouin [mailto:philippe.blouin@inlibro.com] Sent: Wednesday, 20 January 2016 12:11 AM To: David Cook <dcook(a)prosentient.com.au> Subject: Re: [Koha-devel] Help testing a bug (willing to swap sign offs) I have just the guy for you. I'll ask bouzid to have a look today. I want him to become an expert in indexes... Philippe Blouin, Responsable du développement informatique Tél. : (888) 604-2627 <mailto:philippe.blouin@inLibro.com> philippe.blouin(a)inLibro.com inLibro | pour esprit libre | <http://www.inLibro.com> www.inLibro.com On 01/19/2016 01:10 AM, David Cook wrote: Hey all, I would appreciate it if someone could test bug 15541 for me (http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15541) or even just provide feedback. I would be willing to swap testing/sign offs as well. Im hoping to get this one into the 3.24 release along with 15555, which is already signed off. Happy to walk people through any of the Zebra stuff as well if that makes anyone uncomfortable. Cheers, David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007 _______________________________________________ Koha-devel mailing list Koha-devel(a)lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/

1 0

Help testing a bug (willing to swap sign offs)
by David Cook 19 Jan '16

19 Jan '16

Hey all, I would appreciate it if someone could test bug 15541 for me (http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15541) or even just provide feedback. I would be willing to swap testing/sign offs as well. I'm hoping to get this one into the 3.24 release along with 15555, which is already signed off. Happy to walk people through any of the Zebra stuff as well if that makes anyone uncomfortable. Cheers, David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007

1 0

Re: [Koha-devel] Searching URLs in Zebra
by David Cook 19 Jan '16

19 Jan '16

Sorry to spam you all. The following works: $QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser') && ! ( grep { $_->{'index'} =~ m/\w,\w/ } @{ $self->{'matchpoints'} })); We might want to actually double-check other places in Koha where we're checking for 'UseQueryParser', as it might not always be possible to use the QueryParser due to limitations such as the "qualifier,qualifier" CCL syntax. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007 From: David Cook [mailto:dcook@prosentient.com.au] Sent: Tuesday, 19 January 2016 3:31 PM To: 'Koha Devel' <koha-devel(a)lists.koha-community.org> Subject: RE: Searching URLs in Zebra Ah, just realized again that qq($QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser') && ! ($query =~ m/\w,\w|\w=\w/));) won't work in C4::Matcher because there's no $query. The only equivalent of that would be to check all the $matchpoint->{'index'} fields in the $self->{'matchpoints'} arrayref. Which we might have to do because the QueryParser doesn't seem to be able to handle the "index,phr" (ie "qualifier,qualifier") format. That's verified by the fact that C4::Search::SimpleSearch is doing that regex. Sure, we could just not use a list of qualifiers for a Record Matching Rule index, but then we have no control over what index register we're using and that's also a problem. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007 From: David Cook [mailto:dcook@prosentient.com.au] Sent: Tuesday, 19 January 2016 3:11 PM To: 'Koha Devel' <koha-devel(a)lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> > Subject: Searching URLs in Zebra Hey brains trust, I'm reaching out to everyone as I'm a bit stuck with http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15541 at the moment. There's currently no way to search an index with the "url" (ie "u") register. A URL is stored "as is" in that index, but all our Koha search methods use s/:/=/g or s/=/:/g and that plays havoc with the URLs used in a search query, which makes the match fail against the indexed URL. C4::Matcher::get_matches() strips spaces and punctuation, so I've added a "none" or "raw" normalizer that performs no normalization, but C4::Search::SimpleSearch() does that global replacement for ":" and "=" (depending on your QueryParser settings and what qualifiers you've specified in your Record Matching Rules). I thought about adding a "skip normalization" flag to C4::Search::SimpleSearch, but that doesn't work because C4::Matcher uses : for QueryParser and = for non-QueryParser. except C4::Search::SimpleSearch() deactivates QueryParser mode if you have a "index,phr" set of qualifiers. which means that you get a ":" when you need a "=" for the non-QueryParser CCL query. I suppose the solution might be the following: Current Matcher.pm: $QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser')); Current Search.pm $QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser') && ! ($query =~ m/\w,\w|\w=\w/)); Future Matcher.pm: $QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser') && ! ($query =~ m/\w,\w|\w=\w/)); If I just have that conditional the same in both Search.pm and Matcher.pm. then maybe a "skip_normalization" flag in SimpleSearch() would work. I might try that in a minute. but opening it up because I could really use a hand with this. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007

1 0

Re: [Koha-devel] Searching URLs in Zebra
by David Cook 19 Jan '16

19 Jan '16

Ah, just realized again that qq($QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser') && ! ($query =~ m/\w,\w|\w=\w/));) won't work in C4::Matcher because there's no $query. The only equivalent of that would be to check all the $matchpoint->{'index'} fields in the $self->{'matchpoints'} arrayref. Which we might have to do because the QueryParser doesn't seem to be able to handle the "index,phr" (ie "qualifier,qualifier") format. That's verified by the fact that C4::Search::SimpleSearch is doing that regex. Sure, we could just not use a list of qualifiers for a Record Matching Rule index, but then we have no control over what index register we're using and that's also a problem. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007 From: David Cook [mailto:dcook@prosentient.com.au] Sent: Tuesday, 19 January 2016 3:11 PM To: 'Koha Devel' <koha-devel(a)lists.koha-community.org> Subject: Searching URLs in Zebra Hey brains trust, I'm reaching out to everyone as I'm a bit stuck with http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15541 at the moment. There's currently no way to search an index with the "url" (ie "u") register. A URL is stored "as is" in that index, but all our Koha search methods use s/:/=/g or s/=/:/g and that plays havoc with the URLs used in a search query, which makes the match fail against the indexed URL. C4::Matcher::get_matches() strips spaces and punctuation, so I've added a "none" or "raw" normalizer that performs no normalization, but C4::Search::SimpleSearch() does that global replacement for ":" and "=" (depending on your QueryParser settings and what qualifiers you've specified in your Record Matching Rules). I thought about adding a "skip normalization" flag to C4::Search::SimpleSearch, but that doesn't work because C4::Matcher uses : for QueryParser and = for non-QueryParser. except C4::Search::SimpleSearch() deactivates QueryParser mode if you have a "index,phr" set of qualifiers. which means that you get a ":" when you need a "=" for the non-QueryParser CCL query. I suppose the solution might be the following: Current Matcher.pm: $QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser')); Current Search.pm $QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser') && ! ($query =~ m/\w,\w|\w=\w/)); Future Matcher.pm: $QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser') && ! ($query =~ m/\w,\w|\w=\w/)); If I just have that conditional the same in both Search.pm and Matcher.pm. then maybe a "skip_normalization" flag in SimpleSearch() would work. I might try that in a minute. but opening it up because I could really use a hand with this. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007

1 0

Searching URLs in Zebra
by David Cook 19 Jan '16

19 Jan '16

Hey brains trust, I'm reaching out to everyone as I'm a bit stuck with http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15541 at the moment. There's currently no way to search an index with the "url" (ie "u") register. A URL is stored "as is" in that index, but all our Koha search methods use s/:/=/g or s/=/:/g and that plays havoc with the URLs used in a search query, which makes the match fail against the indexed URL. C4::Matcher::get_matches() strips spaces and punctuation, so I've added a "none" or "raw" normalizer that performs no normalization, but C4::Search::SimpleSearch() does that global replacement for ":" and "=" (depending on your QueryParser settings and what qualifiers you've specified in your Record Matching Rules). I thought about adding a "skip normalization" flag to C4::Search::SimpleSearch, but that doesn't work because C4::Matcher uses : for QueryParser and = for non-QueryParser. except C4::Search::SimpleSearch() deactivates QueryParser mode if you have a "index,phr" set of qualifiers. which means that you get a ":" when you need a "=" for the non-QueryParser CCL query. I suppose the solution might be the following: Current Matcher.pm: $QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser')); Current Search.pm $QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser') && ! ($query =~ m/\w,\w|\w=\w/)); Future Matcher.pm: $QParser = C4::Context->queryparser if (C4::Context->preference('UseQueryParser') && ! ($query =~ m/\w,\w|\w=\w/)); If I just have that conditional the same in both Search.pm and Matcher.pm. then maybe a "skip_normalization" flag in SimpleSearch() would work. I might try that in a minute. but opening it up because I could really use a hand with this. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007

1 0

Fwd: Call for testers: next release of Net::OAI::Harvester Perl module may break legacy custom handlers
by Tajoli Zeno 18 Jan '16

18 Jan '16

-------- Messaggio Inoltrato -------- Oggetto: Call for testers: next release of Net::OAI::Harvester Perl module may break legacy custom handlers Data: Fri, 15 Jan 2016 12:38:17 +0100 Mittente: Thomas Berger <ThB(a)Gymel.com> Rispondi-a: ThB(a)Gymel.com Organizzazione: Gymel.com A: Perl for libraries <perl4lib(a)perl.org>, Using XML in libraries <xml4lib(a)listserv.nd.ed> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 [Apologies for cross-posting] The Perl module Net::OAI::Harvester implements a client framework for the OAI Protocol for Metadata Harvesting (OAI-PMH) and was authored and originally maintained by Ed Summers. It has been available on CPAN ever since 2003 and its last stable version 1.15 has been released almost four years ago: < http://search.cpan.org/~thb/OAI-Harvester-1.15/ >. Since one of the repositories used for testing vanished from the web some time ago and this is breaking the test suite a new version has to be released fairly soon. Over time I had been tackling various minor issues and published developer releases on CPAN, cf. the list at the end of this mail or the Changes document linked at the CPAN page for the current release 1.16_12: < http://search.cpan.org/~thb/OAI-Harvester-1.16_12/ > However the sum of these changes is not negligible and specifically their impact on "custom metadata handlers" (which are to be used when processing other metadata formats than oai_dc) may affect applications using the module: >>>>> Up to version 1.15 the metadataHandler was inconsistently fed with input : - GetRecord exposed the almost complete XML response to the Handler (including start_document/end_document events) - ListRecords exposed the (OAI)record element (header, metadata and optional about containers) but did not propagate start_document or end_document events. In both cases the events for the header tags itself and for the optional setSpec subelements had not been forwarded Version 1.20 introduces a modified behavior for metadataHandler and an additional recordHandler: - a metadataHandler will see only the (single) subelement of the OAI metadata element (so for an deleted record it might never be invoced at all) - a recordHandler will see the OAI record element and its subelements Therefore a metadataHandler will now be confined to the metadata fragment(s) of the response, and the new recordHandler approximates the old behavior of ListRecords, however OAI-PMH:identifier and OAI-PMH:datestamp will now be properly encapsulated within their OAI-PMH:header element. Additionally, two new methods responseDate() and request() allow access to the corresponding top-level OAI-PMH elements in all response types. A SAX filter of class Net::OAI::Record::DocumentHelper may be used to inject start_document and end_document events into the chain if they are needed. As a temporary measure, you may set $Net::OAI::Harvester::OLDmetadataHandler =1 to change the behavior of handlers passed as "metadataHandler" into that of a recordHandler. <<<<<< Obviously the change of semantics for a metadataHandler to deal with the "metadata" elements of the response instead of the "record" elements is a design decision and may be questioned by users of the module. The current version also contains several changes which solve deficiencies of Net::OAI::Harvester 1.15 but possibly break existing workarounds for these deficiencies. For example officially (per documentation) you never could acccess the responseDate of the OAI-PMH result, but due to a sloppy implementation of processing for the identify verb it was possible to extract it in this case by an undocumented method. The current version supplies a dedicated responseDate() accessor for all verbs but at the same time fixes the behavior in the identify case. I may be overly optimistic but my impression is that the changes between the current 1.15 and the coming version (most probably numbered 1.20) do actually fix many issues but the fear is realistic (I experienced that myself with an old application of mine using the module) that these fixes may conflict with workarounds introduced by users to make things work before. *** So please, if you are currently using Net::OAI::Harvester *and* had been forced to introduce workarounds or tweak internals of the module, perform thorough testing before upgrading to the coming stable version, preferably already now with the developer version 1.16_12. And, please, please: provide feedback if you should run into trouble, either via the CPAN request tracker for the module at < https://rt.cpan.org/Public/Dist/Display.html?Name=OAI-Harvester > or by direct mail. Sorry for the inconvenience viele Gruesse Thomas Berger Changes to Net::OAI::Harvester since version 1.15 1.16_12 Tue, Jan 12 00:20:05 CET 2016 - - dealing with CPANTS Kwalitee issues, esp. version number mess - - new filter class Net::OAI::Record::DocumentHelper for tweaking 1.16_11 Tue, Jan 12 00:20:05 CET 2016 - - minor cleanup 1.16_10 Mon, Jan 11 01:29:46 CET 2016 - - renamed alldata() method for accessing recordHandler results to recorddata() - - better propagation of namespace prefix mapping events - - Net::OAI::NamespaceFilter with a result() method - - Net::OAI::NamespaceFilter tested with XML::SAX::Writer - - AUTHOR formatting 1.16_09 Sun, Feb 14 17:29:39 CET 2014 - - Net::OAI::NamespaceFilter as kind of generic metadata handler - - Queries are now constructed basing on a copy of the Harvester's baseURL - - pass parameters to URI->query_form() more reproducably, (esp. "verb" should now always be first to accommodate some allegedly broken repositories) - - temporary? tests for correctness of LWP operations 1.16_07 Tue, Apr 30 01:26:40 CEST 2013 - - added new methods: response(), responseDate(), error() - - Smoke still tests failed on 'Bad Host' tests (wrong error codes induced by HTTP proxies?) - - aligned behavior of metadataHandler for listRecords() and getRecord() - - introducing alternative recordHandler for listRecords() and getRecord() - - removed erroneous resumptionToken handling for identify() 1.16_04 Fri Dec 7 09:49:03 CET 2012 - - consider HTTP proxies in design of t/003.error.t - - 'Bad Host' tests failing b/c error code 500 is not the expected code 404 (due to some recent change in LWP)? 1.16_01 Mon Apr 2 23:14:35 CEST 2012 - - Modules were not namespace aware. - - Add HTTPRetryAfter() method (catches HTTP Retry-After header) - - Check responses for Content-Type and charset before parsing - - Net::OAI::Header handed up (empty) header elements and other stuff to the request's metadataHandler - - SKIP tests when HTTP errors are encountered -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iJwEAQECAAYFAlaY2iYACgkQYhMlmJ6W47MMCwP/Yhij11TfEL1dfYtimdXG8hkf FYLvvXECzECPxKbHIC0dKvf5v4myW8oedlK3B+oOzIjjOY60pT7pdC4KB/xgU+a1 N1djewSgT4hJ3IoacmUkLpnh81NSM1oA0osw48qVco4qpxDOY2HrR3bdBZksKBcI lQH10kIYqo/TZYGXHYQ= =v03A -----END PGP SIGNATURE-----

2 1

Hide records on Leader 05 = d in OPAC
by David Cook 18 Jan '16

18 Jan '16

Hi all: Recently, http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11084 was pushed to master. It adds a cronjob which deletes bibliographic records, if their LDR05 is "d" (ie status deleted). While I'm all for that, I'm thinking that it would also be a good idea to hide these records with LDR05 "d" in the OPAC before the cronjob is run. We can't necessarily rely on all Koha instances running this cronjob, nor can we rely on the frequency. Shouldn't we be hiding these records from the OPAC as soon as they're marked as "deleted"? I've opened a bug for this purpose: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15537 It should be a pretty easy change, especially when using special attribute @attr 14=1 to avoid past OpacSuppression problems, but I thought I'd ask folk if it's something they'd be interested in. We'd probably still want to see these records in the Staff Client, as we want to be able to find all records in the database in the Staff Client. I admit that I have a special interest in this where I might be overlaying existing records using a mostly empty skeleton record generated from an OAI-PMH identifier and a OAI-PMH deleted status (OAI-PMH doesn't send metadata for deleted records). I'd match the existing record in Koha using the identifier, and then set LDR05 to "d" in accordance with the OAI-PMH deleted status. Then, that record would disappear from the OPAC, so that end users don't see this skeleton record. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St, Ultimo, NSW 2007

5 11

Call for news: January 2016 Newsletter
by kohanews 16 Jan '16

16 Jan '16

Fellow Koha users ~ I'm collecting news for the January newsletter. Send anything noteworthy to: k o h a news AT gmail dot com News criteria: --------------------------- ** For events **: - Please include dates for past events. If I can't find dates I may not add it. - Announcements for future events with dates T.B.A. are fine ...Eg., Kohacon - For past events -- **** one month back is the cut-off ***. * News items can be of any length. * Images are fine * Anything and everything Koha. * Submit by the 26th of the month. If you are working on an interesting project or development related to Koha, please let me know and I'll include it in the development section. -- Chad Roseburg Editor, Koha Community Newsletter

1 0

Call for development news: January 2016 Newsletter
by kohanews 15 Jan '16

15 Jan '16

Let the community hear what cool things you're working on! Send me some bugs that need testing, sign-offs and user feedback. Suggestions welcome! k o h a news AT gmail dot com Thank you! -- Chad Roseburg Editor, Koha Community Newsletter

1 0

Merge borrowers and deletedborrowers tables
by Jonathan Druart 15 Jan '16

15 Jan '16

Hi devs, I am sure the community has already discussed about that: I am wondering why do we have a borrowers and a deletedborrowers tables when a "delete" flag in the borrowers table could do the trick. I am sure it's a matter of optimizing the sql queries, but did someone already compare on a descent database? It would help a lot to have the foreign keys set on other tables, to display the data if the library wants to displayed them. See bugs 13534 13668 14849 Doing this way, we make the DB structure consistent, but we will lost info. Cheers, Jonathan

5 5