From philippe.blouin at inlibro.com Mon Aug 1 22:30:14 2016 From: philippe.blouin at inlibro.com (Philippe Blouin) Date: Mon, 1 Aug 2016 16:30:14 -0400 Subject: [Koha-devel] How long is a call to schema() supposed to take Message-ID: <579FB156.7060808@inlibro.com> Hello Koha developpers! I got a question to all the dbix experts out there (I'm not one of them, unfortunatly). I have performance issues in the checkout that remains even after boosting my server's performance with a SSD drive. In C4/Circulation.pm/*AddIssue*, a lot of transactions take >1s for the call to schema $issue = Koha::Database->new()->_*schema()*_->resultset('Issue')->create( This is *3.20*. I specifically isolated that call with timers, I do a bunch of checkouts for the same user, and repetitively, that call take over 1s. The machine is a beast (ram, cpu...), so what should I be doing? Thanks, -- Philippe Blouin, Responsable du d?veloppement informatique T?l. : (888) 604-2627 philippe.blouin at inLibro.com inLibro | pour esprit libre | www.inLibro.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From kyle.m.hall at gmail.com Mon Aug 1 22:32:56 2016 From: kyle.m.hall at gmail.com (Kyle Hall) Date: Mon, 1 Aug 2016 16:32:56 -0400 Subject: [Koha-devel] How long is a call to schema() supposed to take In-Reply-To: <579FB156.7060808@inlibro.com> References: <579FB156.7060808@inlibro.com> Message-ID: Or send that newer versions of dbix class have improved efficiencies. You might want to experiment with using newer versions of the module than what comes with that version of Koha by default. Kyle Sent from my phone. Please excuse my brevity. On Aug 1, 2016 4:30 PM, "Philippe Blouin" wrote: > Hello Koha developpers! > > I got a question to all the dbix experts out there (I'm not one of them, > unfortunatly). I have performance issues in the checkout that remains even > after boosting my server's performance with a SSD drive. > > In C4/Circulation.pm/*AddIssue*, a lot of transactions take >1s for the > call to schema $issue = > > Koha::Database->new()->*schema()*->resultset('Issue')->create( > > This is *3.20*. I specifically isolated that call with timers, I do a > bunch of checkouts for the same user, and repetitively, that call take over > 1s. The machine is a beast (ram, cpu...), so what should I be doing? > > Thanks, > -- > Philippe Blouin, > Responsable du d?veloppement informatique > > T?l. : (888) 604-2627 > philippe.blouin at inLibro.com > inLibro | pour esprit libre | www.inLibro.com > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From philippe.blouin at inlibro.com Mon Aug 1 22:56:36 2016 From: philippe.blouin at inlibro.com (Philippe Blouin) Date: Mon, 1 Aug 2016 16:56:36 -0400 Subject: [Koha-devel] How long is a call to schema() supposed to take In-Reply-To: References: <579FB156.7060808@inlibro.com> Message-ID: <579FB784.3050306@inlibro.com> Hi Kyle! Not sure I understand your first sentence... But what you tell me is to try to move to 3.20.X, with X great than the 3.20.01 that I currently run? Not easily feasible with the stack over it, but just knowing that it is software and has been worked on is good news to me. Thanks! Philippe Blouin, Responsable du d?veloppement informatique T?l. : (888) 604-2627 philippe.blouin at inLibro.com inLibro | pour esprit libre | www.inLibro.com On 08/01/2016 04:32 PM, Kyle Hall wrote: > > Or send that newer versions of dbix class have improved efficiencies. > You might want to experiment with using newer versions of the module > than what comes with that version of Koha by default. > > Kyle > > Sent from my phone. Please excuse my brevity. > > > On Aug 1, 2016 4:30 PM, "Philippe Blouin" > wrote: > > Hello Koha developpers! > > I got a question to all the dbix experts out there (I'm not one of > them, unfortunatly). I have performance issues in the checkout > that remains even after boosting my server's performance with a > SSD drive. > > In C4/Circulation.pm/*AddIssue*, a lot of transactions take >1s > for the call to schema $issue = > > Koha::Database->new()->_*schema()*_->resultset('Issue')->create( > > This is *3.20*. I specifically isolated that call with timers, I > do a bunch of checkouts for the same user, and repetitively, that > call take over 1s. The machine is a beast (ram, cpu...), so what > should I be doing? > > Thanks, > -- > Philippe Blouin, > Responsable du d?veloppement informatique > > T?l. : (888) 604-2627 > philippe.blouin at inLibro.com > > inLibro | pour esprit libre | www.inLibro.com > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Katrin.Fischer.83 at web.de Mon Aug 1 22:57:32 2016 From: Katrin.Fischer.83 at web.de (Katrin Fischer) Date: Mon, 1 Aug 2016 22:57:32 +0200 Subject: [Koha-devel] Pre-RFC on new discrete calendar In-Reply-To: <579606CB.9030201@inlibro.com> References: <5790FBB6.8020009@inlibro.com> <579606CB.9030201@inlibro.com> Message-ID: <60f001c4-5c84-57b4-0a39-fb81c3a044c3@web.de> Hi Philippe, I still got some question marks about this idea... mostly it doesn't feel quite right so far. > > Let's start with the backend: this is just a simple script that fills > the calendar ahead of time (waaay ahead of time) to allow the user to > modify schedules easily. 1 year is just a hardcoded value, it could > as well be an argument allowing any number of days in the future. > Same for the past: during updatedatabase, this creates one but could > create two years in the past based on the info in the current calendar > tables. In the end, the point is to have a table entry for each day, > specifying which day it WAS opened (for hourly fine calculation) and > which day it will be opened (for hourly checkout or just for > displaying the library open hours on the OPAC). Going a year in the > future, things are a bit dumb when creating, but always replicate the > week before, except for items with notes (holiday) that are fetched on > the calendar year a year before. Then the librarian has plenty of > time to adjust anything. I think depending on the notes is not a good idea. You can add notes to every kind of holiday at the moment - so you'd have to take a look at the type of holiday - repeated yearly/weekly and unique. You'd also need to take into account the exceptions to repeated holidays that you can currently define. > > Which brings us to the UI. Mine is ugly, but it would be easy to > create a nice one AND SIMPLE one (coding) and powerful one (for > users). With that simple backend, it's very easy to simply allow > multiple selections in the calendar widget, the modify opening hours, > or holiday close with a note. Or better: select a week anywhere in > the calendar, then copy that to a given range. 3 days, a month, 3 > months... Very simple in the UI, very few clicks. Very simple to > code in the backend. It sounds like this type of calendar would need more regular maintenance than the current system? > > So in the end, recovering the original work or Kyle's work or defining > new standard, we have a calendar page with a simple calendar and below > it a few edit box (opening hours, closing, notes, closed checkbox) and > a apply button. On the right, like right now, we can display all > "special dates", which are the ones with a "note" entry. In yellow > those that are on days still opened, in pink those days that are > marked as closed. Of course, all UI schemes are very open to > suggestion. But it would be simple and naturally intuitive. If I was to make a change to the calendar - would I have to wait for the cronjob/script to run and update the table or would it take effect immediately? What will happen if a due date or other date is outside of the calendar? I feel like by relying on a fixed date range, we are going to create problems along the road if there is no rule based system as a backup at least. Katrin > > Philippe Blouin, > Responsable du d?veloppement informatique > > T?l. : (888) 604-2627 > philippe.blouin at inLibro.com > > inLibro | pour esprit libre | www.inLibro.com > On 07/24/2016 04:42 PM, Katrin Fischer wrote: >> Hi Philippe, >> >> thx for trying to get things moving again - I know there are quite a lot >> calendar related bugs to be found in bugzilla. >> >> Can you explain a bit about how this would change the GUI for the users? >> Do you have to keep it up to date or does the table get filled >> automatically for recurring events? >> >> I am a bit concerned about the limitation of one year into the past and >> one year into the future. What happens if a due date goes beyond that or >> an item is overdue before that? >> >> Katrin >> >> Am 21.07.2016 um 18:43 schrieb Philippe Blouin: >>> Hi! >>> >>> I'm throwing a line here, and I'd just like to get a feel for the value >>> of offering some work to the community. Mind you, the work is "big" so >>> honest responses could save us lot of wasted hours. >>> >>> We've developed a parallel calendar table to specify each individual day >>> if it's opened or not (instead of rules and exception). We added to it >>> the opening hours, and keep a year of them in the past, and a year in >>> the future. >>> The reasonning being: >>> - We need the opening hours. They need to vary season to seasons. We >>> need them for hourly and minute loans. >>> - Exception and holidays and etc... are complicated. To manage, to >>> calculate, to fix. We need the past info as well, to calculate precisely. >>> - Performance. Calculating with C4/Koha Calendars is sloooooooooow. >>> Our little table cut fines.pl calculation times by 97%. Not a typo. >>> Checkout improvement by 30-60% but metric is unreliable so take with >>> grain of salt this one. >>> >>> So before I go and write a wiki RFC, then open bugzillas, make the code >>> community acceptable (we're not using Schemas), complete it, write >>> tests, etc... Is there an interest? Would it answer a need (outside of >>> our clients) ? Maybe a subset? >>> >>> All comments, suggestions, questions are welcomed. >>> >>> High regards, >>> >>> Philippe Blouin, >>> Responsable du d?veloppement informatique >>> >>> T?l. : (888) 604-2627 >>> philippe.blouin at inLibro.com >>> >>> inLibro | pour esprit libre | www.inLibro.com >>> >>> >>> >>> _______________________________________________ >>> Koha-devel mailing list >>> Koha-devel at lists.koha-community.org >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>> website : http://www.koha-community.org/ >>> git : http://git.koha-community.org/ >>> bugs : http://bugs.koha-community.org/ >>> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at bigballofwax.co.nz Mon Aug 1 23:13:50 2016 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Tue, 2 Aug 2016 09:13:50 +1200 Subject: [Koha-devel] General IRC Meeting 3 August 2016 / Kohacon 2017 bidder information In-Reply-To: References: Message-ID: Hi All I just want to second Mirko's email about Kohacon17, at the moment we only have the one bid (For the Philippines) as Kohacon was in Europe this year and can't be in the same continent within 3 years. However there is a real lack of information, or contact with any people in Philippines about their bid. For example the hosting organisations are named as the National Library and Onstrike, but the community has had no contact with anyone from either organisation. So we really need either a mail to the list of them to show at the next meeting. Also I would heartily recommend any organisation outside of Europe or Africa (the last 2 continents) to propose also. It isn't too late, but we do need an actual contact so we can be sure things will happen. Thanks Chris On 29 July 2016 at 20:56, Mirko Tietgen wrote: > Hi everyone, > > the next general IRC meeting is next week, 3 August 2016, 10:00 UTC. > > Conversion to your local time: > http://www.timeanddate.com/worldclock/fixedtime.html?msg=Koha+IRC+General+Meeting&iso=20160803T10 > > You can find the agenda here: > https://wiki.koha-community.org/wiki/General_IRC_meeting_3_August_2016 > > This is the last meeting before we close bidding to host Kohacon > 2017. It would be nice if bidders would join us at the meeting and > tell a little about their plans. > > If the meeting is not an option, please send a message to the > mailing list explaining that you are still interested in hosting > Kohacon 2017, and please update your proposals in the Koha wiki now > if necessary: Narrow down the proposed dates if possible, update > conference locations, add contact emails(!). > https://wiki.koha-community.org/wiki/KohaCon17_Proposals > > We are still open for proposals to host Kohacon 2017 until 10 August > 2016 UTC. > > Cheers, > > Mirko > > > -- > > Mirko Tietgen > mirko at abunchofthings.net > http://koha.abunchofthings.net > http://meinkoha.de > > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From jonathan.druart at bugs.koha-community.org Tue Aug 2 11:02:49 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Tue, 2 Aug 2016 10:02:49 +0100 Subject: [Koha-devel] How long is a call to schema() supposed to take In-Reply-To: <579FB156.7060808@inlibro.com> References: <579FB156.7060808@inlibro.com> Message-ID: Hi Philippe, For the last 2 versions, we have been working a lot on performances, so it's hard to tell you what we did to fix that. But have a look at bug 14778, you can try to apply it on 3.20.x and see if it fixes your issue. You should plan to move to 16.05 using plack :) Cheers, Jonathan 2016-08-01 21:30 GMT+01:00 Philippe Blouin : > Hello Koha developpers! > > I got a question to all the dbix experts out there (I'm not one of them, > unfortunatly). I have performance issues in the checkout that remains even > after boosting my server's performance with a SSD drive. > > In C4/Circulation.pm/AddIssue, a lot of transactions take >1s for the call > to schema $issue = > > Koha::Database->new()->schema()->resultset('Issue')->create( > > This is 3.20. I specifically isolated that call with timers, I do a bunch > of checkouts for the same user, and repetitively, that call take over 1s. > The machine is a beast (ram, cpu...), so what should I be doing? > > Thanks, > -- > Philippe Blouin, > Responsable du d?veloppement informatique > > T?l. : (888) 604-2627 > philippe.blouin at inLibro.com > > inLibro | pour esprit libre | www.inLibro.com > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From M.de.Rooy at rijksmuseum.nl Tue Aug 2 11:27:16 2016 From: M.de.Rooy at rijksmuseum.nl (Marcel de Rooy) Date: Tue, 2 Aug 2016 09:27:16 +0000 Subject: [Koha-devel] General IRC Meeting 3 August 2016 / Kohacon 2017 bidder information In-Reply-To: References: , Message-ID: > as Kohacon was in Europe this year and can't be in the same continent within 3 years. Little bit a side note: As I understand, this is some kind of unwritten rule in the community. Maybe we should write it down somewhere and relax it a little bit? KohaCon should preferably not be held on the same continent twice in a row (unless an alternative is not available). Or something similar ?? Marcel -------------- next part -------------- An HTML attachment was scrubbed... URL: From M.de.Rooy at rijksmuseum.nl Tue Aug 2 11:30:39 2016 From: M.de.Rooy at rijksmuseum.nl (Marcel de Rooy) Date: Tue, 2 Aug 2016 09:30:39 +0000 Subject: [Koha-devel] How long is a call to schema() supposed to take In-Reply-To: <579FB156.7060808@inlibro.com> References: <579FB156.7060808@inlibro.com> Message-ID: Philippe, Are you using Jessie or Wheezy ? I have seen some interesting performance differences between the versions of DBIx shipped with these Debian versions. Jessie is much faster. Marcel ________________________________ Van: koha-devel-bounces at lists.koha-community.org namens Philippe Blouin Verzonden: maandag 1 augustus 2016 22:30:14 Aan: koha-devel at lists.koha-community.org Onderwerp: [Koha-devel] How long is a call to schema() supposed to take Hello Koha developpers! I got a question to all the dbix experts out there (I'm not one of them, unfortunatly). I have performance issues in the checkout that remains even after boosting my server's performance with a SSD drive. In C4/Circulation.pm/AddIssue, a lot of transactions take >1s for the call to schema $issue = Koha::Database->new()->schema()->resultset('Issue')->create( This is 3.20. I specifically isolated that call with timers, I do a bunch of checkouts for the same user, and repetitively, that call take over 1s. The machine is a beast (ram, cpu...), so what should I be doing? Thanks, -- Philippe Blouin, Responsable du d?veloppement informatique T?l. : (888) 604-2627 philippe.blouin at inLibro.com inLibro | pour esprit libre | www.inLibro.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From philippe.blouin at inlibro.com Tue Aug 2 15:05:53 2016 From: philippe.blouin at inlibro.com (Philippe Blouin) Date: Tue, 2 Aug 2016 09:05:53 -0400 Subject: [Koha-devel] How long is a call to schema() supposed to take In-Reply-To: References: <579FB156.7060808@inlibro.com> Message-ID: <57A09AB1.7060902@inlibro.com> This is Jessie. That's maybe part of the issue, though. While changing hardware, we also upgraded the OS/apache/mysql. I was kinda hoping for some big gain, but this 1sec delay really stands out. But following everyone's comments, ll look into 14478, plack... and simply moving everything to 16.05 :) Any good doc on plack? I got https://wiki.koha-community.org/wiki/Plack, but 5 lines aren't much, not even a clue what is :) Thanks everyone! Philippe Blouin, Responsable du d?veloppement informatique T?l. : (888) 604-2627 philippe.blouin at inLibro.com inLibro | pour esprit libre | www.inLibro.com On 08/02/2016 05:30 AM, Marcel de Rooy wrote: > > Philippe, > > Are you using Jessie or Wheezy ? > > I have seen some interesting performance differences between the > versions of DBIx shipped with these Debian versions. > > Jessie is much faster. > > > Marcel > > ------------------------------------------------------------------------ > *Van:* koha-devel-bounces at lists.koha-community.org > namens Philippe Blouin > > *Verzonden:* maandag 1 augustus 2016 22:30:14 > *Aan:* koha-devel at lists.koha-community.org > *Onderwerp:* [Koha-devel] How long is a call to schema() supposed to take > Hello Koha developpers! > > I got a question to all the dbix experts out there (I'm not one of > them, unfortunatly). I have performance issues in the checkout that > remains even after boosting my server's performance with a SSD drive. > > In C4/Circulation.pm/*AddIssue*, a lot of transactions take >1s for > the call to schema $issue = > > Koha::Database->new()->_*schema()*_->resultset('Issue')->create( > > This is *3.20*. I specifically isolated that call with timers, I do a > bunch of checkouts for the same user, and repetitively, that call take > over 1s. The machine is a beast (ram, cpu...), so what should I be doing? > > Thanks, > -- > Philippe Blouin, > Responsable du d?veloppement informatique > > T?l. : (888) 604-2627 > philippe.blouin at inLibro.com > > inLibro | pour esprit libre | www.inLibro.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomascohen at gmail.com Tue Aug 2 15:27:37 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Tue, 02 Aug 2016 13:27:37 +0000 Subject: [Koha-devel] How long is a call to schema() supposed to take In-Reply-To: <57A09AB1.7060902@inlibro.com> References: <579FB156.7060808@inlibro.com> <57A09AB1.7060902@inlibro.com> Message-ID: It is 3 lines :-D And is your instance name, provided you have a packages-based setup, with created instances. El mar., 2 ago. 2016 a las 10:06, Philippe Blouin (< philippe.blouin at inlibro.com>) escribi?: > This is Jessie. That's maybe part of the issue, though. While changing > hardware, we also upgraded the OS/apache/mysql. I was kinda hoping for > some big gain, but this 1sec delay really stands out. > > But following everyone's comments, ll look into 14478, plack... and simply > moving everything to 16.05 :) > > Any good doc on plack? I got https://wiki.koha-community.org/wiki/Plack, > but 5 lines aren't much, not even a clue what is :) > > Thanks everyone! > > > > Philippe Blouin, > Responsable du d?veloppement informatique > > T?l. : (888) 604-2627 > philippe.blouin at inLibro.com > inLibro | pour esprit libre | www.inLibro.com > On 08/02/2016 05:30 AM, Marcel de Rooy wrote: > > Philippe, > > Are you using Jessie or Wheezy ? > > I have seen some interesting performance differences between the versions > of DBIx shipped with these Debian versions. > > Jessie is much faster. > > > Marcel > ------------------------------ > *Van:* koha-devel-bounces at lists.koha-community.org > > namens Philippe Blouin > > *Verzonden:* maandag 1 augustus 2016 22:30:14 > *Aan:* koha-devel at lists.koha-community.org > *Onderwerp:* [Koha-devel] How long is a call to schema() supposed to take > > Hello Koha developpers! > > I got a question to all the dbix experts out there (I'm not one of them, > unfortunatly). I have performance issues in the checkout that remains even > after boosting my server's performance with a SSD drive. > > In C4/Circulation.pm/*AddIssue*, a lot of transactions take >1s for the > call to schema $issue = > > Koha::Database->new()->*schema()*->resultset('Issue')->create( > > This is *3.20*. I specifically isolated that call with timers, I do a > bunch of checkouts for the same user, and repetitively, that call take over > 1s. The machine is a beast (ram, cpu...), so what should I be doing? > > Thanks, > -- > Philippe Blouin, > Responsable du d?veloppement informatique > > T?l. : (888) 604-2627 > philippe.blouin at inLibro.com > inLibro | pour esprit libre | www.inLibro.com > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From philippe.blouin at inlibro.com Tue Aug 2 15:36:26 2016 From: philippe.blouin at inlibro.com (Philippe Blouin) Date: Tue, 2 Aug 2016 09:36:26 -0400 Subject: [Koha-devel] How long is a call to schema() supposed to take In-Reply-To: References: <579FB156.7060808@inlibro.com> <57A09AB1.7060902@inlibro.com> Message-ID: <57A0A1DA.6050408@inlibro.com> haaaa, No, got git setups. Any instructions on that? two lines maybe? one? :-) Was wondering actually if having plack for one install on 16.05 would break another installation on 3.20 sharing apache. Most things Apache are secluded by virtual host, but some are global. I live in fear... Philippe Blouin, Responsable du d?veloppement informatique T?l. : (888) 604-2627 philippe.blouin at inLibro.com inLibro | pour esprit libre | www.inLibro.com On 08/02/2016 09:27 AM, Tomas Cohen Arazi wrote: > It is 3 lines :-D > > And is your instance name, provided you have a > packages-based setup, with created instances. > > > El mar., 2 ago. 2016 a las 10:06, Philippe Blouin > (>) > escribi?: > > This is Jessie. That's maybe part of the issue, though. While > changing hardware, we also upgraded the OS/apache/mysql. I was > kinda hoping for some big gain, but this 1sec delay really stands out. > > But following everyone's comments, ll look into 14478, plack... > and simply moving everything to 16.05 :) > > Any good doc on plack? I got > https://wiki.koha-community.org/wiki/Plack, but 5 lines aren't > much, not even a clue what is :) > > Thanks everyone! > > > > Philippe Blouin, > Responsable du d?veloppement informatique > > T?l. : (888) 604-2627 > philippe.blouin at inLibro.com > > inLibro | pour esprit libre | www.inLibro.com > > On 08/02/2016 05:30 AM, Marcel de Rooy wrote: >> >> Philippe, >> >> Are you using Jessie or Wheezy ? >> >> I have seen some interesting performance differences between the >> versions of DBIx shipped with these Debian versions. >> >> Jessie is much faster. >> >> >> Marcel >> >> ------------------------------------------------------------------------ >> *Van:* koha-devel-bounces at lists.koha-community.org >> >> >> namens >> Philippe Blouin >> >> *Verzonden:* maandag 1 augustus 2016 22:30:14 >> *Aan:* koha-devel at lists.koha-community.org >> >> *Onderwerp:* [Koha-devel] How long is a call to schema() supposed >> to take >> Hello Koha developpers! >> >> I got a question to all the dbix experts out there (I'm not one >> of them, unfortunatly). I have performance issues in the >> checkout that remains even after boosting my server's performance >> with a SSD drive. >> >> In C4/Circulation.pm/*AddIssue*, a lot of transactions take >1s >> for the call to schema $issue = >> >> Koha::Database->new()->_*schema()*_->resultset('Issue')->create( >> >> This is *3.20*. I specifically isolated that call with timers, I >> do a bunch of checkouts for the same user, and repetitively, that >> call take over 1s. The machine is a beast (ram, cpu...), so what >> should I be doing? >> >> Thanks, >> -- >> Philippe Blouin, >> Responsable du d?veloppement informatique >> >> T?l. : (888) 604-2627 >> philippe.blouin at inLibro.com >> >> inLibro | pour esprit libre | >> www.inLibro.com > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > > -- > Tom?s Cohen Arazi > Theke Solutions (https://theke.io ) > ? +54 9351 3513384 > GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomascohen at gmail.com Tue Aug 2 15:40:02 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Tue, 02 Aug 2016 13:40:02 +0000 Subject: [Koha-devel] How long is a call to schema() supposed to take In-Reply-To: <57A0A1DA.6050408@inlibro.com> References: <579FB156.7060808@inlibro.com> <57A09AB1.7060902@inlibro.com> <57A0A1DA.6050408@inlibro.com> Message-ID: The Plack process can be running without being used by Apache. And some virtual hosts can point to the Plack backend, and some others not. That's not a problem. I'm not familiar to plackup.sh, (at misc/plack) but I think that's what you need to use in your scenario. If I had to choose, I'd go with the packages, building your own from your own branch. That's what I do. Regards El mar., 2 ago. 2016 a las 10:36, Philippe Blouin (< philippe.blouin at inlibro.com>) escribi?: > haaaa, > > No, got git setups. Any instructions on that? two lines maybe? one? :-) > Was wondering actually if having plack for one install on 16.05 would > break another installation on 3.20 sharing apache. Most things Apache are > secluded by virtual host, but some are global. I live in fear... > > > Philippe Blouin, > Responsable du d?veloppement informatique > > T?l. : (888) 604-2627 > philippe.blouin at inLibro.com > inLibro | pour esprit libre | www.inLibro.com > On 08/02/2016 09:27 AM, Tomas Cohen Arazi wrote: > > It is 3 lines :-D > > And is your instance name, provided you have a packages-based > setup, with created instances. > > > El mar., 2 ago. 2016 a las 10:06, Philippe Blouin (< > philippe.blouin at inlibro.com>) escribi?: > >> This is Jessie. That's maybe part of the issue, though. While changing >> hardware, we also upgraded the OS/apache/mysql. I was kinda hoping for >> some big gain, but this 1sec delay really stands out. >> >> But following everyone's comments, ll look into 14478, plack... and >> simply moving everything to 16.05 :) >> >> Any good doc on plack? I got https://wiki.koha-community.org/wiki/Plack, >> but 5 lines aren't much, not even a clue what is :) >> >> Thanks everyone! >> >> >> >> Philippe Blouin, >> Responsable du d?veloppement informatique >> >> T?l. : (888) 604-2627 >> philippe.blouin at inLibro.com >> inLibro | pour esprit libre | www.inLibro.com >> On 08/02/2016 05:30 AM, Marcel de Rooy wrote: >> >> Philippe, >> >> Are you using Jessie or Wheezy ? >> >> I have seen some interesting performance differences between the versions >> of DBIx shipped with these Debian versions. >> >> Jessie is much faster. >> >> >> Marcel >> ------------------------------ >> *Van:* koha-devel-bounces at lists.koha-community.org >> >> namens Philippe Blouin >> >> *Verzonden:* maandag 1 augustus 2016 22:30:14 >> *Aan:* koha-devel at lists.koha-community.org >> *Onderwerp:* [Koha-devel] How long is a call to schema() supposed to take >> >> Hello Koha developpers! >> >> I got a question to all the dbix experts out there (I'm not one of them, >> unfortunatly). I have performance issues in the checkout that remains even >> after boosting my server's performance with a SSD drive. >> >> In C4/Circulation.pm/*AddIssue*, a lot of transactions take >1s for the >> call to schema $issue = >> >> Koha::Database->new()->*schema()*->resultset('Issue')->create( >> >> This is *3.20*. I specifically isolated that call with timers, I do a >> bunch of checkouts for the same user, and repetitively, that call take over >> 1s. The machine is a beast (ram, cpu...), so what should I be doing? >> >> Thanks, >> -- >> Philippe Blouin, >> Responsable du d?veloppement informatique >> >> T?l. : (888) 604-2627 >> philippe.blouin at inLibro.com >> inLibro | pour esprit libre | www.inLibro.com >> >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ > > -- > Tom?s Cohen Arazi > Theke Solutions (https://theke.io ) > ? +54 9351 3513384 > GPG: B2F3C15F > > > -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From philippe.blouin at inlibro.com Tue Aug 2 15:59:40 2016 From: philippe.blouin at inlibro.com (Philippe Blouin) Date: Tue, 2 Aug 2016 09:59:40 -0400 Subject: [Koha-devel] Pre-RFC on new discrete calendar In-Reply-To: <60f001c4-5c84-57b4-0a39-fb81c3a044c3@web.de> References: <5790FBB6.8020009@inlibro.com> <579606CB.9030201@inlibro.com> <60f001c4-5c84-57b4-0a39-fb81c3a044c3@web.de> Message-ID: <57A0A74C.3090904@inlibro.com> Hi Katrin! To cut to the chase: yes, no rule at this moment. So you'd lose what they bring you right now in term of saving time. They could be added, easily, but doing so I would go for rules that apply some localised calendar by default, and allows to handle things like Easter, Memorial day, Thanksgiving and other floating holidays that represent 2/3 of of holidays here and aren't covered with the current rules and need editing. The fixed holidays are covered with 17015 since the days created in the future consider the week and the year before, looking for holidays. So I do not believe this way of handling things is a real minus versus the old one. (And we still have some ideas to make it smarter before considering a rule system. ) As for the days in the future, and regular maintenance, I don't have any case here where documents have a due date two years in the future. And our UI makes it extremely easy to modify swat of days in one click (and some input). Cheers! Philippe Blouin, Responsable du d?veloppement informatique T?l. : (888) 604-2627 philippe.blouin at inLibro.com inLibro | pour esprit libre | www.inLibro.com On 08/01/2016 04:57 PM, Katrin Fischer wrote: > Hi Philippe, > > I still got some question marks about this idea... mostly it doesn't > feel quite right so far. >> >> Let's start with the backend: this is just a simple script that fills >> the calendar ahead of time (waaay ahead of time) to allow the user to >> modify schedules easily. 1 year is just a hardcoded value, it could >> as well be an argument allowing any number of days in the future. >> Same for the past: during updatedatabase, this creates one but could >> create two years in the past based on the info in the current >> calendar tables. In the end, the point is to have a table entry for >> each day, specifying which day it WAS opened (for hourly fine >> calculation) and which day it will be opened (for hourly checkout or >> just for displaying the library open hours on the OPAC). Going a >> year in the future, things are a bit dumb when creating, but always >> replicate the week before, except for items with notes (holiday) that >> are fetched on the calendar year a year before. Then the librarian >> has plenty of time to adjust anything. > I think depending on the notes is not a good idea. You can add notes > to every kind of holiday at the moment - so you'd have to take a look > at the type of holiday - repeated yearly/weekly and unique. You'd also > need to take into account the exceptions to repeated holidays that you > can currently define. >> >> Which brings us to the UI. Mine is ugly, but it would be easy to >> create a nice one AND SIMPLE one (coding) and powerful one (for >> users). With that simple backend, it's very easy to simply allow >> multiple selections in the calendar widget, the modify opening hours, >> or holiday close with a note. Or better: select a week anywhere in >> the calendar, then copy that to a given range. 3 days, a month, 3 >> months... Very simple in the UI, very few clicks. Very simple to >> code in the backend. > It sounds like this type of calendar would need more regular > maintenance than the current system? >> >> So in the end, recovering the original work or Kyle's work or >> defining new standard, we have a calendar page with a simple calendar >> and below it a few edit box (opening hours, closing, notes, closed >> checkbox) and a apply button. On the right, like right now, we can >> display all "special dates", which are the ones with a "note" entry. >> In yellow those that are on days still opened, in pink those days >> that are marked as closed. Of course, all UI schemes are very open >> to suggestion. But it would be simple and naturally intuitive. > > If I was to make a change to the calendar - would I have to wait for > the cronjob/script to run and update the table or would it take effect > immediately? > > What will happen if a due date or other date is outside of the > calendar? I feel like by relying on a fixed date range, we are going > to create problems along the road if there is no rule based system as > a backup at least. > > Katrin > >> >> Philippe Blouin, >> Responsable du d?veloppement informatique >> >> T?l. : (888) 604-2627 >> philippe.blouin at inLibro.com >> >> inLibro | pour esprit libre | www.inLibro.com >> On 07/24/2016 04:42 PM, Katrin Fischer wrote: >>> Hi Philippe, >>> >>> thx for trying to get things moving again - I know there are quite a lot >>> calendar related bugs to be found in bugzilla. >>> >>> Can you explain a bit about how this would change the GUI for the users? >>> Do you have to keep it up to date or does the table get filled >>> automatically for recurring events? >>> >>> I am a bit concerned about the limitation of one year into the past and >>> one year into the future. What happens if a due date goes beyond that or >>> an item is overdue before that? >>> >>> Katrin >>> >>> Am 21.07.2016 um 18:43 schrieb Philippe Blouin: >>>> Hi! >>>> >>>> I'm throwing a line here, and I'd just like to get a feel for the value >>>> of offering some work to the community. Mind you, the work is "big" so >>>> honest responses could save us lot of wasted hours. >>>> >>>> We've developed a parallel calendar table to specify each individual day >>>> if it's opened or not (instead of rules and exception). We added to it >>>> the opening hours, and keep a year of them in the past, and a year in >>>> the future. >>>> The reasonning being: >>>> - We need the opening hours. They need to vary season to seasons. We >>>> need them for hourly and minute loans. >>>> - Exception and holidays and etc... are complicated. To manage, to >>>> calculate, to fix. We need the past info as well, to calculate precisely. >>>> - Performance. Calculating with C4/Koha Calendars is sloooooooooow. >>>> Our little table cut fines.pl calculation times by 97%. Not a typo. >>>> Checkout improvement by 30-60% but metric is unreliable so take with >>>> grain of salt this one. >>>> >>>> So before I go and write a wiki RFC, then open bugzillas, make the code >>>> community acceptable (we're not using Schemas), complete it, write >>>> tests, etc... Is there an interest? Would it answer a need (outside of >>>> our clients) ? Maybe a subset? >>>> >>>> All comments, suggestions, questions are welcomed. >>>> >>>> High regards, >>>> >>>> Philippe Blouin, >>>> Responsable du d?veloppement informatique >>>> >>>> T?l. : (888) 604-2627 >>>> philippe.blouin at inLibro.com >>>> >>>> inLibro | pour esprit libre |www.inLibro.com >>>> >>>> >>>> >>>> _______________________________________________ >>>> Koha-devel mailing list >>>> Koha-devel at lists.koha-community.org >>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>>> website :http://www.koha-community.org/ >>>> git :http://git.koha-community.org/ >>>> bugs :http://bugs.koha-community.org/ >>>> >>> _______________________________________________ >>> Koha-devel mailing list >>> Koha-devel at lists.koha-community.org >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>> website :http://www.koha-community.org/ >>> git :http://git.koha-community.org/ >>> bugs :http://bugs.koha-community.org/ >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From M.de.Rooy at rijksmuseum.nl Tue Aug 2 17:02:18 2016 From: M.de.Rooy at rijksmuseum.nl (Marcel de Rooy) Date: Tue, 2 Aug 2016 15:02:18 +0000 Subject: [Koha-devel] [Koha] search quits working In-Reply-To: <57A0B277.60406@inlibro.com> References: <5.2.1.1.2.20160720111112.05c34e78@pop.navalmarinearchive.com> <2bdd94e5-7e00-e441-5af0-eb98c63413ac@yahoo.es> , <57A0B277.60406@inlibro.com> Message-ID: Adding this comment to bug 16885. Thanks. ________________________________ Van: Koha namens Philippe Blouin Verzonden: dinsdag 2 augustus 2016 16:47 Aan: Scott Owen; Nombre CC: koha Onderwerp: Re: [Koha] search quits working (sorry, jumping into the thread) We've had this problem on 10% of our installations. I solved it by going nuclear with logrotate since that was really hurting the users to not have zebra working (it would appear like working on our monitor, but was actually just dying and restarting). Here is the postrotate I've end up with, after many iterations of Next Level Apocalypse You :) postrotate service apache2 reload find /etc/init.d/ -name "*zebra-daemon" -exec bash -c 'service "$(basename "$0")" stop' {} \; sleep 3 find /etc/init.d/ -name "*zebra-daemon" -exec bash -c 'service "$(basename "$0")" stop' {} \; sleep 3 pkill -9 zebra sleep 2 pkill -9 zebra sleep 2 find /etc/init.d/ -name "*zebra-daemon" -exec bash -c 'service "$(basename "$0")" start' {} \; endscript As you see, no subtlety, but I've had no problem for a year now... Philippe Blouin, Responsable du d?veloppement informatique T?l. : (888) 604-2627 philippe.blouin at inLibro.com inLibro | pour esprit libre | www.inLibro.com On 08/02/2016 10:08 AM, Scott Owen wrote: > Hi, > > I'm finally getting back to my Koha issues..... > > > "I'm going to test adding a line > > /bin/sleep 5 > > as first command in postrotate in logrotate.d/koha-common. I'll see > tomorrow." > > Nombre, did this work for you ?? > > or is the issue maybe fixed in the latest release ?? > > > -S > > > > > > > On Fri, Jul 22, 2016 at 7:52 AM, Nombre wrote: > >> El 21/07/16 a las 14:55, Marcel de Rooy escribi?: >>> Bug 16885 ? >> I'm going to test adding a line >> >> /bin/sleep 5 >> >> as first command in postrotate in logrotate.d/koha-common. I'll see >> tomorrow. >> >> Regards >> >>> -----Oorspronkelijk bericht----- >>> Van: Koha [mailto:koha-bounces at lists.katipo.co.nz] Namens Tomas Cohen >> Arazi >>> Verzonden: donderdag 21 juli 2016 14:27 >>> Aan: Scott Owen ; Paul A < >> paul.a at navalmarinearchive.com> >>> CC: koha >>> Onderwerp: Re: [Koha] search quits working >>> >>> I blame logrotate. >>> >>> El jue., 21 jul. 2016 a las 9:12, Scott Owen () >> escribi?: >>>> Sorry.... >>>> Debian, latest version of Koha. >>>> >>>> problem *seems* to have only started when I updated to the latest Koha >>>> version. >>>> >>>> >>>> 3.22.06.000 >>>> Linux hskoha 3.16.0-4-686-pae #1 SMP Debian 3.16.7-ckt25-2 >>>> (2016-04-08) >>>> i686 >>>> >>>> >>>> The issue seems to be with the zebra server somehow coming to a halt, >>>> and bailing.....seemingly @ approx. 7:35 daily. >>>> >>>> a quick command of " koha-start-zebra " makes things >>>> all-good.....and I can search again. >>>> >>>> However, within a couple of days....the zebra server appears to die >> again. >>>> I have two servers, both at exactly the same rev. and they are both >>>> exhibiting this behavior. Starting to look at some of the cron.daily >>>> commands.....they seem the most likely suspect (in my eyes) >>>> >>>> >>>> >>>> cron.daily -->> >>>> >>>> #!/bin/sh >>>> # /etc/cron.daily/koha-common -- Daily housekeeping tasks for all Kohas. >>>> # Copyright 2010 Catalyst IT, Ltd >>>> # >>>> # This program is free software: you can redistribute it and/or modify >>>> # it under the terms of the GNU General Public License as published by >>>> # the Free Software Foundation, either version 3 of the License, or # >>>> (at your option) any later version. >>>> # >>>> # This program is distributed in the hope that it will be useful, # >>>> but WITHOUT ANY WARRANTY; without even the implied warranty of # >>>> MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU >>>> General Public License for more details. >>>> # >>>> # You should have received a copy of the GNU General Public License # >>>> along with this program. If not, see . >>>> >>>> koha-foreach --enabled --email /usr/share/koha/bin/cronjobs/ >>>> overdue_notices.pl -t koha-foreach --enabled >>>> /usr/share/koha/bin/cronjobs/fines.pl >>>> koha-foreach --enabled --email /usr/share/koha/bin/cronjobs/ >>>> advance_notices.pl -c koha-foreach --enabled >>>> /usr/share/koha/bin/cronjobs/membership_expiry.pl >>>> -c >>>> koha-foreach --enabled /usr/share/koha/bin/cronjobs/holds/ >>>> cancel_expired_holds.pl >/dev/null 2>&1 koha-foreach --enabled >>>> /usr/share/koha/bin/cronjobs/services_throttle.pl > /dev/null 2>&1 >>>> koha-foreach --enabled >>>> /usr/share/koha/bin/cronjobs/cleanup_database.pl --sessions >>>> --zebraqueue 10 --list-invites koha-foreach --enabled --noemail >>>> /usr/share/koha/bin/cronjobs/ cleanup_database.pl --mail koha-foreach >>>> --enabled /usr/share/koha/bin/cronjobs/holds/ >>>> auto_unsuspend_holds.pl > /dev/null 2>&1 koha-foreach --enabled >>>> /usr/share/koha/bin/cronjobs/automatic_renewals.pl >>>> koha-run-backups --days 2 --output /var/spool/koha >>>> >>>> >>>> >>>> >>>> On Wed, Jul 20, 2016 at 6:41 PM, Paul A >>>> >>>> wrote: >>>> >>>>> At 10:20 AM 7/20/2016 -0400, Scott Owen wrote: >>>>> >>>>>> Hi all, >>>>>> I'm having some issues with Koha searching from both the OPAC and >>>>>> the staff Intranet. >>>>>> I have tried several solutions for reindexing etc... >>>>>> including >>>>>> ---Koha-rebuild-zebra -f -v -b mylibrary and ---export >>>>>> PERL5LIB=/usr/share/koha/lib ---export >>>>>> KOHA_CONF=/usr/share/koha/koha-conf.xml >>>>>> ---perl /usr/share/koha/bin/migration_tools/rebuild_zebra.pl -b -r >>>>>> -v -run-as-root >>>>>> >>>>> You don't say which version of Koha, nor what o/s you are using. >>>>> Assuming Debian/Ubuntu, what does printenv give you? You should see >>>>> something like (amongst other lines): >>>>> >>>>> me at my_server:/$ printenv >>>>> PERL5LIB=/usr/share/koha/lib >>>>> >>>>> >>>> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr >>>> /local/lib/perl >>>>> KOHA_CONF=/usr/share/koha/koha-conf.xml >>>>> KOHA_PATH=/usr/share/koha >>>>> >>>>> I'm also a little surprised by your "-run-as-root" -- in slightly >>>>> older versions of Koha, this was a "no-no", but may well have >>>>> changed >>>> (mpm-itk?) >>>>> There is also something I noticed in further emails: >>>>> >>>>>> your zebra error logs. >>>>> [snip] >>>>>> 20160713 07:36:11 dlpms-koha-zebra: client (pid 526) killed by >>>>>> signal >>>>> 15, stopping >>>>>> 20160713 07:40:07 dlpms-koha-zebra: client (pid 10116) killed by >>>>>> signal >>>>> 15, stopping >>>>> >>>>> What were the 9590 processes in the <4 minutes? That log entry would >>>>> certainly get my attention. >>>>> >>>>> Best -- P. >>>>> >>>>> >>>>> none of these commands produce any errors, however, they also do not >>>>> fix >>>>>> the issue. >>>>>> >>>>>> >>>>>> A reboot of the VM will cause the "search" to work again for a day >>>>>> or two....?? >>>>>> (I know....."it shouldn't make a difference" ...but it >>>>>> does...!?!?!?) >>>>>> >>>>>> >>>>>> I'm unsure of exactly where to start troubleshooting ??? >>>>>> >>>>>> Any advise on where to begin ?? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> -S >>>>>> _______________________________________________ >>>>>> Koha mailing list http://koha-community.org >>>>>> Koha at lists.katipo.co.nz >>>>>> https://lists.katipo.co.nz/mailman/listinfo/koha >>>>>> >>>>> --- >>>>> Maritime heritage and history, preservation and conservation, >>>>> research and education through the written word and the arts. >>>>> and >>>>> >>>>> >>>>> _______________________________________________ >>>>> Koha mailing list http://koha-community.org Koha at lists.katipo.co.nz >>>>> https://lists.katipo.co.nz/mailman/listinfo/koha >>>>> >>>> _______________________________________________ >>>> Koha mailing list http://koha-community.org Koha at lists.katipo.co.nz >>>> https://lists.katipo.co.nz/mailman/listinfo/koha >>>> >>> -- >>> Tom?s Cohen Arazi >>> Theke Solutions (https://theke.io ) ? +54 9351 3513384 >>> GPG: B2F3C15F >>> _______________________________________________ >>> Koha mailing list http://koha-community.org Koha at lists.katipo.co.nz >> https://lists.katipo.co.nz/mailman/listinfo/koha >>> _______________________________________________ >>> Koha mailing list http://koha-community.org >>> Koha at lists.katipo.co.nz >>> https://lists.katipo.co.nz/mailman/listinfo/koha >>> >> _______________________________________________ >> Koha mailing list http://koha-community.org >> Koha at lists.katipo.co.nz >> https://lists.katipo.co.nz/mailman/listinfo/koha >> > _______________________________________________ > Koha mailing list http://koha-community.org > Koha at lists.katipo.co.nz > https://lists.katipo.co.nz/mailman/listinfo/koha _______________________________________________ Koha mailing list http://koha-community.org Koha at lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha -------------- next part -------------- An HTML attachment was scrubbed... URL: From Katrin.Fischer.83 at web.de Tue Aug 2 22:16:58 2016 From: Katrin.Fischer.83 at web.de (Katrin Fischer) Date: Tue, 2 Aug 2016 22:16:58 +0200 Subject: [Koha-devel] Reminder: General IRC meeting 3 August 2016 - 10 UTC Message-ID: <682419ed-7073-c543-9e38-83b3ea1f9592@web.de> Hi all, there will be a General Koha IRC meeting in about 14 hours. As always the agenda and a link to find out your local meeting time can be found on the wiki: https://wiki.koha-community.org/wiki/General_IRC_meeting_3_August_2016 Hope to see you there! Katrin From mtompset at hotmail.com Wed Aug 3 06:25:52 2016 From: mtompset at hotmail.com (Mark Tompsett) Date: Wed, 3 Aug 2016 04:25:52 +0000 Subject: [Koha-devel] General IRC Meeting 3 August 2016 / Kohacon 2017 bidder information In-Reply-To: References: Message-ID: Greetings, It has come to my attention that the National Library of the Philippines is still under renovation, and thus the Philippines is withdrawing its bid. Sorry. GPML, Mark Tompsett -----Original Message----- From: Chris Cormack Sent: Monday, August 01, 2016 5:13 PM To: koha Cc: koha-devel at lists.koha-community.org Subject: Re: [Koha-devel] General IRC Meeting 3 August 2016 / Kohacon 2017 bidder information Hi All I just want to second Mirko's email about Kohacon17, at the moment we only have the one bid (For the Philippines) as Kohacon was in Europe this year and can't be in the same continent within 3 years. However there is a real lack of information, or contact with any people in Philippines about their bid. For example the hosting organisations are named as the National Library and Onstrike, but the community has had no contact with anyone from either organisation. So we really need either a mail to the list of them to show at the next meeting. Also I would heartily recommend any organisation outside of Europe or Africa (the last 2 continents) to propose also. It isn't too late, but we do need an actual contact so we can be sure things will happen. Thanks Chris On 29 July 2016 at 20:56, Mirko Tietgen wrote: > Hi everyone, > > the next general IRC meeting is next week, 3 August 2016, 10:00 UTC. > > Conversion to your local time: > http://www.timeanddate.com/worldclock/fixedtime.html?msg=Koha+IRC+General+Meeting&iso=20160803T10 > > You can find the agenda here: > https://wiki.koha-community.org/wiki/General_IRC_meeting_3_August_2016 > > This is the last meeting before we close bidding to host Kohacon > 2017. It would be nice if bidders would join us at the meeting and > tell a little about their plans. > > If the meeting is not an option, please send a message to the > mailing list explaining that you are still interested in hosting > Kohacon 2017, and please update your proposals in the Koha wiki now > if necessary: Narrow down the proposed dates if possible, update > conference locations, add contact emails(!). > https://wiki.koha-community.org/wiki/KohaCon17_Proposals > > We are still open for proposals to host Kohacon 2017 until 10 August > 2016 UTC. > > Cheers, > > Mirko > > > -- > > Mirko Tietgen > mirko at abunchofthings.net > http://koha.abunchofthings.net > http://meinkoha.de > > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ _______________________________________________ Koha-devel mailing list Koha-devel at lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ From bargioni at pusc.it Wed Aug 3 11:08:14 2016 From: bargioni at pusc.it (Stefano Bargioni) Date: Wed, 3 Aug 2016 11:08:14 +0200 Subject: [Koha-devel] General IRC Meeting 3 August 2016 / Kohacon 2017 bidder information In-Reply-To: References: Message-ID: <9700FD03-6E28-464F-86E4-ADE12A89E403@pusc.it> +1 > On 02 ago 2016, at 11:27, Marcel de Rooy wrote: > > > as Kohacon was in Europe this year and can't be in the same continent within 3 years. > > Little bit a side note: > As I understand, this is some kind of unwritten rule in the community. > Maybe we should write it down somewhere and relax it a little bit? > > KohaCon should preferably not be held on the same continent twice in a row (unless an alternative is not available). > > Or something similar ?? > > Marcel > > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at bigballofwax.co.nz Wed Aug 3 11:30:20 2016 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Wed, 3 Aug 2016 21:30:20 +1200 Subject: [Koha-devel] General IRC Meeting 3 August 2016 / Kohacon 2017 bidder information In-Reply-To: References: Message-ID: We actually wrote it down in November of 2010, and it has been in place for every Kohacon since then. It is a safe guard that means that the conference isn't continuously in the same country. I think it has worked spectacularly well, and meant that smaller countries have a good chance of hosting. I don't think it needs to be relaxed to twice in a row, but if there are no other bidders then I think an exception could be made in that case. Chris (Who is grateful that Kohacon has meant I have gotten to places like Greece, Nigeria, Argentina, etc) On 2 August 2016 at 21:27, Marcel de Rooy wrote: >> as Kohacon was in Europe this year and can't be in the same continent >> within 3 years. > > > Little bit a side note: > > As I understand, this is some kind of unwritten rule in the community. > > Maybe we should write it down somewhere and relax it a little bit? > > > KohaCon should preferably not be held on the same continent twice in a row > (unless an alternative is not available). > > > Or something similar ?? > > Marcel > > > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From chris at bigballofwax.co.nz Wed Aug 3 11:39:37 2016 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Wed, 3 Aug 2016 21:39:37 +1200 Subject: [Koha-devel] [Koha] General IRC Meeting 3 August 2016 / Kohacon 2017 bidder information In-Reply-To: References: Message-ID: On 3 August 2016 at 21:36, Joann Ransom wrote: > Is it too soon to bid KohaCon 2019 for NZ ? > We had 2010, how about 2020 to keep it round numbers :) Chris From fridolin.somers at biblibre.com Wed Aug 3 12:13:06 2016 From: fridolin.somers at biblibre.com (Fridolin SOMERS) Date: Wed, 3 Aug 2016 12:13:06 +0200 Subject: [Koha-devel] Biblibre sandboxes out Message-ID: <774ac1de-993c-2f6c-ba88-2dbe437c54d2@biblibre.com> Hie, Our sandboxes are broken (lets say on holiday) : https://wiki.koha-community.org/wiki/Sandboxes#BibLibre This is because there is a new dependency only existing since Jessie, and those machines can not upgrade. We are migrating all our servers to Ubuntu Xenial. We will tell you when sandboxes are done. Best regards, -- Fridolin SOMERS Biblibre - P?les support et syst?me fridolin.somers at biblibre.com From arthur.suzuki at univ-lyon3.fr Wed Aug 3 12:30:37 2016 From: arthur.suzuki at univ-lyon3.fr (SUZUKI Arthur) Date: Wed, 03 Aug 2016 12:30:37 +0200 Subject: [Koha-devel] Regarding the implementation of recommender system in koha In-Reply-To: References: Message-ID: <125BBCA1-BB7C-4672-88D6-0DB10BBEBED5@univ-lyon3.fr> Hello, I've been working on something of this kind since last Koha Symposium in Lyon were I had the chance to meet some pretty advanced users and devs. The current code of my project is available on my Github repo among other things : http://github.com/liliputech Feel free to ask any question, i'm in vacantion now but will try to answer as soon as possible. You may find specific interest in the SQL query in "readsuggestion.pm" file. Sincerely yours, Arthur Suzuki University Jean Moulins Lyon 3 Library bu.univ-lyon3.fr Le 29 juillet 2016 07:52:36 UTC+02:00, ADITYA RAJ <201451032 at iiitvadodara.ac.in> a ?crit : >Sir,In my college I am working on recommender system project and I want >to >implement it in koha search so I need some help >Hoping positive response from you > > >------------------------------------------------------------------------ > >_______________________________________________ >Koha-devel mailing list >Koha-devel at lists.koha-community.org >http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >website : http://www.koha-community.org/ >git : http://git.koha-community.org/ >bugs : http://bugs.koha-community.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at bigballofwax.co.nz Wed Aug 3 22:09:38 2016 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Thu, 4 Aug 2016 08:09:38 +1200 Subject: [Koha-devel] [Koha] General IRC Meeting 3 August 2016 / Kohacon 2017 bidder information In-Reply-To: References: Message-ID: Hi All At the meeting last night we clarified the state of the bid from the Philippines, and the wiki has been updated to reflect it https://wiki.koha-community.org/wiki/KohaCon17_Proposals You can read the mintues of the meeting linked from here https://wiki.koha-community.org/wiki/General_IRC_meeting_3_August_2016#Minutes Chris From mirko at abunchofthings.net Mon Aug 8 08:30:20 2016 From: mirko at abunchofthings.net (Mirko Tietgen) Date: Mon, 8 Aug 2016 08:30:20 +0200 Subject: [Koha-devel] Introducing Koha nightly packages Message-ID: Hello dev people, I have set up a build server and repository for automatic Koha nightly builds (Debian packages) of master, 16.05.x and 3.22.x. It also serves a live installation running the latest master build, unless it is broken. You can find it at https://apt.abunchofthings.net/koha-nightly/ Friendly word of advice: Kids, do not use this in production. It's just a development toy. Cheers, Mirko -- Mirko Tietgen mirko at abunchofthings.net http://koha.abunchofthings.net http://meinkoha.de -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From M.de.Rooy at rijksmuseum.nl Mon Aug 8 10:15:43 2016 From: M.de.Rooy at rijksmuseum.nl (Marcel de Rooy) Date: Mon, 8 Aug 2016 08:15:43 +0000 Subject: [Koha-devel] Signoff request: bug 16929 Message-ID: Hi all, Just rebased bug 16929. Two pending security fixes depend on this report. Not hard to test. I already verified that it resolves a possible delay on some virtual servers. If you do not experience that delay, your signoff still serves to assure that the script is working properly. Please add your signoff to get this further. Thanks, Marcel PS As a sidenote, there is still another problem with opac-memberentry. It is addressed on bug 16960. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomascohen at gmail.com Mon Aug 8 14:47:27 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Mon, 08 Aug 2016 12:47:27 +0000 Subject: [Koha-devel] Introducing Koha nightly packages In-Reply-To: References: Message-ID: Awesome, Mirko! El lun., 8 ago. 2016 3:30, Mirko Tietgen escribi?: > Hello dev people, > > I have set up a build server and repository for automatic Koha > nightly builds (Debian packages) of master, 16.05.x and 3.22.x. It > also serves a live installation running the latest master build, > unless it is broken. > > You can find it at https://apt.abunchofthings.net/koha-nightly/ > > Friendly word of advice: Kids, do not use this in production. It's > just a development toy. > > Cheers, > > Mirko > > > -- > > Mirko Tietgen > mirko at abunchofthings.net > http://koha.abunchofthings.net > http://meinkoha.de > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From Katrin.Fischer.83 at web.de Tue Aug 9 23:32:53 2016 From: Katrin.Fischer.83 at web.de (Katrin Fischer) Date: Tue, 9 Aug 2016 23:32:53 +0200 Subject: [Koha-devel] Reminder: Koha Development IRC meeting - 19 UTC Message-ID: Hi all, there is a Development IRC meeting scheduled for 10 August at 19 UTC. The agenda can as always be found on the wiki: https://wiki.koha-community.org/wiki/Development_IRC_meeting_10_August_2016 Please add the topics you want to discuss to the agenda! Hope to see you there, Katrin From jonathan.druart at bugs.koha-community.org Thu Aug 11 09:36:02 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Thu, 11 Aug 2016 08:36:02 +0100 Subject: [Koha-devel] Koha namespace architecture - calls for discussions In-Reply-To: References: Message-ID: For the record: 2/ is on bug 16966 - Koha::Patrons - Move GetBorrowersWithIssuesHistoryOlderThan to search_patrons_to_anonymise and 3/ is on bug 17089 - Move the star ratings related code to Koha::Ratings 2016-07-21 12:20 GMT+01:00 Jonathan Druart : > Hi devs (me again), > > During the last dev meeting, we discussed about the naming of a new > module in our Koha namespace. The conclusion was to ask the ML in > order to get more ideas (as there was not a lot of attendees...). > I'd like to take this opportunity to bring other architecture problems > I faced recently (Bug 16846 - Move patron related code to > Koha::Patron). > > 1/ Import stuffs modules > The initial discussion comes from 2 modules which have been added in 2 > different bug reports about the same thought [1]. > To provide a module to import patron, the author of bug 12598 chose > Koha::Patrons::Import when I chose Koha::Exporter::Record on bug 14722 > (the first one is about importing patrons, the second one is about > exporting records (biblios, authorities)). > We need to uniformise these 2 namespaces to keep consistency. > The different possibilities are: > a. Koha::Patrons::Import > b. Koha::Importer::Patrons > c. Koha::Importer::Patron > d. Koha::Import::Patrons > e. Koha::Import::Patron > > My opinion is that having Koha::Exporter|Importer::Objects will add > the ability to have a Koha::Exporter|Importer module to group > export/import methods, if we want to (Objects means Patrons, Records, > Lists(Shelves), Reports, etc.) > But we could imagine a generic Koha::Exporter|Importer modules with > specific Koha::Object[s?]::Export|Import modules. > > 2/ Move C4::Members::GetBorrowersWithIssuesHistoryOlderThan to the > Koha namespace > This subroutine is supposed to return the patrons with an issue > history older than a given date. > I first chose to move this code to > Koha::Patrons->search_with_issues_history_older_than > But looking at the code you will find that the subroutine actually > returns the borrowernumber with the number of old issues (matching the > "older than" date). > IMO it would make sense to return the patrons (Koha::Patron) instead > of just the borrowernumber, but we will have to join the 2 tables, > which can lead to a perf issue (ref needed). This idea is enforced by > the fact that GetBorrowersWhoHaveNeverBorrowed and > GetBorrowersToExpunge fit perfectly into Koha::Patrons. > Otherwise we could decide to move the subroutine to a method of > Koha::OldIssues, to avoid the join. But that sounds weird to me. > > 3/ Move C4::Ratings > I have started to move C4::Ratings to Koha::Rating[s] > Everything went fine until I decided to move C4::Rating::GetRating. It > actually calculates the ratings avg for a record and returns a hashref > with different values, and not a Koha::Rating object. > I think we need to split up with > Koha::Ratings->search({ biblionumber => $biblionumber > })->calculate_avg (or something similar) > and > Koha::Ratings->search({ biblionumber => $biblionumber, > borrowernumber => $borrowernumber }); # which will return a > Koha::Rating object > In that case, 1 call to C4::Ratings::GetRating will be replaced with 2 calls. > Does that make sense? > > 4/ Move subroutines not directly related to a Koha::Object > In some cases, the move is not very obvious. For instance we have > C4::Members::checkcardnumber and C4::Members::get_cardnumber_length, > they are strongly linked and I think we would like to keep them > together. > We could imagine a Koha::Patron::Cardnumber to move > get_cardnumber_length in, but it does not represent a patron's > cardnumber, so it could be confusing. > checkcardnumber could go easily to Koha::Patron->check_new_cardnumber( > $new_cardnumber ) > > 5/ patronflags? > Any ideas what we could do with C4::Members::patronflags? :) > > Others will certainly come later! > > Cheers, > Jonathan > > [1] http://meetings.koha-community.org/2016/development_irc_meeting_13_july_2016.2016-07-13-14.08.log.html > from 14:29:48 #info Topic: Koha::Patrons::Import (bug 12598) vs > Koha::Exporter::Record (bug 14722) From jonathan.druart at bugs.koha-community.org Thu Aug 11 09:39:56 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Thu, 11 Aug 2016 08:39:56 +0100 Subject: [Koha-devel] Koha::Object[s] improvements - call for discussions In-Reply-To: References: Message-ID: For the record: The AUTOLOAD is on bug 17091 (Add AUTOLOAD to Koha::Objects) and the DBIC vs Koha::Object returned value for Koha::Virtualshelf methods is fixed on bug 17094 (Methods of Koha::Object[s]-based classed should not return DBIx::Class objects) 2016-06-26 11:04 GMT+01:00 Jonathan Druart : > Hi devs, > > At the KohaCon16 some of us have briefly talked about the limitations > of Koha::Object[s] we encounter when implementing new modules. > Koha::Object[s] has been pushed to master less than 18 months ago and > we already have more than 30 classes using it. > It permits to uniformise our way to code, use powerful objects and is > very easy for refactoring. I have been moving code from C4 to the Koha > namespace and it's very handy to just have to write a concise module > and automatically have the basic methods (CRUD and more). > > However it seems that we need to clearly define what we need in a near > future and adapt it quickly if it's needed. > > = Rooms for Improvement = > There are 3 main rooms for improvement: > - DBIx::Class basic methods > - DBIx::Class relationship accesses > - Koha::Objects vs DBIx::Class objects > > == DBIx::Class basic methods == > If you want to access a method provided by DBIx::Class, like ->store, > ->delete, ->is_changed, we need to create a silly method which just > calls the same method on the DBIx::Class object. > For instance, let's take Koha::Objects->is_changed > sub is_changed { > my ( $self, @columns ) = @_; > return $self->_result()->is_changed(@columns); > } > > Possible solution: > It would be possible to modify our AUTOLOAD method and check if it's a > DBIX::Class method. > If it is a valid one, let's call it, otherwise carp something. > > == DBIx::Class relationships == > From a Koha::Objects, the relationships provided by DBIx::Class will > not be automatically generated. > For instance, from a virtual shelf, you will not have access to the > items on the shelf. > > You will have to create a method which will call the DBIx::Class > relevant relationship. > sub get_contents { > my ( $self ) = @_; > my $contents = $self->{_result}->virtualshelfcontents; > return $contents; > } > This piece of code adds a get_contents method to Koha::Virtualshelf > and call the virtualshelfcontents relationship created by DBIx::Class. > Note that this is finally good here, it is better to call have a > get_contents method than virtualshelfcontents. > But we may prefer to redefine it in the schema file > (Koha::Schema::Result::Virtualshelve). > > Possible solution: > Same as previously, it would be possible to modify our AUTOLOAD method > and check if it's a valid relationship. > > == Koha::Objects vs DBIX::Class objects == > We'd like to always manipulate the same kind of objects, to avoid > confusion when reading the code. > At the moment we already manipulate 2 different entities: hashref from > C4 and objects from Koha::Object. It's confusing and not so easy to > manage in the code. To add flexibility we have added a > Koha::Object->unblessed method to transform the object to an hashref, > it's not ideal but it ease the transition. > Anyway, from a Koha::Object if you want to retrieve data linked to > this object, you would like to get a Koha::Object object base, not a > DBIx::Class object base. > If we continue with the previous example, we will need to modify > get_contents like this: > sub get_contents { > my ( $self ) = @_; > my $rs = $self->{_result}->virtualshelfcontents; > my $contents = Koha::Virtualshelfcontents->_new_from_dbic( $rs ); > return $contents; > } > Basically ->new_from_dbic will just create an empty > Koha::Virtualshelfcontents object and set the internal resultset. No > query or extra processing should be done here. > > I do not know if the changes suggested (and already implemented in > some areas and patches in the queue) is consistent. > It's not very elegant to read, but it looks safe on the performance > side, despite the encapsulation problem we could see at first glance. > > = Concerns = > == API == > Some people raised concerns about the new API to lean. > But finally we stick on DBIx::Class method names, so it should not be > to hard to learn for new devs. > And... the modules in the Koha namespace are much more easier to > read/understand than the legacy code in C4. > > == ResultSet == > It seems that writing the following is not obvious for everybody: > # Create and store > my $biblio = Koha::Biblio->new( { title => "a title", [other => $data] } ); > $biblio->store; > > # Retrieve data > my $biblio = Koha::Biblios->find( $biblionumber ); > my $biblio = Koha::Biblios->search( { title => "a title" }); > > # Access relationship > my $items = $biblio->items; > warn ref( $items ); # Koha::Items > warn ref( $items->first ); # Koha::Item > > This looks very easy to read for me, feel free to suggest something else. > > = A complete example = > > At the moment, the more complete example in master (and 16.05) is the > Koha::Virtualshelf and Koha::Virtualshelves modules. It includes > several search methods, overwrite of the store method and some complex > methods as well as complete test coverage > (t/db_dependent/Virtualshelves.t) > > = Alternatives = > If you think you can provide an alternative or an idea to > improve/replace our Koha::Object[s] implementation, do not hesitate, I > would be glad to review or even implement it! > > If you want to discuss one of these point, add a new one, or have a > specific question, please answer this email. > We could talk about that during the next dev meeting as well. > > Cheers, > Jonathan From Holger.Meissner at hs-gesundheit.de Thu Aug 11 13:10:51 2016 From: Holger.Meissner at hs-gesundheit.de (Holger Meissner) Date: Thu, 11 Aug 2016 11:10:51 +0000 Subject: [Koha-devel] Git bz AttributeError: 'NoneType' object has no attribute 'group' Message-ID: <2C614FEEE2356A428DC65441E30302E0015B671956@nero> Hello koha devs, I was just about to upload a small patch to bugzilla. git bz attach 14434 HEAD But git bz says AttributeError: 'NoneType' object has no attribute 'group' Any ideas how to fix it? I successfully fetched and rebased the fishsoup branch, but that didn't help. Kind regards Holger From jonathan.druart at bugs.koha-community.org Thu Aug 11 14:09:19 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Thu, 11 Aug 2016 13:09:19 +0100 Subject: [Koha-devel] What's on in Koha devel #3 Message-ID: Hi devs, Lot of things have been going on during the last month and lot of patches are in the queue :) = Security issues = Plenty of security issues have been raised and fixed during the last month (and others are still coming). They are related to CSRF and XSS protections. See bug 14568 ([OMNIBUS] XSS in Staff Client) and bug 17096 ([OMIBUS] CSRF protections) for the entry points. Most of them are in the security project and most of you won't have read access, but if you are interested in testing them you can ask someone on #koha to have access. = Caching system = If you are familiar with plack and memcache, please have a look at the POC on bug 16579 (Use separate memcached namespace for caching koha-conf.xml). No need to test it if you don't want, you can just have a glance at it. = RESTful API = A lot of patches related to the RESTful API are in the SO and QA queues. The actors (authors/signoffers) are almost always the same. I'd like to know if someone (from the QA team or not) fancy to technically review these patches. I did not follow closely the discussions and the progress of the RESTful API and I think it could be good to have someone who's implied since the beginning in the QA process to have a look at how the architecture is done. I also would like to know who is willing to maintain this code? For instance on bug 16330 (REST API: add routes to add, update and delete patrons) the delete route is calling # check if loans, reservations, debarrment, etc. before deletion! my $res = $patron->delete So no check is done from the API. But if bug 16907 (Move DelMember and HandleDelBorrower to Koha::Patron) is pushed, $patron->delete will take care of the holds, etc. My question is simple: are we expecting from devs, signoffers and QAers to update and maintain these routes? Or a dedicated person will update them before each release? = Focus on blockers = When criticals or blockers are opened, usually it takes between 3 or 4 days to get a patch and see it pushed. Which is quite fast :) But most of the time, several bug reports are opened meanwhile and marked as duplicate later. So I think we should focus as much as possible on signing off these patches and 1/ ask on #koha if you find a software error and 2/ Search on the bug tracker in the most recent bugs. For instance bug 16960 (critical) has a patch waiting for a signoff for 3 days now... = Refactoring = Lot of things to say here :) == Accounts rewrite == I'd like to start QAing some parts of the accounts rewrite. For instance bug 15899 (Remove the use of recordpayment) is a good first candidate. But to do so I need all the dependencies to be signed off. == Koha::Patrons == Lot of things have been moved last month, the entry point is bug 16846 (Move patron related code to Koha::Patron) if you want to help or just have a look. == C4::Branch is (almost) dead == Bug 15758 (Move the C4::Branch related code to Koha::Libraries - part 4) removes completely C4::Branch. It is a big patch and has been widely tested. We need a quick but deep QA here, it's a nightmare to rebase! == Koha::AuthorisedValues == (I have c/p the following on bug 15799 - Move authorised values related code into Koha::AuthorisedValues - part 2, better to comment on the bug report) We have already started to move the authorised values related code to the Koha namespace, but some are still missing. To continue and move C4::Koha::*AuthorisedValues* subroutines to Koha::AuthorisedValues, we will need to join on marc_subfield_structure(.authorised_value which is actually an authorised value category). To do so we need to define a relationship (fk) between the two tables. But we have "special AV" like Asort1 (and friends) , LOST, DAMAGED, etc.) Which could not be in the authorised_values table [yet]. And 3 other "special special AV) like itemtypes, branches and cn_source. Others (undetermined so far) are linked to a marc_subfield_structure from installer/**/marcflavour/**/*framework*.sql, like STACK. My idea would be to move all the AV categories to a new table authorised_value_categories. It will contain only 1 column category_name refered by 1. authorised_values.category, 2. additional_fields.authorised_value_category (others authorised_value field should be renamed like that when they refer AV categories) and 3. iitems_search_fields.authorised_value_category. And, maybe later, auth_subfield_structure, auth_tag_structure, borrower_attribute_types, marc_tag_structure (some are varchar(10) et 20 when they should be 32!) We could imagine more attributes to this table, but don't ask me to add a authorised_value_categories.id as pk please :) We will have to use temporary tables to deal with the DB constraints, to add more fun. Finally the .sql will have to be updated to reflect these changes. I have just started but before to enter in this tunnel I'd like to know if there are a couple of volunteers to test? :) = Ubuntu 16.04/MySQL 5.7 support = Note that we are still waiting for bug 16554 to support Ubuntu 16.04 and MySQL 5.7 out of the box. A dev meeting is planned on August 24th at 7 UTC (https://wiki.koha-community.org/wiki/Development_IRC_meeting_24_August_2016). Please have a look at the agenda and the different topics we will discuss. Don't forget to register for the Hackfest in Marseille in October (http://lists.koha-community.org/pipermail/koha-devel/2016-June/042790.html) Cheers, Jonathan From oleonard at myacpl.org Fri Aug 12 17:34:16 2016 From: oleonard at myacpl.org (Owen Leonard) Date: Fri, 12 Aug 2016 11:34:16 -0400 Subject: [Koha-devel] Please help clean up bugs on unmaintained branches Message-ID: Recently I've been going through the list of bugs reports on unmaintained 3.x versions of Koha. I've closed bugs which I could not verify and updated to 'master' those which I could reproduce. The list is down to 45 bugs which I had questions about, didn't understand, or didn't know how to test. I would love to get help getting that list down to zero. Please take a look if you can. https://bugs.koha-community.org/bugzilla3/buglist.cgi?cmdtype=runnamed&namedcmd=3.x%20Unmaintained Thanks, Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org From oleonard at myacpl.org Fri Aug 12 17:42:32 2016 From: oleonard at myacpl.org (Owen Leonard) Date: Fri, 12 Aug 2016 11:42:32 -0400 Subject: [Koha-devel] Please help clean up bugs on unmaintained branches In-Reply-To: References: Message-ID: > I would love to get help getting that list down to zero. Please take a > look if you can. Sorry for the bad link. I guess I don't know how to share a saved search correctly. Here's a giant link: https://bugs.koha-community.org/bugzilla3/buglist.cgi?bug_status=UNCONFIRMED&bug_status=NEW&bug_status=REOPENED&bug_status=ASSIGNED&bug_status=In%20Discussion&bug_status=Needs%20Signoff&bug_status=Signed%20Off&bug_status=Passed%20QA&bug_status=Pushed%20for%20QA&bug_status=Failed%20QA&bug_status=Patch%20doesn%27t%20apply&list_id=173200&query_format=advanced&version=3.4&version=3.6&version=3.8&version=3.10&version=3.12&version=3.14&version=3.16&version=3.18 -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org From tomascohen at gmail.com Sat Aug 13 00:35:18 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Fri, 12 Aug 2016 22:35:18 +0000 Subject: [Koha-devel] What's on in Koha devel #3 In-Reply-To: References: Message-ID: Jonathan, nice overview! El jue., 11 ago. 2016 a las 9:09, Jonathan Druart (< jonathan.druart at bugs.koha-community.org>) escribi?: > Hi devs, > > Lot of things have been going on during the last month and lot of patches > are in the queue :) > > = Caching system = > If you are familiar with plack and memcache, please have a look at the > POC on bug 16579 (Use separate memcached namespace for caching > koha-conf.xml). No need to test it if you don't want, you can just > have a glance at it. > I'll try to! > = RESTful API = > A lot of patches related to the RESTful API are in the SO and QA queues. > The actors (authors/signoffers) are almost always the same. > I'd like to know if someone (from the QA team or not) fancy to > technically review these patches. > I did not follow closely the discussions and the progress of the > RESTful API and I think it could be good to have someone who's implied > since > the beginning in the QA process to have a look at how the architecture > is done. > I also would like to know who is willing to maintain this code? For > instance on bug 16330 (REST API: add routes to add, update and delete > patrons) the delete route is calling > # check if loans, reservations, debarrment, etc. before deletion! > my $res = $patron->delete > So no check is done from the API. > But if bug 16907 (Move DelMember and HandleDelBorrower to > Koha::Patron) is pushed, $patron->delete will take care of the holds, > etc. My question is simple: are we expecting from devs, signoffers and > QAers to update and maintain these routes? > Or a dedicated person will update them before each release? > Until 17044 and 17030, the REST api was of no use for the general public. Right now, it will start being available and this bugs should be reported. Ultimately, the 'routes' could be used for replacing the controller scripts on some portions of the UI, so they should respect the behaviour from the present scripts regarding this kind of things. Maybe we should recommend devs to write routes only on top of already migrated libs (moved to Koha:: namespace). Anyway, lets write those routes! And push them! > = Refactoring = > > Lot of things to say here :) > > == Accounts rewrite == > I'd like to start QAing some parts of the accounts rewrite. > For instance bug 15899 (Remove the use of recordpayment) is a good > first candidate. But to do so I need all the dependencies to be signed > off. > > == Koha::Patrons == > Lot of things have been moved last month, the entry point is bug 16846 > (Move patron related code to Koha::Patron) if you want to help or just > have a look. > Awesome! > == C4::Branch is (almost) dead == > Bug 15758 (Move the C4::Branch related code to Koha::Libraries - part > 4) removes completely C4::Branch. It is a big patch and has been > widely tested. > We need a quick but deep QA here, it's a nightmare to rebase! > > == Koha::AuthorisedValues == > (I have c/p the following on bug 15799 - Move authorised values > related code into Koha::AuthorisedValues - part 2, better to comment > on the bug report) > > We have already started to move the authorised values related code to > the Koha namespace, but some are still missing. > To continue and move C4::Koha::*AuthorisedValues* subroutines to > Koha::AuthorisedValues, we will need to join on > marc_subfield_structure(.authorised_value which is actually an > authorised value category). > To do so we need to define a relationship (fk) between the two tables. > But we have "special AV" like Asort1 (and friends) , LOST, DAMAGED, etc.) > Which could not be in the authorised_values table [yet]. And 3 other > "special special AV) like itemtypes, branches and cn_source. > Others (undetermined so far) are linked to a marc_subfield_structure > from installer/**/marcflavour/**/*framework*.sql, like STACK. > My idea would be to move all the AV categories to a new table > authorised_value_categories. It will contain only 1 column > category_name refered by 1. authorised_values.category, 2. > additional_fields.authorised_value_category (others authorised_value > field should be renamed like that when they refer AV categories) and > 3. iitems_search_fields.authorised_value_category. > And, maybe later, auth_subfield_structure, auth_tag_structure, > borrower_attribute_types, marc_tag_structure (some are varchar(10) et > 20 when they should be 32!) > We could imagine more attributes to this table, but don't ask me to > add a authorised_value_categories.id as pk please :) > We will have to use temporary tables to deal with the DB constraints, > to add more fun. > Finally the .sql will have to be updated to reflect these changes. > I have just started but before to enter in this tunnel I'd like to > know if there are a couple of volunteers to test? :) > > = Ubuntu 16.04/MySQL 5.7 support = > Note that we are still waiting for bug 16554 to support Ubuntu 16.04 > and MySQL 5.7 out of the box. > > A dev meeting is planned on August 24th at 7 UTC > ( > https://wiki.koha-community.org/wiki/Development_IRC_meeting_24_August_2016 > ). > Please have a look at the agenda and the different topics we will > discuss. > > Don't forget to register for the Hackfest in Marseille in October > ( > http://lists.koha-community.org/pipermail/koha-devel/2016-June/042790.html > ) > > Cheers, > Jonathan > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirko at abunchofthings.net Sat Aug 13 21:05:51 2016 From: mirko at abunchofthings.net (Mirko Tietgen) Date: Sat, 13 Aug 2016 21:05:51 +0200 Subject: [Koha-devel] Proposal to join the QA team for packaging related bugs Message-ID: <8b77c54e-3814-1cef-11a5-6f4b1d7ec782@abunchofthings.net> Hello everyone, part of the idea of nightly packages is to constantly update the Debian related files that deal with depenencies. I would like to find and solve possible dependency problems early and minimize the delay between Koha release and related package release. The process to build nightlies is getting more stable now. I have tested automatic patches to the debian/control file -- that is the file that contains the package dependencies -- and I think it will be a helpful tool. I have created bugs labelled "Automatic debian/control updates" for master, stable and unstable, and added Kyle, Frederic and Julian respectively. https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17084 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17108 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17111 The bugs get automatic updates after each nightly build process, as long as they lead to changes in debian/control. To avoid spamming the bugs I have changed that now to "send patch only if bug status is 'Assigned'" (untested :P) There is not really anything to test here, besides a manual sanity check. But it will consume resources to go through the QA process every single time there is a change. For that reason I propose I join the QA team for packaging matters, do a signoff on the automatic patches (as long as they are ok), label them "Passed QA" and send them over to Kyle, Frederic and Julian so they can be pushed early. Any objections? Comments? Cheers, Mirko -- Mirko Tietgen mirko at abunchofthings.net http://koha.abunchofthings.net http://meinkoha.de -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From tomascohen at gmail.com Sat Aug 13 23:55:12 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Sat, 13 Aug 2016 21:55:12 +0000 Subject: [Koha-devel] Proposal to join the QA team for packaging related bugs In-Reply-To: <8b77c54e-3814-1cef-11a5-6f4b1d7ec782@abunchofthings.net> References: <8b77c54e-3814-1cef-11a5-6f4b1d7ec782@abunchofthings.net> Message-ID: +1 for me El s?b., 13 ago. 2016 16:06, Mirko Tietgen escribi?: > Hello everyone, > > part of the idea of nightly packages is to constantly update the > Debian related files that deal with depenencies. I would like to > find and solve possible dependency problems early and minimize the > delay between Koha release and related package release. > > The process to build nightlies is getting more stable now. I have > tested automatic patches to the debian/control file -- that is the > file that contains the package dependencies -- and I think it will > be a helpful tool. > > I have created bugs labelled "Automatic debian/control updates" for > master, stable and unstable, and added Kyle, Frederic and Julian > respectively. > > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17084 > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17108 > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17111 > > The bugs get automatic updates after each nightly build process, as > long as they lead to changes in debian/control. To avoid spamming > the bugs I have changed that now to "send patch only if bug status > is 'Assigned'" (untested :P) > > There is not really anything to test here, besides a manual sanity > check. But it will consume resources to go through the QA process > every single time there is a change. For that reason I propose I > join the QA team for packaging matters, do a signoff on the > automatic patches (as long as they are ok), label them "Passed QA" > and send them over to Kyle, Frederic and Julian so they can be > pushed early. > > Any objections? Comments? > > Cheers, > > Mirko > > > -- > > Mirko Tietgen > mirko at abunchofthings.net > http://koha.abunchofthings.net > http://meinkoha.de > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From kyle.m.hall at gmail.com Sun Aug 14 03:36:04 2016 From: kyle.m.hall at gmail.com (Kyle Hall) Date: Sat, 13 Aug 2016 21:36:04 -0400 Subject: [Koha-devel] Proposal to join the QA team for packaging related bugs In-Reply-To: References: <8b77c54e-3814-1cef-11a5-6f4b1d7ec782@abunchofthings.net> Message-ID: +1 from me as well! Kyle http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Sat, Aug 13, 2016 at 5:55 PM, Tomas Cohen Arazi wrote: > +1 for me > > El s?b., 13 ago. 2016 16:06, Mirko Tietgen > escribi?: > >> Hello everyone, >> >> part of the idea of nightly packages is to constantly update the >> Debian related files that deal with depenencies. I would like to >> find and solve possible dependency problems early and minimize the >> delay between Koha release and related package release. >> >> The process to build nightlies is getting more stable now. I have >> tested automatic patches to the debian/control file -- that is the >> file that contains the package dependencies -- and I think it will >> be a helpful tool. >> >> I have created bugs labelled "Automatic debian/control updates" for >> master, stable and unstable, and added Kyle, Frederic and Julian >> respectively. >> >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17084 >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17108 >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17111 >> >> The bugs get automatic updates after each nightly build process, as >> long as they lead to changes in debian/control. To avoid spamming >> the bugs I have changed that now to "send patch only if bug status >> is 'Assigned'" (untested :P) >> >> There is not really anything to test here, besides a manual sanity >> check. But it will consume resources to go through the QA process >> every single time there is a change. For that reason I propose I >> join the QA team for packaging matters, do a signoff on the >> automatic patches (as long as they are ok), label them "Passed QA" >> and send them over to Kyle, Frederic and Julian so they can be >> pushed early. >> >> Any objections? Comments? >> >> Cheers, >> >> Mirko >> >> >> -- >> >> Mirko Tietgen >> mirko at abunchofthings.net >> http://koha.abunchofthings.net >> http://meinkoha.de >> >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ > > -- > Tom?s Cohen Arazi > Theke Solutions (https://theke.io ) > ? +54 9351 3513384 > GPG: B2F3C15F > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin.renvoize at ptfs-europe.com Sun Aug 14 09:28:05 2016 From: martin.renvoize at ptfs-europe.com (Martin Renvoize) Date: Sun, 14 Aug 2016 07:28:05 +0000 Subject: [Koha-devel] Proposal to join the QA team for packaging related bugs In-Reply-To: References: <8b77c54e-3814-1cef-11a5-6f4b1d7ec782@abunchofthings.net> Message-ID: +1 from me too On Sun, 14 Aug 2016 2:36 am Kyle Hall, wrote: > +1 from me as well! > > Kyle > > > > > http://www.kylehall.info > ByWater Solutions ( http://bywatersolutions.com ) > Meadville Public Library ( http://www.meadvillelibrary.org ) > Crawford County Federated Library System ( http://www.ccfls.org ) > > On Sat, Aug 13, 2016 at 5:55 PM, Tomas Cohen Arazi > wrote: > >> +1 for me >> >> El s?b., 13 ago. 2016 16:06, Mirko Tietgen >> escribi?: >> >>> Hello everyone, >>> >>> part of the idea of nightly packages is to constantly update the >>> Debian related files that deal with depenencies. I would like to >>> find and solve possible dependency problems early and minimize the >>> delay between Koha release and related package release. >>> >>> The process to build nightlies is getting more stable now. I have >>> tested automatic patches to the debian/control file -- that is the >>> file that contains the package dependencies -- and I think it will >>> be a helpful tool. >>> >>> I have created bugs labelled "Automatic debian/control updates" for >>> master, stable and unstable, and added Kyle, Frederic and Julian >>> respectively. >>> >>> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17084 >>> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17108 >>> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17111 >>> >>> The bugs get automatic updates after each nightly build process, as >>> long as they lead to changes in debian/control. To avoid spamming >>> the bugs I have changed that now to "send patch only if bug status >>> is 'Assigned'" (untested :P) >>> >>> There is not really anything to test here, besides a manual sanity >>> check. But it will consume resources to go through the QA process >>> every single time there is a change. For that reason I propose I >>> join the QA team for packaging matters, do a signoff on the >>> automatic patches (as long as they are ok), label them "Passed QA" >>> and send them over to Kyle, Frederic and Julian so they can be >>> pushed early. >>> >>> Any objections? Comments? >>> >>> Cheers, >>> >>> Mirko >>> >>> >>> -- >>> >>> Mirko Tietgen >>> mirko at abunchofthings.net >>> http://koha.abunchofthings.net >>> http://meinkoha.de >>> >>> >>> _______________________________________________ >>> Koha-devel mailing list >>> Koha-devel at lists.koha-community.org >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>> website : http://www.koha-community.org/ >>> git : http://git.koha-community.org/ >>> bugs : http://bugs.koha-community.org/ >> >> -- >> Tom?s Cohen Arazi >> Theke Solutions (https://theke.io ) >> ? +54 9351 3513384 >> GPG: B2F3C15F >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ >> > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From Katrin.Fischer.83 at web.de Sun Aug 14 09:44:56 2016 From: Katrin.Fischer.83 at web.de (Katrin Fischer) Date: Sun, 14 Aug 2016 09:44:56 +0200 Subject: [Koha-devel] Proposal to join the QA team for packaging related bugs In-Reply-To: <8b77c54e-3814-1cef-11a5-6f4b1d7ec782@abunchofthings.net> References: <8b77c54e-3814-1cef-11a5-6f4b1d7ec782@abunchofthings.net> Message-ID: +1, of course :) Am 13.08.2016 um 21:05 schrieb Mirko Tietgen: > Hello everyone, > > part of the idea of nightly packages is to constantly update the > Debian related files that deal with depenencies. I would like to > find and solve possible dependency problems early and minimize the > delay between Koha release and related package release. > > The process to build nightlies is getting more stable now. I have > tested automatic patches to the debian/control file -- that is the > file that contains the package dependencies -- and I think it will > be a helpful tool. > > I have created bugs labelled "Automatic debian/control updates" for > master, stable and unstable, and added Kyle, Frederic and Julian > respectively. > > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17084 > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17108 > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17111 > > The bugs get automatic updates after each nightly build process, as > long as they lead to changes in debian/control. To avoid spamming > the bugs I have changed that now to "send patch only if bug status > is 'Assigned'" (untested :P) > > There is not really anything to test here, besides a manual sanity > check. But it will consume resources to go through the QA process > every single time there is a change. For that reason I propose I > join the QA team for packaging matters, do a signoff on the > automatic patches (as long as they are ok), label them "Passed QA" > and send them over to Kyle, Frederic and Julian so they can be > pushed early. > > Any objections? Comments? > > Cheers, > > Mirko > > > -- > > Mirko Tietgen > mirko at abunchofthings.net > http://koha.abunchofthings.net > http://meinkoha.de > > > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > From Katrin.Fischer.83 at web.de Sun Aug 14 11:16:02 2016 From: Katrin.Fischer.83 at web.de (Katrin Fischer) Date: Sun, 14 Aug 2016 11:16:02 +0200 Subject: [Koha-devel] Git bz AttributeError: 'NoneType' object has no attribute 'group' In-Reply-To: <2C614FEEE2356A428DC65441E30302E0015B671956@nero> References: <2C614FEEE2356A428DC65441E30302E0015B671956@nero> Message-ID: Hi Holger, I am not sure this is the reason, but did you set: config --global bz-tracker.bugs.koha-community.org.https true I think that was the last change we made to the configuration of git bz. Hope this helps, Katrin Am 11.08.2016 um 13:10 schrieb Holger Meissner: > Hello koha devs, > > I was just about to upload a small patch to bugzilla. git bz attach 14434 HEAD > But git bz says > AttributeError: 'NoneType' object has no attribute 'group' > > Any ideas how to fix it? I successfully fetched and rebased the fishsoup branch, but that didn't help. > > Kind regards > Holger > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > From veron at veron.ch Sun Aug 14 12:00:14 2016 From: veron at veron.ch (=?UTF-8?Q?Marc_V=c3=a9ron?=) Date: Sun, 14 Aug 2016 12:00:14 +0200 Subject: [Koha-devel] Proposal to join the QA team for packaging related bugs In-Reply-To: References: <8b77c54e-3814-1cef-11a5-6f4b1d7ec782@abunchofthings.net> Message-ID: <92b801f0-1747-0022-39b1-c33baf9203c5@veron.ch> +1 :-) Marc Am 14.08.2016 um 09:44 schrieb Katrin Fischer: > +1, of course :) > > Am 13.08.2016 um 21:05 schrieb Mirko Tietgen: >> Hello everyone, >> >> part of the idea of nightly packages is to constantly update the >> Debian related files that deal with depenencies. I would like to >> find and solve possible dependency problems early and minimize the >> delay between Koha release and related package release. >> >> The process to build nightlies is getting more stable now. I have >> tested automatic patches to the debian/control file -- that is the >> file that contains the package dependencies -- and I think it will >> be a helpful tool. >> >> I have created bugs labelled "Automatic debian/control updates" for >> master, stable and unstable, and added Kyle, Frederic and Julian >> respectively. >> >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17084 >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17108 >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17111 >> >> The bugs get automatic updates after each nightly build process, as >> long as they lead to changes in debian/control. To avoid spamming >> the bugs I have changed that now to "send patch only if bug status >> is 'Assigned'" (untested :P) >> >> There is not really anything to test here, besides a manual sanity >> check. But it will consume resources to go through the QA process >> every single time there is a change. For that reason I propose I >> join the QA team for packaging matters, do a signoff on the >> automatic patches (as long as they are ok), label them "Passed QA" >> and send them over to Kyle, Frederic and Julian so they can be >> pushed early. >> >> Any objections? Comments? >> >> Cheers, >> >> Mirko >> >> >> -- >> >> Mirko Tietgen >> mirko at abunchofthings.net >> http://koha.abunchofthings.net >> http://meinkoha.de >> >> >> >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ >> > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > From M.de.Rooy at rijksmuseum.nl Mon Aug 15 08:26:03 2016 From: M.de.Rooy at rijksmuseum.nl (Marcel de Rooy) Date: Mon, 15 Aug 2016 06:26:03 +0000 Subject: [Koha-devel] Proposal to join the QA team for packaging related bugs In-Reply-To: <92b801f0-1747-0022-39b1-c33baf9203c5@veron.ch> References: <8b77c54e-3814-1cef-11a5-6f4b1d7ec782@abunchofthings.net> <92b801f0-1747-0022-39b1-c33baf9203c5@veron.ch> Message-ID: +1 >> There is not really anything to test here, besides a manual sanity >> check. But it will consume resources to go through the QA process >> every single time there is a change. For that reason I propose I join >> the QA team for packaging matters, do a signoff on the automatic >> patches (as long as they are ok), label them "Passed QA" >> and send them over to Kyle, Frederic and Julian so they can be pushed >> early. From dcook at prosentient.com.au Mon Aug 15 08:33:04 2016 From: dcook at prosentient.com.au (David Cook) Date: Mon, 15 Aug 2016 16:33:04 +1000 Subject: [Koha-devel] Interface patterns (tabs) Message-ID: <00f201d1f6be$e09204a0$a1b60de0$@prosentient.com.au> Hi Owen, Do you have any documentation for preferred us of jQuery tabs? I'm looking at https://wiki.koha-community.org/wiki/Interface_patterns but nothing is jumping out at me. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 -------------- next part -------------- An HTML attachment was scrubbed... URL: From M.de.Rooy at rijksmuseum.nl Mon Aug 15 09:28:39 2016 From: M.de.Rooy at rijksmuseum.nl (Marcel de Rooy) Date: Mon, 15 Aug 2016 07:28:39 +0000 Subject: [Koha-devel] What's on in Koha devel #3 (RESTful API) Message-ID: > = RESTful API = > I also would like to know who is willing to maintain this code? For instance on bug 16330 (REST API: add routes to add, update and delete > patrons) the delete route is calling > # check if loans, reservations, debarrment, etc. before deletion! > my $res = $patron->delete > So no check is done from the API. > But if bug 16907 (Move DelMember and HandleDelBorrower to > Koha::Patron) is pushed, $patron->delete will take care of the holds, etc. My question is simple: are we expecting from devs, signoffers and > QAers to update and maintain these routes? > Or a dedicated person will update them before each release? Just formally, I would say that if the community agreed to push this code, the community should maintain it too. We should keep the code base in a consistent state. Similarly, we should prevent redundant code: the checks for deleting a patron in one place, etc. If we are on a road now creating a lot of redundant code between Koha/REST and Koha/Objects, we should re-evaluate first and not just keep pushing more code. Is this mainly a communication issue? Which goals do we want to achieve? Should our scripts go via REST some day and should all business logic remain in Objects? Etc. etc. Marcel From Holger.Meissner at hs-gesundheit.de Mon Aug 15 11:39:36 2016 From: Holger.Meissner at hs-gesundheit.de (Holger Meissner) Date: Mon, 15 Aug 2016 09:39:36 +0000 Subject: [Koha-devel] Git bz AttributeError: 'NoneType' object has no attribute 'group' Message-ID: <2C614FEEE2356A428DC65441E30302E0015B671BE0@nero> Hi Katrin, problem solved, thank you! > Hi Holger, > I am not sure this is the reason, but did you set: > config --global bz-tracker.bugs.koha-community.org.https true > I think that was the last change we made to the configuration of git bz. > Hope this helps, > Katrin > Am 11.08.2016 um 13:10 schrieb Holger Meissner: >> Hello koha devs, >> >> I was just about to upload a small patch to bugzilla. git bz attach >> 14434 HEAD But git bz says >> AttributeError: 'NoneType' object has no attribute 'group' >> >> Any ideas how to fix it? I successfully fetched and rebased the fishsoup branch, but that didn't help. >> >> Kind regards >> Holger From tomascohen at gmail.com Mon Aug 15 14:54:06 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Mon, 15 Aug 2016 12:54:06 +0000 Subject: [Koha-devel] What's on in Koha devel #3 (RESTful API) In-Reply-To: References: Message-ID: El lun., 15 ago. 2016 a las 4:28, Marcel de Rooy () escribi?: > > = RESTful API = > > I also would like to know who is willing to maintain this code? For > instance on bug 16330 (REST API: add routes to add, update and delete > > patrons) the delete route is calling > > # check if loans, reservations, debarrment, etc. before deletion! > > my $res = $patron->delete > > So no check is done from the API. > > But if bug 16907 (Move DelMember and HandleDelBorrower to > > Koha::Patron) is pushed, $patron->delete will take care of the holds, > etc. My question is simple: are we expecting from devs, signoffers and > > QAers to update and maintain these routes? > > Or a dedicated person will update them before each release? > > Should our scripts go via REST some day and should all business logic > remain in Objects? Etc. etc. > Exactly! What Jonathan noticed, is that no one realized there was a bug on the REST api, this should be reported, and fixed. The more we use the API the more bugs we will find, and the more we will enhance it. I think Marseille will be a good place to work on this. Regards -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From Katrin.Fischer.83 at web.de Mon Aug 15 23:01:23 2016 From: Katrin.Fischer.83 at web.de (Katrin Fischer) Date: Mon, 15 Aug 2016 23:01:23 +0200 Subject: [Koha-devel] What's on in Koha devel #3 (RESTful API) In-Reply-To: References: Message-ID: <48032621-172b-4c44-6bce-e57f2e82c768@web.de> Am 15.08.2016 um 14:54 schrieb Tomas Cohen Arazi: > > > El lun., 15 ago. 2016 a las 4:28, Marcel de Rooy > (>) escribi?: > > > = RESTful API = > > I also would like to know who is willing to maintain this code? > For instance on bug 16330 (REST API: add routes to add, update and > delete > > patrons) the delete route is calling > > # check if loans, reservations, debarrment, etc. before deletion! > > my $res = $patron->delete > > So no check is done from the API. > > But if bug 16907 (Move DelMember and HandleDelBorrower to > > Koha::Patron) is pushed, $patron->delete will take care of the > holds, etc. My question is simple: are we expecting from devs, > signoffers and > QAers to update and maintain these routes? > > Or a dedicated person will update them before each release? > > Should our scripts go via REST some day and should all business > logic remain in Objects? Etc. etc. > > > Exactly! What Jonathan noticed, is that no one realized there was a bug > on the REST api, this should be reported, and fixed. The more we use the > API the more bugs we will find, and the more we will enhance it. > > I think Marseille will be a good place to work on this. > > Regards Maybe Marseille will also be a good place for a 'beginners tutorial'? I'd like to learn more about the RESTful API and how to use/test it in order to help with improving it :) Katrin From benjamin.rokseth at kul.oslo.kommune.no Tue Aug 16 13:00:01 2016 From: benjamin.rokseth at kul.oslo.kommune.no (Benjamin Rokseth) Date: Tue, 16 Aug 2016 11:00:01 +0000 Subject: [Koha-devel] Koha-devel Digest, Vol 129, Issue 18 In-Reply-To: References: Message-ID: <1471345200922.66063@kul.oslo.kommune.no> ________________________________________ Fra: koha-devel-bounces at lists.koha-community.org p? vegne av koha-devel-request at lists.koha-community.org Sendt: 16. august 2016 12:00 Til: koha-devel at lists.koha-community.org Emne: Koha-devel Digest, Vol 129, Issue 18 Send Koha-devel mailing list submissions to koha-devel at lists.koha-community.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel or, via email, send a message with subject or body 'help' to koha-devel-request at lists.koha-community.org You can reach the person managing the list at koha-devel-owner at lists.koha-community.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Koha-devel digest..." Today's Topics: 1. Re: What's on in Koha devel #3 (RESTful API) (Tomas Cohen Arazi) 2. Re: What's on in Koha devel #3 (RESTful API) (Katrin Fischer) ---------------------------------------------------------------------- Message: 1 Date: Mon, 15 Aug 2016 12:54:06 +0000 From: Tomas Cohen Arazi To: Marcel de Rooy , "koha-devel at lists.koha-community.org" Subject: Re: [Koha-devel] What's on in Koha devel #3 (RESTful API) Message-ID: Content-Type: text/plain; charset="utf-8" El lun., 15 ago. 2016 a las 4:28, Marcel de Rooy () escribi?: > > = RESTful API = > > I also would like to know who is willing to maintain this code? For > instance on bug 16330 (REST API: add routes to add, update and delete > > patrons) the delete route is calling > > # check if loans, reservations, debarrment, etc. before deletion! > > my $res = $patron->delete > > So no check is done from the API. > > But if bug 16907 (Move DelMember and HandleDelBorrower to > > Koha::Patron) is pushed, $patron->delete will take care of the holds, > etc. My question is simple: are we expecting from devs, signoffers and > > QAers to update and maintain these routes? > > Or a dedicated person will update them before each release? > > Should our scripts go via REST some day and should all business logic > remain in Objects? Etc. etc. > Exactly! What Jonathan noticed, is that no one realized there was a bug on the REST api, this should be reported, and fixed. The more we use the API the more bugs we will find, and the more we will enhance it. I think Marseille will be a good place to work on this. Regards -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ Message: 2 Date: Mon, 15 Aug 2016 23:01:23 +0200 From: Katrin Fischer To: koha-devel at lists.koha-community.org Subject: Re: [Koha-devel] What's on in Koha devel #3 (RESTful API) Message-ID: <48032621-172b-4c44-6bce-e57f2e82c768 at web.de> Content-Type: text/plain; charset=windows-1252 Am 15.08.2016 um 14:54 schrieb Tomas Cohen Arazi: > > > El lun., 15 ago. 2016 a las 4:28, Marcel de Rooy > (>) escribi?: > > > = RESTful API = > > I also would like to know who is willing to maintain this code? > For instance on bug 16330 (REST API: add routes to add, update and > delete > > patrons) the delete route is calling > > # check if loans, reservations, debarrment, etc. before deletion! > > my $res = $patron->delete > > So no check is done from the API. > > But if bug 16907 (Move DelMember and HandleDelBorrower to > > Koha::Patron) is pushed, $patron->delete will take care of the > holds, etc. My question is simple: are we expecting from devs, > signoffers and > QAers to update and maintain these routes? > > Or a dedicated person will update them before each release? > > Should our scripts go via REST some day and should all business > logic remain in Objects? Etc. etc. > > > Exactly! What Jonathan noticed, is that no one realized there was a bug > on the REST api, this should be reported, and fixed. The more we use the > API the more bugs we will find, and the more we will enhance it. > > I think Marseille will be a good place to work on this. > > Regards > Maybe Marseille will also be a good place for a 'beginners tutorial'? > I'd like to learn more about the RESTful API and how to use/test it in > order to help with improving it :) > Katrin Great with some attention on the RESTful API! My initial reaction on reading this is that the primary resposibility should be on the community and developers that push new code need to run all tests. Test coverage and a more automatic Signoff/QA would save everyone some headaches. The REST api is something we make heavy use of for any non-staff operation, and we develop it mainly as an abstraction on existing Koha modules or, if existing, preferrably Koha Objects. So much great work has been done recently on Koha Objects and refactoring Koha modules that expanding the API is mere joy. The API should, as Jonathan's keen eye spotted, have as little business logic as possible. CRUD operations are best if they simply use Koha objects. Hope to be able to attend Marseille so we can talk more and agree on rules on the API! Benjamin Oslo Public Library ------------------------------ _______________________________________________ Koha-devel mailing list Koha-devel at lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ End of Koha-devel Digest, Vol 129, Issue 18 ******************************************* From kyle.m.hall at gmail.com Tue Aug 16 20:31:19 2016 From: kyle.m.hall at gmail.com (Kyle Hall) Date: Tue, 16 Aug 2016 14:31:19 -0400 Subject: [Koha-devel] What's on in Koha devel #3 (RESTful API) In-Reply-To: References: Message-ID: In my mind, we should be moving all appropriate logic to Koha/Objects with Koha/REST being just an api wrapper around the appropriate object. That will keep us from duplicating code and reduce bugs. Kyle http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Mon, Aug 15, 2016 at 3:28 AM, Marcel de Rooy wrote: > > = RESTful API = > > I also would like to know who is willing to maintain this code? For > instance on bug 16330 (REST API: add routes to add, update and delete > > patrons) the delete route is calling > > # check if loans, reservations, debarrment, etc. before deletion! > > my $res = $patron->delete > > So no check is done from the API. > > But if bug 16907 (Move DelMember and HandleDelBorrower to > > Koha::Patron) is pushed, $patron->delete will take care of the holds, > etc. My question is simple: are we expecting from devs, signoffers and > > QAers to update and maintain these routes? > > Or a dedicated person will update them before each release? > > Just formally, I would say that if the community agreed to push this code, > the community should maintain it too. We should keep the code base in a > consistent state. Similarly, we should prevent redundant code: the checks > for deleting a patron in one place, etc. > > If we are on a road now creating a lot of redundant code between Koha/REST > and Koha/Objects, we should re-evaluate first and not just keep pushing > more code. Is this mainly a communication issue? Which goals do we want to > achieve? Should our scripts go via REST some day and should all business > logic remain in Objects? Etc. etc. > > Marcel > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.a at navalmarinearchive.com Tue Aug 16 17:07:22 2016 From: paul.a at navalmarinearchive.com (Paul A) Date: Tue, 16 Aug 2016 11:07:22 -0400 Subject: [Koha-devel] Search by itemnumber Message-ID: <5.2.1.1.2.20160816110332.02a3fbe8@stormy.ca> I must be missing something very simple -- maybe related back to bug#168 ;={ How to search (staff/admin) for [items.]itemnumber? Scenario: staff logs give adds, edits etc of itemnumbers, but without the biblionumber. Thanks and best regards -- Paul From barton at bywatersolutions.com Wed Aug 17 20:55:32 2016 From: barton at bywatersolutions.com (Barton Chittenden) Date: Wed, 17 Aug 2016 14:55:32 -0400 Subject: [Koha-devel] Search by itemnumber In-Reply-To: <5.2.1.1.2.20160816110332.02a3fbe8@stormy.ca> References: <5.2.1.1.2.20160816110332.02a3fbe8@stormy.ca> Message-ID: If you do a keyword search using itemnumber: That will take you to the item record. E.g. Log in to http://intranet.bywatersolutions.com/ (user:pass is bywater:bywater) Then click 'Search the catalog' and type itemnumber:37771 This will take you to the record for *Juggling for the complete klutz / by John Cassidy and B.C. Rimbeau:* http://intranet.bywatersolutions.com/cgi-bin/koha/catalogue/detail.pl?biblionumber=36263 On Tue, Aug 16, 2016 at 11:07 AM, Paul A wrote: > I must be missing something very simple -- maybe related back to bug#168 > ;={ > > How to search (staff/admin) for [items.]itemnumber? > > Scenario: staff logs give adds, edits etc of itemnumbers, but without the > biblionumber. > > Thanks and best regards -- Paul > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barton at bywatersolutions.com Wed Aug 17 21:10:14 2016 From: barton at bywatersolutions.com (Barton Chittenden) Date: Wed, 17 Aug 2016 15:10:14 -0400 Subject: [Koha-devel] Search by itemnumber In-Reply-To: References: <5.2.1.1.2.20160816110332.02a3fbe8@stormy.ca> Message-ID: P.s. this works for barcode as well. On the bywater demo site, try barcode:TVSN5000136467 You'll get the same record Take a look at getIndexes() in C4/Search.pm -- a number of those indexes should be searchable the same way. The actual logic about which can and which can't is probably in some corner of C4/Search.pm where angels fear to tread, but those should at least be the candidates... On Wed, Aug 17, 2016 at 2:55 PM, Barton Chittenden < barton at bywatersolutions.com> wrote: > If you do a keyword search using > itemnumber: > > That will take you to the item record. > > E.g. > > Log in to http://intranet.bywatersolutions.com/ (user:pass is > bywater:bywater) > > Then click 'Search the catalog' and type > > itemnumber:37771 > > This will take you to the record for *Juggling for the complete klutz / > by John Cassidy and B.C. Rimbeau:* > > http://intranet.bywatersolutions.com/cgi-bin/koha/catalogue/detail.pl? > biblionumber=36263 > > > > On Tue, Aug 16, 2016 at 11:07 AM, Paul A > wrote: > >> I must be missing something very simple -- maybe related back to bug#168 >> ;={ >> >> How to search (staff/admin) for [items.]itemnumber? >> >> Scenario: staff logs give adds, edits etc of itemnumbers, but without the >> biblionumber. >> >> Thanks and best regards -- Paul >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kohanews at gmail.com Thu Aug 18 20:18:52 2016 From: kohanews at gmail.com (kohanews) Date: Thu, 18 Aug 2016 11:18:52 -0700 Subject: [Koha-devel] Call for News: August 2016 Newsletter Message-ID: <2800df5c-dae6-cc3a-7bd5-29ba621c48b1@gmail.com> Fellow Koha users ~ I'm collecting news for the August newsletter. Send anything noteworthy to: k o h a news AT gmail dot com News criteria: --------------------------- ** For events **: * - Please include dates for past events. If I can't find dates I may not add it. * - Announcements for future events with dates T.B.A. are fine ...Eg., Kohacon * - For past events -- **** one month back is the cut-off ***. * News items can be of any length. * Images are fine * Anything and everything Koha. * Submit by the 26th of the month. If you are working on an interesting project or development related to Koha, please let me know and I'll include it in the development section. -- Chad Roseburg Editor, Koha Community Newsletter -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.druart at bugs.koha-community.org Mon Aug 22 16:38:55 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Mon, 22 Aug 2016 15:38:55 +0100 Subject: [Koha-devel] Move the AV code to the Koha namespace Message-ID: Hi devs, I have already started to move the authorised values related code to the Koha namespace, but to continue I'd like to get feedback on the method to continue. I have asked you in the last "what's on in koha-devel" email, but did not get any answer. Please see bug 15799 for the step-by-step plan. Cheers, Jonathan https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15799 From philippe.blouin at inlibro.com Mon Aug 22 20:40:26 2016 From: philippe.blouin at inlibro.com (Philippe Blouin) Date: Mon, 22 Aug 2016 14:40:26 -0400 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: References: Message-ID: <57BB471A.6070701@inlibro.com> Hello! We're trying to find the correction combination. We're new on Jessie, so we still have some tweaking needed... - By default, we get zebra 2.00.59 installed on Jessie through the packages. - On 16.05, we get some very bad results in the search when the itemtype contains an hyphen (-), like 'A-DOC'. - So we installed zebra 2.00.62. This fixes the search... - But now we do not have facets. - So we set 0 - And now we have facets. But this feels... wrong? My dummy question: what is the supposedly correct version of Zebra on Jessie ? And we're we correct in setting the config to 0 ? Thanks Blou From mtompset at hotmail.com Mon Aug 22 20:49:44 2016 From: mtompset at hotmail.com (Mark Tompsett) Date: Mon, 22 Aug 2016 18:49:44 +0000 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: <57BB471A.6070701@inlibro.com> References: <57BB471A.6070701@inlibro.com> Message-ID: Greetings, Similar problem. I hope someone has a better solution than setting it to 0. GPML, Mark Tompsett -----Original Message----- From: Philippe Blouin Sent: Monday, August 22, 2016 2:40 PM To: koha-devel at lists.koha-community.org Subject: [Koha-devel] 16.05, zebra and jessie Hello! We're trying to find the correction combination. We're new on Jessie, so we still have some tweaking needed... - By default, we get zebra 2.00.59 installed on Jessie through the packages. - On 16.05, we get some very bad results in the search when the itemtype contains an hyphen (-), like 'A-DOC'. - So we installed zebra 2.00.62. This fixes the search... - But now we do not have facets. - So we set 0 - And now we have facets. But this feels... wrong? My dummy question: what is the supposedly correct version of Zebra on Jessie ? And we're we correct in setting the config to 0 ? Thanks Blou _______________________________________________ Koha-devel mailing list Koha-devel at lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ From barton at bywatersolutions.com Mon Aug 22 22:31:57 2016 From: barton at bywatersolutions.com (Barton Chittenden) Date: Mon, 22 Aug 2016 16:31:57 -0400 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: References: <57BB471A.6070701@inlibro.com> Message-ID: I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I have run into this, when using ICU-Chains: Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 URL : https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 Priority : P5 - low Urgency : enhancement Status : NEW I also know that when use_zebra_facets was first introduced, it was *very* slow -- I can't find any bugs about that though. It's possible that it got so slow under idzebra-2.0 2.61 that the searches are timing out. It should be possible to set the logging for zebra so that you can see the PQF queries: See Bug 15714 : Remove zebra.log from debian scripts and add optional log levels URL : https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714 Priority : P5 - low Urgency : enhancement Status : RESOLVED For setting the log levels And http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html For a general discussion of how to use them. ... This should give you some idea of what's failing, both in terms of the dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. My general feeling is that 2.0.59 is irredeemably broken by bug 16581, and we need at least 2.0.60, but I don't have any experience with zebra facets. --Barton On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett wrote: > Greetings, > > Similar problem. I hope someone has a better solution than setting it to 0. > > GPML, > Mark Tompsett > > -----Original Message----- > From: Philippe Blouin > Sent: Monday, August 22, 2016 2:40 PM > To: koha-devel at lists.koha-community.org > Subject: [Koha-devel] 16.05, zebra and jessie > > Hello! > > We're trying to find the correction combination. We're new on Jessie, > so we still have some tweaking needed... > > - By default, we get zebra 2.00.59 installed on Jessie through the > packages. > - On 16.05, we get some very bad results in the search when the itemtype > contains an hyphen (-), like 'A-DOC'. > - So we installed zebra 2.00.62. This fixes the search... > - But now we do not have facets. > - So we set 0 > - And now we have facets. But this feels... wrong? > > My dummy question: what is the supposedly correct version of Zebra on > Jessie ? > And we're we correct in setting the config to 0 ? > > Thanks > Blou > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at bigballofwax.co.nz Tue Aug 23 01:15:40 2016 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Tue, 23 Aug 2016 11:15:40 +1200 Subject: [Koha-devel] Koha 3.20.14 released Message-ID: The Koha community is happy to announce the release of Koha 3.20.14. This is a security release and users of 3.20.x are encouraged to upgrade. Please read the full release notes at https://koha-community.org/koha-3-20-14-released/ Chris From mtompset at hotmail.com Tue Aug 23 03:50:46 2016 From: mtompset at hotmail.com (Mark Tompsett) Date: Tue, 23 Aug 2016 01:50:46 +0000 Subject: [Koha-devel] schema.koha-community.org Message-ID: Greetings, Currently, it lists whatever is run as of a particular date. Is there a way to version it? There are currently 181 tables, not 175. So, this is clearly the difference between master and a release. https://schema.koha-community.org/{version}/... Just inquiring and thinking out loud. GPML, Mark Tompsett -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisc at catalyst.net.nz Tue Aug 23 04:30:57 2016 From: chrisc at catalyst.net.nz (Chris Cormack) Date: Tue, 23 Aug 2016 14:30:57 +1200 Subject: [Koha-devel] schema.koha-community.org In-Reply-To: References: Message-ID: On 23/08/16 13:50, Mark Tompsett wrote: > Greetings, > > Currently, it lists whatever is run as of a particular date. Is there a way to version it? > There are currently 181 tables, not 175. So, this is clearly the difference between master and a release. > > https://schema.koha-community.org/{version}/... > > Just inquiring and thinking out loud. > Yep, can definitely version it. I'll update the scripts tonight and see how we go Chris -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From jonathan.druart at bugs.koha-community.org Tue Aug 23 11:38:10 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Tue, 23 Aug 2016 10:38:10 +0100 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: References: <57BB471A.6070701@inlibro.com> Message-ID: See bug 13665 - Retrieve facets from zebra is slow To understand why and when use_zebra_facet=1 is slow 2016-08-22 21:31 GMT+01:00 Barton Chittenden : > I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I > have run into this, when using ICU-Chains: > > Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 > URL : https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 > Priority : P5 - low > Urgency : enhancement > Status : NEW > > I also know that when use_zebra_facets was first introduced, it was *very* > slow -- I can't find any bugs about that though. It's possible that it got > so slow under idzebra-2.0 2.61 that the searches are timing out. > > It should be possible to set the logging for zebra so that you can see the > PQF queries: > > See > > Bug 15714 : Remove zebra.log from debian scripts and add optional log levels > URL : https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714 > Priority : P5 - low > Urgency : enhancement > Status : RESOLVED > > For setting the log levels > > And http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html > > For a general discussion of how to use them. > > ... This should give you some idea of what's failing, both in terms of the > dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. > > My general feeling is that 2.0.59 is irredeemably broken by bug 16581, and > we need at least 2.0.60, but I don't have any experience with zebra facets. > > --Barton > > > > On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett wrote: >> >> Greetings, >> >> Similar problem. I hope someone has a better solution than setting it to >> 0. >> >> GPML, >> Mark Tompsett >> >> -----Original Message----- >> From: Philippe Blouin >> Sent: Monday, August 22, 2016 2:40 PM >> To: koha-devel at lists.koha-community.org >> Subject: [Koha-devel] 16.05, zebra and jessie >> >> Hello! >> >> We're trying to find the correction combination. We're new on Jessie, >> so we still have some tweaking needed... >> >> - By default, we get zebra 2.00.59 installed on Jessie through the >> packages. >> - On 16.05, we get some very bad results in the search when the itemtype >> contains an hyphen (-), like 'A-DOC'. >> - So we installed zebra 2.00.62. This fixes the search... >> - But now we do not have facets. >> - So we set 0 >> - And now we have facets. But this feels... wrong? >> >> My dummy question: what is the supposedly correct version of Zebra on >> Jessie ? >> And we're we correct in setting the config to 0 ? >> >> Thanks >> Blou >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ > > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From barton at bywatersolutions.com Tue Aug 23 15:21:19 2016 From: barton at bywatersolutions.com (Barton Chittenden) Date: Tue, 23 Aug 2016 09:21:19 -0400 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: References: <57BB471A.6070701@inlibro.com> Message-ID: Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to disk (~16G/query on large libraries). Bug 13665 mentions that searches could be taking upwards of 40 seconds to complete -- I think that we've seen searches time out and return no results at about 1 minute. Is it possible to tune Zebra's space/time optimizations in any way so that it doesn't write such large files to disk? On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart < jonathan.druart at bugs.koha-community.org> wrote: > See bug 13665 - Retrieve facets from zebra is slow > To understand why and when use_zebra_facet=1 is slow > > 2016-08-22 21:31 GMT+01:00 Barton Chittenden >: > > I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I > > have run into this, when using ICU-Chains: > > > > Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 > > URL : https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id= > 16581 > > Priority : P5 - low > > Urgency : enhancement > > Status : NEW > > > > I also know that when use_zebra_facets was first introduced, it was > *very* > > slow -- I can't find any bugs about that though. It's possible that it > got > > so slow under idzebra-2.0 2.61 that the searches are timing out. > > > > It should be possible to set the logging for zebra so that you can see > the > > PQF queries: > > > > See > > > > Bug 15714 : Remove zebra.log from debian scripts and add optional log > levels > > URL : https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id= > 15714 > > Priority : P5 - low > > Urgency : enhancement > > Status : RESOLVED > > > > For setting the log levels > > > > And http://koha.1045719.n5.nabble.com/Improving-Zebra-logging- > td5861827.html > > > > For a general discussion of how to use them. > > > > ... This should give you some idea of what's failing, both in terms of > the > > dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. > > > > My general feeling is that 2.0.59 is irredeemably broken by bug 16581, > and > > we need at least 2.0.60, but I don't have any experience with zebra > facets. > > > > --Barton > > > > > > > > On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett > wrote: > >> > >> Greetings, > >> > >> Similar problem. I hope someone has a better solution than setting it to > >> 0. > >> > >> GPML, > >> Mark Tompsett > >> > >> -----Original Message----- > >> From: Philippe Blouin > >> Sent: Monday, August 22, 2016 2:40 PM > >> To: koha-devel at lists.koha-community.org > >> Subject: [Koha-devel] 16.05, zebra and jessie > >> > >> Hello! > >> > >> We're trying to find the correction combination. We're new on Jessie, > >> so we still have some tweaking needed... > >> > >> - By default, we get zebra 2.00.59 installed on Jessie through the > >> packages. > >> - On 16.05, we get some very bad results in the search when the itemtype > >> contains an hyphen (-), like 'A-DOC'. > >> - So we installed zebra 2.00.62. This fixes the search... > >> - But now we do not have facets. > >> - So we set 0 > >> - And now we have facets. But this feels... wrong? > >> > >> My dummy question: what is the supposedly correct version of Zebra on > >> Jessie ? > >> And we're we correct in setting the config to 0 ? > >> > >> Thanks > >> Blou > >> _______________________________________________ > >> Koha-devel mailing list > >> Koha-devel at lists.koha-community.org > >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > >> website : http://www.koha-community.org/ > >> git : http://git.koha-community.org/ > >> bugs : http://bugs.koha-community.org/ > >> > >> _______________________________________________ > >> Koha-devel mailing list > >> Koha-devel at lists.koha-community.org > >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > >> website : http://www.koha-community.org/ > >> git : http://git.koha-community.org/ > >> bugs : http://bugs.koha-community.org/ > > > > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomascohen at gmail.com Tue Aug 23 15:41:57 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Tue, 23 Aug 2016 13:41:57 +0000 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: References: <57BB471A.6070701@inlibro.com> Message-ID: I have seen use_zebra_facets=1 cause no facets rendered when GRS-1 configuration files are kept during upgrades up to where GRS-1 got deprecated (3.20?). Is it the case? What does the About > System information page says about your config? The slowliness is not in zebra per se, but in the way we retrieve the facets from it (so Koha/Perl side). We retrieve each facet at a time instead of fetching them all in one call. And they come in XML format, so need to be parsed. So, if anyone is willing to improve it, just need to optimize this function (read the TODO): sub _get_facet_from_result_set { my $facet_idx = shift; my $rs = shift; my $sep = shift; my $internal_sep = '<*>'; my $facetMaxCount = C4::Context->preference('FacetMaxCount') // 20; return if ( ! defined $facet_idx || ! defined $rs ); # zebra's facet element, untokenized index my $facet_element = 'zebra::facet::' . $facet_idx . ':0:' . $facetMaxCount; # configure zebra results for retrieving the desired facet $rs->option( elementSetName => $facet_element ); # get the facet record from result set my $facet = $rs->record( 0 )->raw; # if the facet has no restuls... return if !defined $facet; # TODO: benchmark DOM vs. SAX performance my $facet_dom = XML::LibXML->load_xml( string => ($facet) ); my @terms = $facet_dom->getElementsByTagName('term'); return if ! @terms; my $facets = {}; foreach my $term ( @terms ) { my $facet_value = $term->textContent; $facet_value =~ s/\Q$internal_sep\E/$sep/ if defined $sep; $facets->{ $facet_value } = $term->getAttribute( 'occur' ); } return $facets; } Another option would be to make _get_facets_from_zebra build the element set containing all facets so they are read in one call (comma-separate all elements). The problem is that Zebra returns zero if one of the elements is empty. Jared proposed to create a ghost record with all facet fields. I didn't manage to make it work. Another option is to patch Zebra. I started that, but abandoned once the ES code got pushed. So, if use_zebra_facets=0 is good enough, maybe it should be recommended. Problem is it is not a real facet, but the sole extraction of the fields from the first x records. As I said, it could be good enough anyway. Regards El mar., 23 ago. 2016 a las 10:21, Barton Chittenden (< barton at bywatersolutions.com>) escribi?: > Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to > disk (~16G/query on large libraries). Bug 13665 mentions that searches > could be taking upwards of 40 seconds to complete -- I think that we've > seen searches time out and return no results at about 1 minute. > > Is it possible to tune Zebra's space/time optimizations in any way so that > it doesn't write such large files to disk? > > On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart < > jonathan.druart at bugs.koha-community.org> wrote: > >> See bug 13665 - Retrieve facets from zebra is slow >> To understand why and when use_zebra_facet=1 is slow >> >> 2016-08-22 21:31 GMT+01:00 Barton Chittenden > >: >> > I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but >> I >> > have run into this, when using ICU-Chains: >> > >> > Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 >> > URL : >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 >> > Priority : P5 - low >> > Urgency : enhancement >> > Status : NEW >> > >> > I also know that when use_zebra_facets was first introduced, it was >> *very* >> > slow -- I can't find any bugs about that though. It's possible that it >> got >> > so slow under idzebra-2.0 2.61 that the searches are timing out. >> > >> > It should be possible to set the logging for zebra so that you can see >> the >> > PQF queries: >> > >> > See >> > >> > Bug 15714 : Remove zebra.log from debian scripts and add optional log >> levels >> > URL : >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714 >> > Priority : P5 - low >> > Urgency : enhancement >> > Status : RESOLVED >> > >> > For setting the log levels >> > >> > And >> http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html >> > >> > For a general discussion of how to use them. >> > >> > ... This should give you some idea of what's failing, both in terms of >> the >> > dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. >> > >> > My general feeling is that 2.0.59 is irredeemably broken by bug 16581, >> and >> > we need at least 2.0.60, but I don't have any experience with zebra >> facets. >> > >> > --Barton >> > >> > >> > >> > On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett >> wrote: >> >> >> >> Greetings, >> >> >> >> Similar problem. I hope someone has a better solution than setting it >> to >> >> 0. >> >> >> >> GPML, >> >> Mark Tompsett >> >> >> >> -----Original Message----- >> >> From: Philippe Blouin >> >> Sent: Monday, August 22, 2016 2:40 PM >> >> To: koha-devel at lists.koha-community.org >> >> Subject: [Koha-devel] 16.05, zebra and jessie >> >> >> >> Hello! >> >> >> >> We're trying to find the correction combination. We're new on Jessie, >> >> so we still have some tweaking needed... >> >> >> >> - By default, we get zebra 2.00.59 installed on Jessie through the >> >> packages. >> >> - On 16.05, we get some very bad results in the search when the >> itemtype >> >> contains an hyphen (-), like 'A-DOC'. >> >> - So we installed zebra 2.00.62. This fixes the search... >> >> - But now we do not have facets. >> >> - So we set 0 >> >> - And now we have facets. But this feels... wrong? >> >> >> >> My dummy question: what is the supposedly correct version of Zebra on >> >> Jessie ? >> >> And we're we correct in setting the config to 0 ? >> >> >> >> Thanks >> >> Blou >> >> _______________________________________________ >> >> Koha-devel mailing list >> >> Koha-devel at lists.koha-community.org >> >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> >> website : http://www.koha-community.org/ >> >> git : http://git.koha-community.org/ >> >> bugs : http://bugs.koha-community.org/ >> >> >> >> _______________________________________________ >> >> Koha-devel mailing list >> >> Koha-devel at lists.koha-community.org >> >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> >> website : http://www.koha-community.org/ >> >> git : http://git.koha-community.org/ >> >> bugs : http://bugs.koha-community.org/ >> > >> > >> > >> > _______________________________________________ >> > Koha-devel mailing list >> > Koha-devel at lists.koha-community.org >> > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> > website : http://www.koha-community.org/ >> > git : http://git.koha-community.org/ >> > bugs : http://bugs.koha-community.org/ >> > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.a at navalmarinearchive.com Tue Aug 23 16:43:29 2016 From: paul.a at navalmarinearchive.com (Paul A) Date: Tue, 23 Aug 2016 10:43:29 -0400 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: References: <57BB471A.6070701@inlibro.com> Message-ID: <5.2.1.1.2.20160823103433.05497cc0@pop.navalmarinearchive.com> At 09:21 AM 8/23/2016 -0400, Barton Chittenden wrote: >Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to >disk (~16G/query on large libraries). Bug? 13665 mentions that searches >could be taking upwards of 40 seconds to complete -- I think that we've >seen searches time out and return no results at about 1 minute. When I looked into "speed" some time ago , it was the fact that multi-core processing was not possible (Intel 8-core I7 machine.) Speed is the only reason that we decided to stay with 3.8.24 rather than upgrade. Best -- Paul >Is it possible to tune Zebra's space/time optimizations in any way so that >it doesn't write such large files to disk? > >On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart ><jonathan.druart at bugs.koha-community.org> >wrote: >See bug 13665 - Retrieve facets from zebra is slow >To understand why and when use_zebra_facet=1 is slow > >2016-08-22 21:31 GMT+01:00 Barton Chittenden ><barton at bywatersolutions.com>: > > I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I > > have run into this, when using ICU-Chains: > > > > Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 > > URL? ? ? ? : > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 > > Priority? : P5 - low > > Urgency? ? : enhancement > > Status? ? : NEW > > > > I also know that when use_zebra_facets was first introduced, it was *very* > > slow -- I can't find any bugs about that though. It's possible that it got > > so slow under idzebra-2.0 2.61 that the searches are timing out. > > > > It should be possible to set the logging for zebra so that you can see the > > PQF queries: > > > > See > > > > Bug 15714 : Remove zebra.log from debian scripts and add optional log > levels > > URL? ? ? ? : > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714 > > Priority? : P5 - low > > Urgency? ? : enhancement > > Status? ? : RESOLVED > > > > For setting the log levels > > > > And > http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html > > > > For a general discussion of how to use them. > > > > ... This should give you some idea of what's failing, both in terms of the > > dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. > > > > My general feeling is that 2.0.59 is irredeemably broken by bug 16581, and > > we need at least 2.0.60, but I don't have any experience with zebra facets. > > > > --Barton > > > > > > > > On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett > <mtompset at hotmail.com> wrote: > >> > >> Greetings, > >> > >> Similar problem. I hope someone has a better solution than setting it to > >> 0. > >> > >> GPML, > >> Mark Tompsett > >> > >> -----Original Message----- > >> From: Philippe Blouin > >> Sent: Monday, August 22, 2016 2:40 PM > >> To: > koha-devel at lists.koha-community.org > >> Subject: [Koha-devel] 16.05, zebra and jessie > >> > >> Hello! > >> > >> We're trying to find the correction combination.? We're new on Jessie, > >> so we still have some tweaking needed... > >> > >> - By default, we get zebra 2.00.59 installed on Jessie through the > >> packages. > >> - On 16.05, we get some very bad results in the search when the itemtype > >> contains an hyphen (-), like 'A-DOC'. > >> - So we installed zebra 2.00.62.? This fixes the search... > >> - But now we do not have facets. > >> - So we set 0 > >> - And now we have facets.? But this feels... wrong? > >> > >> My dummy question: what is the supposedly correct version of Zebra on > >> Jessie ? > >> And we're we correct in setting the config to 0 ? > >> > >> Thanks > >> Blou > >> _______________________________________________ > >> Koha-devel mailing list > >> > Koha-devel at lists.koha-community.org > >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > >> website : http://www.koha-community.org/ > >> git : http://git.koha-community.org/ > >> bugs : http://bugs.koha-community.org/ > >> > >> _______________________________________________ > >> Koha-devel mailing list > >> > Koha-devel at lists.koha-community.org > >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > >> website : http://www.koha-community.org/ > >> git : http://git.koha-community.org/ > >> bugs : http://bugs.koha-community.org/ > > > > > > > > _______________________________________________ > > Koha-devel mailing list > > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ > > >_______________________________________________ >Koha-devel mailing list >Koha-devel at lists.koha-community.org >http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >website : http://www.koha-community.org/ >git : http://git.koha-community.org/ >bugs : http://bugs.koha-community.org/ --- Maritime heritage and history, preservation and conservation, research and education through the written word and the arts. and -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.druart at bugs.koha-community.org Tue Aug 23 16:57:43 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Tue, 23 Aug 2016 15:57:43 +0100 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: <5.2.1.1.2.20160823103433.05497cc0@pop.navalmarinearchive.com> References: <57BB471A.6070701@inlibro.com> <5.2.1.1.2.20160823103433.05497cc0@pop.navalmarinearchive.com> Message-ID: 2016-08-23 15:43 GMT+01:00 Paul A : > At 09:21 AM 8/23/2016 -0400, Barton Chittenden wrote: > > Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to > disk (~16G/query on large libraries). Bug? 13665 mentions that searches > could be taking upwards of 40 seconds to complete -- I think that we've seen > searches time out and return no results at about 1 minute. > > > When I looked into "speed" some time ago > , it was the fact that multi-core > processing was not possible (Intel 8-core I7 machine.) Speed is the only > reason that we decided to stay with 3.8.24 rather than upgrade. Could you stop propagating this argument and have a look at what has been done recently on the performance side? You should reconsider your choice and have a look at 16.05, or at least maintained releases. Unless you backport the security / bug fixes by yourself? Did you compare with 16.05.x with memcached + plack? That's what you should do, instead of comparing 3.8 vs 3.18 > Best -- Paul > > > Is it possible to tune Zebra's space/time optimizations in any way so that > it doesn't write such large files to disk? > > On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart > wrote: > See bug 13665 - Retrieve facets from zebra is slow > To understand why and when use_zebra_facet=1 is slow > > 2016-08-22 21:31 GMT+01:00 Barton Chittenden : >> I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I >> have run into this, when using ICU-Chains: >> >> Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 >> URL? ? ? ? : >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 >> Priority? : P5 - low >> Urgency? ? : enhancement >> Status? ? : NEW >> >> I also know that when use_zebra_facets was first introduced, it was *very* >> slow -- I can't find any bugs about that though. It's possible that it got >> so slow under idzebra-2.0 2.61 that the searches are timing out. >> >> It should be possible to set the logging for zebra so that you can see the >> PQF queries: >> >> See >> >> Bug 15714 : Remove zebra.log from debian scripts and add optional log >> levels >> URL? ? ? ? : >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714 >> Priority? : P5 - low >> Urgency? ? : enhancement >> Status? ? : RESOLVED >> >> For setting the log levels >> >> And >> http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html >> >> For a general discussion of how to use them. >> >> ... This should give you some idea of what's failing, both in terms of the >> dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. >> >> My general feeling is that 2.0.59 is irredeemably broken by bug 16581, and >> we need at least 2.0.60, but I don't have any experience with zebra >> facets. >> >> --Barton >> >> >> >> On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett >> wrote: >>> >>> Greetings, >>> >>> Similar problem. I hope someone has a better solution than setting it to >>> 0. >>> >>> GPML, >>> Mark Tompsett >>> >>> -----Original Message----- >>> From: Philippe Blouin >>> Sent: Monday, August 22, 2016 2:40 PM >>> To: koha-devel at lists.koha-community.org >>> Subject: [Koha-devel] 16.05, zebra and jessie >>> >>> Hello! >>> >>> We're trying to find the correction combination.? We're new on Jessie, >>> so we still have some tweaking needed... >>> >>> - By default, we get zebra 2.00.59 installed on Jessie through the >>> packages. >>> - On 16.05, we get some very bad results in the search when the itemtype >>> contains an hyphen (-), like 'A-DOC'. >>> - So we installed zebra 2.00.62.? This fixes the search... >>> - But now we do not have facets. >>> - So we set 0 >>> - And now we have facets.? But this feels... wrong? > >>> >>> My dummy question: what is the supposedly correct version of Zebra on >>> Jessie ? >>> And we're we correct in setting the config to 0 ? >>> >>> Thanks >>> Blou >>> _______________________________________________ >>> Koha-devel mailing list >>> Koha-devel at lists.koha-community.org >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>> website : http://www.koha-community.org/ >>> git : http://git.koha-community.org/ >>> bugs : http://bugs.koha-community.org/ >>> >>> _______________________________________________ >>> Koha-devel mailing list >>> Koha-devel at lists.koha-community.org >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>> website : http://www.koha-community.org/ >>> git : http://git.koha-community.org/ >>> bugs : http://bugs.koha-community.org/ >> >> >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > > --- > Maritime heritage and history, preservation and conservation, > research and education through the written word and the arts. > and > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From paul.a at navalmarinearchive.com Tue Aug 23 19:26:55 2016 From: paul.a at navalmarinearchive.com (Paul A) Date: Tue, 23 Aug 2016 13:26:55 -0400 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: References: <5.2.1.1.2.20160823103433.05497cc0@pop.navalmarinearchive.com> <57BB471A.6070701@inlibro.com> <5.2.1.1.2.20160823103433.05497cc0@pop.navalmarinearchive.com> Message-ID: <5.2.1.1.2.20160823130646.05497cc0@pop.navalmarinearchive.com> At 03:57 PM 8/23/2016 +0100, Jonathan Druart wrote: >2016-08-23 15:43 GMT+01:00 Paul A : > > At 09:21 AM 8/23/2016 -0400, Barton Chittenden wrote: > > > > Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to > > disk (~16G/query on large libraries). Bug?? 13665 mentions that searches > > could be taking upwards of 40 seconds to complete -- I think that we've > seen > > searches time out and return no results at about 1 minute. > > > > > > When I looked into "speed" some time ago > > , it was the fact that multi-core > > processing was not possible (Intel 8-core I7 machine.) Speed is the only > > reason that we decided to stay with 3.8.24 rather than upgrade. > >Could you stop propagating this argument and have a look at what has >been done recently on the performance side? >You should reconsider your choice and have a look at 16.05, or at >least maintained releases. >Unless you backport the security / bug fixes by yourself? > >Did you compare with 16.05.x with memcached + plack? That's what you >should do, instead of comparing 3.8 vs 3.18 Yes, I did. Please see my email to this list Tue, 28 Jun 2016 20:46:04 -0400. (8 core Intel I7, 16 Gigs RAM, SSD) -- and it's still between three and four times slower. Slight difference, it was on Ubuntu 16.04, but with MySQL re-installed to the "older" standard. I only "propagated" the 3.18 comparison because (a) I wrote it up, fully annotated, and it has all the NYTProf analyses, and (b) it makes the point about (not) multi-threading. I did not do such a full analysis with 16.05. And I do backport (or otherwise) fix security issues if and when I become aware of them... Best -- Paul > > Best -- Paul > > > > > > Is it possible to tune Zebra's space/time optimizations in any way so that > > it doesn't write such large files to disk? > > > > On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart > > wrote: > > See bug 13665 - Retrieve facets from zebra is slow > > To understand why and when use_zebra_facet=1 is slow > > > > 2016-08-22 21:31 GMT+01:00 Barton Chittenden : > >> I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I > >> have run into this, when using ICU-Chains: > >> > >> Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 > >> URL?? ?? ?? ?? : > >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 > >> Priority?? : P5 - low > >> Urgency?? ?? : enhancement > >> Status?? ?? : NEW > >> > >> I also know that when use_zebra_facets was first introduced, it was *very* > >> slow -- I can't find any bugs about that though. It's possible that it got > >> so slow under idzebra-2.0 2.61 that the searches are timing out. > >> > >> It should be possible to set the logging for zebra so that you can see the > >> PQF queries: > >> > >> See > >> > >> Bug 15714 : Remove zebra.log from debian scripts and add optional log > >> levels > >> URL?? ?? ?? ?? : > >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714 > >> Priority?? : P5 - low > >> Urgency?? ?? : enhancement > >> Status?? ?? : RESOLVED > >> > >> For setting the log levels > >> > >> And > >> http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html > >> > >> For a general discussion of how to use them. > >> > >> ... This should give you some idea of what's failing, both in terms of the > >> dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. > >> > >> My general feeling is that 2.0.59 is irredeemably broken by bug 16581, and > >> we need at least 2.0.60, but I don't have any experience with zebra > >> facets. > >> > >> --Barton > >> > >> > >> > >> On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett > >> wrote: > >>> > >>> Greetings, > >>> > >>> Similar problem. I hope someone has a better solution than setting it to > >>> 0. > >>> > >>> GPML, > >>> Mark Tompsett > >>> > >>> -----Original Message----- > >>> From: Philippe Blouin > >>> Sent: Monday, August 22, 2016 2:40 PM > >>> To: koha-devel at lists.koha-community.org > >>> Subject: [Koha-devel] 16.05, zebra and jessie > >>> > >>> Hello! > >>> > >>> We're trying to find the correction combination.?? We're new on Jessie, > >>> so we still have some tweaking needed... > >>> > >>> - By default, we get zebra 2.00.59 installed on Jessie through the > >>> packages. > >>> - On 16.05, we get some very bad results in the search when the itemtype > >>> contains an hyphen (-), like 'A-DOC'. > >>> - So we installed zebra 2.00.62.?? This fixes the search... > >>> - But now we do not have facets. > >>> - So we set 0 > >>> - And now we have facets.?? But this feels... wrong? > > > >>> > >>> My dummy question: what is the supposedly correct version of Zebra on > >>> Jessie ? > >>> And we're we correct in setting the config to 0 ? > >>> > >>> Thanks > >>> Blou > >>> _______________________________________________ > >>> Koha-devel mailing list > >>> Koha-devel at lists.koha-community.org > >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > >>> website : http://www.koha-community.org/ > >>> git : http://git.koha-community.org/ > >>> bugs : http://bugs.koha-community.org/ > >>> > >>> _______________________________________________ > >>> Koha-devel mailing list > >>> Koha-devel at lists.koha-community.org > >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > >>> website : http://www.koha-community.org/ > >>> git : http://git.koha-community.org/ > >>> bugs : http://bugs.koha-community.org/ > >> > >> > >> > >> _______________________________________________ > >> Koha-devel mailing list > >> Koha-devel at lists.koha-community.org > >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > >> website : http://www.koha-community.org/ > >> git : http://git.koha-community.org/ > >> bugs : http://bugs.koha-community.org/ > > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ > > > > --- > > Maritime heritage and history, preservation and conservation, > > research and education through the written word and the arts. > > and > > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ >_______________________________________________ >Koha-devel mailing list >Koha-devel at lists.koha-community.org >http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >website : http://www.koha-community.org/ >git : http://git.koha-community.org/ >bugs : http://bugs.koha-community.org/ --- Maritime heritage and history, preservation and conservation, research and education through the written word and the arts. and From tomascohen at gmail.com Tue Aug 23 20:25:05 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Tue, 23 Aug 2016 18:25:05 +0000 Subject: [Koha-devel] Move the AV code to the Koha namespace In-Reply-To: References: Message-ID: I can help with testing. El lun., 22 ago. 2016 a las 11:39, Jonathan Druart (< jonathan.druart at bugs.koha-community.org>) escribi?: > Hi devs, > > I have already started to move the authorised values related code to > the Koha namespace, but to continue I'd like to get feedback on the > method to continue. > I have asked you in the last "what's on in koha-devel" email, but did > not get any answer. > > Please see bug 15799 for the step-by-step plan. > > Cheers, > Jonathan > > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15799 > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From Katrin.Fischer.83 at web.de Tue Aug 23 22:25:56 2016 From: Katrin.Fischer.83 at web.de (Katrin Fischer) Date: Tue, 23 Aug 2016 22:25:56 +0200 Subject: [Koha-devel] Reminder: Development IRC Meeting - 24 August 2016, 7 UTC Message-ID: Hi all, just a quick reminder about tomorrow's Development IRC Meeting. The agenda and a link to the time converter for finding out your local meeting time can as always be found on the wiki: https://wiki.koha-community.org/wiki/Development_IRC_meeting_24_August_2016 Please add any topics you want to discuss! Hope to see you there, Katrin From larry at bywatersolutions.com Wed Aug 24 02:16:04 2016 From: larry at bywatersolutions.com (Larry Baerveldt) Date: Tue, 23 Aug 2016 17:16:04 -0700 Subject: [Koha-devel] Git server downtime for maintenance, Aug 24, 2016 Message-ID: Greetings, The Koha Community git server (git.koha-community.org) will be down for maintenance on Wed, Aug. 24, 2016, starting on or shortly after 7 pm PDT (2 am UTC). We cannot say for certain exactly how long the site will be down, but we hope it will be less than an hour. It would probably be best to avoid committing any updates to git during that hour. - Larry -- *Larry Baerveldt* Systems Administrator ByWater Solutions Support and Consulting for Open Source Software Headquarters: Santa Barbara, CA Office: Seattle, WA Phone/Fax (888)900-8944 http://bywatersolutions.com larry at bywatersolutions.com What is Koha? -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at bigballofwax.co.nz Wed Aug 24 23:40:57 2016 From: chris at bigballofwax.co.nz (Chris Cormack) Date: Thu, 25 Aug 2016 09:40:57 +1200 Subject: [Koha-devel] schema.koha-community.org In-Reply-To: References: Message-ID: Done http://schema.koha-community.org/master/ and http://schema.koha-community.org/16_05/ Chris On 23 August 2016 at 14:30, Chris Cormack wrote: > > > On 23/08/16 13:50, Mark Tompsett wrote: >> Greetings, >> >> Currently, it lists whatever is run as of a particular date. Is there a way to version it? >> There are currently 181 tables, not 175. So, this is clearly the difference between master and a release. >> >> https://schema.koha-community.org/{version}/... >> >> Just inquiring and thinking out loud. >> > > Yep, can definitely version it. > > I'll update the scripts tonight and see how we go > > Chris > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From mtompset at hotmail.com Thu Aug 25 13:38:26 2016 From: mtompset at hotmail.com (Mark Tompsett) Date: Thu, 25 Aug 2016 11:38:26 +0000 Subject: [Koha-devel] Sending Mail... In-Reply-To: References: <1472112084364-5900517.post@n5.nabble.com> <6a059783eeb14e03b231ba236c6399a0@gwp> Message-ID: Greetings, Jonathan Druart wrote on the Koha list: > When you claim issues, the email is sent directly and does not pass by > the message_queue table. This sounds like an area for massive refactorization: the standardization of sending mail. Does it not? GPML, Mark Tompsett From jonathan.druart at bugs.koha-community.org Thu Aug 25 14:14:41 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Thu, 25 Aug 2016 13:14:41 +0100 Subject: [Koha-devel] Sending Mail... In-Reply-To: References: <1472112084364-5900517.post@n5.nabble.com> <6a059783eeb14e03b231ba236c6399a0@gwp> Message-ID: Yes it does. Sending baskets and shelves also use the sendmail subroutine instead of adding a message to the message_queue table. 2016-08-25 12:38 GMT+01:00 Mark Tompsett : > Greetings, > > Jonathan Druart wrote on the Koha list: >> When you claim issues, the email is sent directly and does not pass by >> the message_queue table. > > This sounds like an area for massive refactorization: the standardization of > sending mail. Does it not? > > GPML, > Mark Tompsett > From jonathan.druart at bugs.koha-community.org Fri Aug 26 11:38:57 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Fri, 26 Aug 2016 10:38:57 +0100 Subject: [Koha-devel] biblioitems.marcxml & biblioitems.marc / HUGE performance issue ! In-Reply-To: <57851E32.5000202@biblibre.com> References: <57851E32.5000202@biblibre.com> Message-ID: For the record: Bug 10455 - remove redundant 'biblioitems.marc' field and Bug 17196 - Move marcxml out of the biblioitems table have now patches! 2016-07-12 17:43 GMT+01:00 Paul Poulain : > Hi all, > > Those days, we're working on a pretty large DB ( >1M biblio), for a customer > that want to do many statistics on some fields. > We discovered that something "simple" like: > SELECT publicationyear, count(publicationyear) FROM biblioitems GROUP BY > publicationyear; > > was giving no result in 10mn. > This is a test DB, not optimized, but we were surprised by the results. > After investigating we had the idea to create a biblioitems2 table with the > same structure EXCEPT MARCXML and MARC fields > > launch the same SQL query : result in 3seconds ! > This could be reproduced on any query (on fields without index). > > I think it's because the innoDB is storing each line in one "object", so, > even if you need only one column, you have to read everything. > In our case, that was 12GB+ of data to read. > biblioitems2 is just a few dozen MB. > (all caching values are minimum and there's no index, so not involved in the > results) > > MY CONCLUSIONS: > * the biblioitems.marc field must be removed quickly: it's useless since > years, and is only resulting in slowing things > * the bilbioitems.marcxml field should be moved outside from this table. > Something like biblio_blob, with biblionumber, biblioitemnumber and marcxml. > When we need it, just join the tables. > > I'm almost sure it would have an important impact on Koha, as biblioitems > table is called and used "everywhere". > > any opinion ? > > -- > Paul Poulain, Associ?-g?rant / co-owner > BibLibre, Services en logiciels libres pour les biblioth?ques > BibLibre, Open Source software and services for libraries > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From barton at bywatersolutions.com Fri Aug 26 15:25:59 2016 From: barton at bywatersolutions.com (Barton Chittenden) Date: Fri, 26 Aug 2016 09:25:59 -0400 Subject: [Koha-devel] schema.koha-community.org In-Reply-To: References: Message-ID: +1 On Wed, Aug 24, 2016 at 5:40 PM, Chris Cormack wrote: > Done > > http://schema.koha-community.org/master/ > and > http://schema.koha-community.org/16_05/ > > Chris > > On 23 August 2016 at 14:30, Chris Cormack wrote: > > > > > > On 23/08/16 13:50, Mark Tompsett wrote: > >> Greetings, > >> > >> Currently, it lists whatever is run as of a particular date. Is there a > way to version it? > >> There are currently 181 tables, not 175. So, this is clearly the > difference between master and a release. > >> > >> https://schema.koha-community.org/{version}/... > >> > >> Just inquiring and thinking out loud. > >> > > > > Yep, can definitely version it. > > > > I'll update the scripts tonight and see how we go > > > > Chris > > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirko at abunchofthings.net Fri Aug 26 16:57:00 2016 From: mirko at abunchofthings.net (Mirko Tietgen) Date: Fri, 26 Aug 2016 16:57:00 +0200 Subject: [Koha-devel] Release unification: Security fixes and releases Message-ID: <4a67dd66-bdda-bc9e-4c9d-148bfcae5353@abunchofthings.net> Hello everyone, can we maybe find a consistent way of labelling releases/ fixes "security"? All new releases contain XSS patches? - 3.20 and 3.22 are labelled "security release" in the emails/ release notes - 16.05 is not - 16.05 and 3.20 list the fixes under the headline "Critical bugs/ Koha - 3.22 as "Security bugs fixed" on top of the list - 3.22 features "security" in the release notes headline (and so in the URL in Wordpress) - The others don't The first question is probably? - Is a security release only a special release in between the regular schedule or - is a security release everything that contains patches with security fixes? It seems to me that the second option is more common within the Koha community and personally I'd prefer it. We should shout it at people and make them upgrade. To underline that intention, showing the security fixes seperately on top of the release notes makes sense to me. I propose to? - use the label "security release" in the release notes headline and first paragraph (and Wordpress URL) - mention number of security fixes seperately - display them under the headline "Security bugs fixed" on top of the list ?whenever there are patches from "Koha security" in a release. Basically that is what we have in the 3.22.10 release notes: https://koha-community.org/koha-3-22-10-security-release/ Does that make sense? Any reasons not to do it? Is it harming artistic freedom of RMaints? Would it complicate stuff a lot? I think it would be a good thing to have some consistency here. Cheers, Mirko -- Mirko Tietgen mirko at abunchofthings.net http://koha.abunchofthings.net http://meinkoha.de -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From frederic at tamil.fr Fri Aug 26 17:41:52 2016 From: frederic at tamil.fr (=?UTF-8?B?RnLDqWTDqXJpYyBEZW1pYW5z?=) Date: Fri, 26 Aug 2016 17:41:52 +0200 Subject: [Koha-devel] Release unification: Security fixes and releases In-Reply-To: <4a67dd66-bdda-bc9e-4c9d-148bfcae5353@abunchofthings.net> References: <4a67dd66-bdda-bc9e-4c9d-148bfcae5353@abunchofthings.net> Message-ID: > I propose to? > - use the label "security release" in the release notes headline and > first paragraph (and Wordpress URL) > - mention number of security fixes seperately > - display them under the headline "Security bugs fixed" on top of > the list > ?whenever there are patches from "Koha security" in a release. It seems that Julian do it this way with 3.20 branch. For 16.05, I use a script which generates automatically release notes. This script doesn't distinguish "security" patches from others. The script could be modified of course. I'd be curious to know how Julian creates his release notes, automatically or by hand, or a mix. Kind regards, -- Fr?d?ric DEMIANS http://www.tamil.fr/fdemians From julian.maurice at biblibre.com Mon Aug 29 08:58:47 2016 From: julian.maurice at biblibre.com (Julian Maurice) Date: Mon, 29 Aug 2016 08:58:47 +0200 Subject: [Koha-devel] Release unification: Security fixes and releases In-Reply-To: References: <4a67dd66-bdda-bc9e-4c9d-148bfcae5353@abunchofthings.net> Message-ID: On 26/08/2016 17:41, Fr?d?ric Demians wrote: >> I propose to? > >> - use the label "security release" in the release notes headline and >> first paragraph (and Wordpress URL) >> - mention number of security fixes seperately >> - display them under the headline "Security bugs fixed" on top of >> the list > >> ?whenever there are patches from "Koha security" in a release. > > It seems that Julian do it this way with 3.20 branch. > > For 16.05, I use a script which generates automatically release notes. This > script doesn't distinguish "security" patches from others. The script could be > modified of course. I'd be curious to know how Julian creates his release > notes, automatically or by hand, or a mix. > > Kind regards, > I believe I use the same script as you, but I make some minor changes by hand -- Julian Maurice BibLibre From frederic at tamil.fr Mon Aug 29 12:16:24 2016 From: frederic at tamil.fr (=?UTF-8?B?RnLDqWTDqXJpYyBEZW1pYW5z?=) Date: Mon, 29 Aug 2016 12:16:24 +0200 Subject: [Koha-devel] Release unification: Security fixes and releases In-Reply-To: References: <4a67dd66-bdda-bc9e-4c9d-148bfcae5353@abunchofthings.net> Message-ID: If there is a consensus on this point, I can modify the script which generates release notes in order to have the notes the way Julian formats them. From magnus at enger.priv.no Tue Aug 30 13:23:55 2016 From: magnus at enger.priv.no (Magnus Enger) Date: Tue, 30 Aug 2016 13:23:55 +0200 Subject: [Koha-devel] Sessions terminated at random under Plack Message-ID: Dear Community, I am seeing a strange problem, and I'm not sure where to start digging. I have a (big) server with 30 odd Koha instances. One of these instances has been running under Plack for quite some time now, without any problems. Two new instances have a problem where librarians get kicked out of the intranet more or less frequently, with a message like "Your session has expired, please log in again". We had this problem on 3.22.x. After upgrading to 16.05.02 it went away. After upgrading to 16.05.03 last night it came back again. It is not consistent. Earlier today it looked like you could log in, click on a link, get thrown out, login again and then things would work. Later users got kicked out every single time they clicked on a link in the intranet. All sites are running off the official Debian packages, on the same server. Memcached is installed, enabled and used for storing sessions. Switching SessionStorage to the DB does not stop the problem. Stopping and disabling Plack for these two instances makes the problem go away. I have not found anything interesting in Plack or Apache logs. Anyone got a hunch what might be causing this? Or where to start digging? Best regards, Magnus From oleonard at myacpl.org Tue Aug 30 20:00:02 2016 From: oleonard at myacpl.org (Owen Leonard) Date: Tue, 30 Aug 2016 14:00:02 -0400 Subject: [Koha-devel] Interface patterns (tabs) In-Reply-To: <00f201d1f6be$e09204a0$a1b60de0$@prosentient.com.au> References: <00f201d1f6be$e09204a0$a1b60de0$@prosentient.com.au> Message-ID: > Do you have any documentation for preferred use of jQuery tabs? Hi David, Sorry to have let your question hang for so long! If I had to describe how tabs are generally used in Koha I would say that they are used to present distinct sub-categories of information about a thing in a way which conserves space. I consider the bibliographic detail page to be the best example. There are other pages like members/pay.pl which use "static" tabs. These are tabs which look like jQueryUI tabs but are actually links to separate pages. That doesn't feel very consistent to me but it seems to work. I realized I'd forgotten it when I looked again at Bug 12773 ("Branch transfer limits show codes instead of descriptions (item types/collections)"). The transfer limits page uses tabs to allow for fast switching between collection code configurations. In my library's system the tabs get a little crazy: http://zivotdesign.com/p/#1/14725785555465 Of course we may simply have too many collection codes! I think the screenshot is a strong hint that tabs are not the correct way to be handling that interface. I would be interested to hear if anyone has other examples of where a tabbed interface becomes unwieldy because of a large number of generated tabs. David does that answer your question? -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org From tomascohen at gmail.com Tue Aug 30 20:40:39 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Tue, 30 Aug 2016 18:40:39 +0000 Subject: [Koha-devel] Sessions terminated at random under Plack In-Reply-To: References: Message-ID: Magnus, there's been a lot of movement on the caching layer, and some of that work has been backported to the stable releases. Also, the RestrictSessionByIP setting was getting in the middle ( https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17050). There's also a problem with memcached initialization in C4::Context that makes sessionStorage=memcache fail to persist sessions. My suggestion would be to set sessionStorage to 'mysql' and disable the RestricSessionByIP syspref. Regards El mar., 30 ago. 2016 a las 8:24, Magnus Enger () escribi?: > Dear Community, > > I am seeing a strange problem, and I'm not sure where to start digging. > > I have a (big) server with 30 odd Koha instances. One of these > instances has been running under Plack for quite some time now, > without any problems. > > Two new instances have a problem where librarians get kicked out of > the intranet more or less frequently, with a message like "Your > session has expired, please log in again". > > We had this problem on 3.22.x. > After upgrading to 16.05.02 it went away. > After upgrading to 16.05.03 last night it came back again. > > It is not consistent. Earlier today it looked like you could log in, > click on a link, get thrown out, login again and then things would > work. Later users got kicked out every single time they clicked on a > link in the intranet. > > All sites are running off the official Debian packages, on the same server. > Memcached is installed, enabled and used for storing sessions. > Switching SessionStorage to the DB does not stop the problem. > Stopping and disabling Plack for these two instances makes the problem go > away. > > I have not found anything interesting in Plack or Apache logs. > > Anyone got a hunch what might be causing this? Or where to start digging? > > Best regards, > Magnus > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisc at catalyst.net.nz Tue Aug 30 20:44:32 2016 From: chrisc at catalyst.net.nz (Chris Cormack) Date: Wed, 31 Aug 2016 06:44:32 +1200 Subject: [Koha-devel] Sessions terminated at random under Plack In-Reply-To: References: Message-ID: <98C52795-5225-4342-AC5D-4852DA161A08@catalyst.net.nz> You will need to be aware though that reduces your users protection from session hijacking tremendously. We really need to make fixing it a priority, without reducing security. Chris On 31 August 2016 6:40:39 AM NZST, Tomas Cohen Arazi wrote: >Magnus, there's been a lot of movement on the caching layer, and some >of >that work has been backported to the stable releases. Also, the >RestrictSessionByIP setting was getting in the middle ( >https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17050). > >There's also a problem with memcached initialization in C4::Context >that >makes sessionStorage=memcache fail to persist sessions. > >My suggestion would be to set sessionStorage to 'mysql' and disable the >RestricSessionByIP syspref. > >Regards > > > >El mar., 30 ago. 2016 a las 8:24, Magnus Enger () >escribi?: > >> Dear Community, >> >> I am seeing a strange problem, and I'm not sure where to start >digging. >> >> I have a (big) server with 30 odd Koha instances. One of these >> instances has been running under Plack for quite some time now, >> without any problems. >> >> Two new instances have a problem where librarians get kicked out of >> the intranet more or less frequently, with a message like "Your >> session has expired, please log in again". >> >> We had this problem on 3.22.x. >> After upgrading to 16.05.02 it went away. >> After upgrading to 16.05.03 last night it came back again. >> >> It is not consistent. Earlier today it looked like you could log in, >> click on a link, get thrown out, login again and then things would >> work. Later users got kicked out every single time they clicked on a >> link in the intranet. >> >> All sites are running off the official Debian packages, on the same >server. >> Memcached is installed, enabled and used for storing sessions. >> Switching SessionStorage to the DB does not stop the problem. >> Stopping and disabling Plack for these two instances makes the >problem go >> away. >> >> I have not found anything interesting in Plack or Apache logs. >> >> Anyone got a hunch what might be causing this? Or where to start >digging? >> >> Best regards, >> Magnus >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ >> >-- >Tom?s Cohen Arazi >Theke Solutions (https://theke.io ) >? +54 9351 3513384 >GPG: B2F3C15F > > >------------------------------------------------------------------------ > >_______________________________________________ >Koha-devel mailing list >Koha-devel at lists.koha-community.org >http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >website : http://www.koha-community.org/ >git : http://git.koha-community.org/ >bugs : http://bugs.koha-community.org/ -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomascohen at gmail.com Tue Aug 30 20:48:00 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Tue, 30 Aug 2016 18:48:00 +0000 Subject: [Koha-devel] Sessions terminated at random under Plack In-Reply-To: <98C52795-5225-4342-AC5D-4852DA161A08@catalyst.net.nz> References: <98C52795-5225-4342-AC5D-4852DA161A08@catalyst.net.nz> Message-ID: Chris, of course! We are talking about debugging. The patches that solve the issue are already pushed! El mar., 30 ago. 2016 a las 15:44, Chris Cormack () escribi?: > You will need to be aware though that reduces your users protection from > session hijacking tremendously. We really need to make fixing it a > priority, without reducing security. > > Chris > > > On 31 August 2016 6:40:39 AM NZST, Tomas Cohen Arazi > wrote: >> >> Magnus, there's been a lot of movement on the caching layer, and some of >> that work has been backported to the stable releases. Also, the >> RestrictSessionByIP setting was getting in the middle ( >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17050). >> >> There's also a problem with memcached initialization in C4::Context that >> makes sessionStorage=memcache fail to persist sessions. >> >> My suggestion would be to set sessionStorage to 'mysql' and disable the >> RestricSessionByIP syspref. >> >> Regards >> >> >> >> El mar., 30 ago. 2016 a las 8:24, Magnus Enger () >> escribi?: >> >>> Dear Community, >>> >>> I am seeing a strange problem, and I'm not sure where to start digging. >>> >>> I have a (big) server with 30 odd Koha instances. One of these >>> instances has been running under Plack for quite some time now, >>> without any problems. >>> >>> Two new instances have a problem where librarians get kicked out of >>> the intranet more or less frequently, with a message like "Your >>> session has expired, please log in again". >>> >>> We had this problem on 3.22.x. >>> After upgrading to 16.05.02 it went away. >>> After upgrading to 16.05.03 last night it came back again. >>> >>> It is not consistent. Earlier today it looked like you could log in, >>> click on a link, get thrown out, login again and then things would >>> work. Later users got kicked out every single time they clicked on a >>> link in the intranet. >>> >>> All sites are running off the official Debian packages, on the same >>> server. >>> Memcached is installed, enabled and used for storing sessions. >>> Switching SessionStorage to the DB does not stop the problem. >>> Stopping and disabling Plack for these two instances makes the problem >>> go away. >>> >>> I have not found anything interesting in Plack or Apache logs. >>> >>> Anyone got a hunch what might be causing this? Or where to start digging? >>> >>> Best regards, >>> Magnus >>> _______________________________________________ >>> Koha-devel mailing list >>> Koha-devel at lists.koha-community.org >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>> website : http://www.koha-community.org/ >>> git : http://git.koha-community.org/ >>> bugs : http://bugs.koha-community.org/ >>> >> -- >> Tom?s Cohen Arazi >> Theke Solutions (https://theke.io ) >> ? +54 9351 3513384 >> GPG: B2F3C15F >> >> ------------------------------ >> >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ >> >> > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomascohen at gmail.com Tue Aug 30 22:11:18 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Tue, 30 Aug 2016 20:11:18 +0000 Subject: [Koha-devel] Koha::* business objects authorization check Message-ID: Hi everyone, adding to the discussion about Koha::Objects I have a subject that needs broad (an urgent) discussion. The REST api patches are shaping up, and I'd like to focus on an important one: 14868 [1]. This bug introduces a way of specifying on the Swagger files, the required privileges to do stuff on objects. For example: + "x-koha-authorization": { + "allow-owner": true, + "allow-guarantor": true, + "permissions": { + "borrowers": "1" + } } specifies that to get a patron object you need to either have the 'borrowers' permission, be the object owner, or the guarantor of the object's owner. This is really nice, because permissions definition are now higher level entities (just a configuration thing), and they are part of the API documentation. There's even a bug with patches that introduce required permissions in the error messages. The current implementation introduces checks for each object class on the controller code. My feeling is that each business object should be responsible for such a check. The first step I made was to extend Koha::Object so it exposes an ->owner method that returns the Koha::Patron object that owns the object (because we can get the guarantees from there too). I did this to eliminate the need of class-specific checks on the controller. But at the end of the day, this still didn't feel it was good enough. I belive such a business matter should not be so tied to the controller/routing code and it is still there. Talking with Jonathan we thought we could make Koha::Objects->search accept more parameters (current userid, required permissions) and have the code checking authorization there, basically creating a filter on the search. The main problem is how to propagate the swagger-specified required permissions down to the Koha::Object in a clean way. I'd like to hear how people belive we should handle on this new+fresh code permissions check, so we have good foundations for the future of the project. Cleaning untested business logic from controller scripts is a good start! And this is the time to do it! Thanks in advance PS. I'm sure Jonathan will explain the discussion better once he comes back from the pub :-D [1] https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14868 -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisc at catalyst.net.nz Tue Aug 30 22:23:05 2016 From: chrisc at catalyst.net.nz (Chris Cormack) Date: Wed, 31 Aug 2016 08:23:05 +1200 Subject: [Koha-devel] Koha::* business objects authorization check In-Reply-To: References: Message-ID: <20160830202305.tples2og7k2ljc6l@rorohiko.wgtn.cat-it.co.nz> * Tomas Cohen Arazi (tomascohen at gmail.com) wrote: > Hi everyone, adding to the discussion about Koha::Objects I have a subject that > needs broad (an urgent) discussion. > > The REST api patches are shaping up, and I'd like to focus on an important one: > 14868 [1]. > > This bug introduces a way of specifying on the Swagger files, the required > privileges to do stuff on objects. For example: > > > + "x-koha-authorization": { > + "allow-owner": true, > + "allow-guarantor": true, > + "permissions": { > + "borrowers": "1" > + } > } > > specifies that to get a patron object you need to either have the 'borrowers' > permission, be the object owner, or the guarantor of the object's owner. > > This is really nice, because permissions definition are now higher level > entities (just a configuration thing), and they are part of the API > documentation. There's even a bug with patches that introduce required > permissions in the error messages. > > The current implementation introduces checks for each object class on the > controller code. > > My feeling is that each business object should be responsible for such a check. > The first step I made was to extend Koha::Object so it exposes an ->owner > method that returns the Koha::Patron object that owns the object (because we > can get the guarantees from there too). I like it, it is only when we do things like this that the Objects become useful, otherwise they are a shim over db access (and a slow shim at that). Once they do more things like this, they will be a lot more useful. Chris > I did this to eliminate the need of class-specific checks on the controller. > But at the end of the day, this still didn't feel it was good enough. I belive > such a business matter should not be so tied to the controller/routing code and > it is still there. > > Talking with Jonathan we thought we could make Koha::Objects->search accept > more parameters (current userid, required permissions) and have the code > checking authorization there, basically creating a filter on the search. The > main problem is how to propagate the swagger-specified required permissions > down to the Koha::Object in a clean way. > > I'd like to hear how people belive we should handle on this new+fresh code > permissions check, so we have good foundations for the future of the project. > Cleaning untested business logic from controller scripts is a good start! And > this is the time to do it! > > Thanks in advance > > PS. I'm sure Jonathan will explain the discussion better once he comes back > from the pub :-D > > [1]?https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14868 > -- > Tom?s Cohen Arazi > Theke Solutions (https://theke.io) > ? +54 9351 3513384 > GPG: B2F3C15F > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ -- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: not available URL: From indradg at gmail.com Wed Aug 31 00:36:02 2016 From: indradg at gmail.com (Indranil Das Gupta) Date: Wed, 31 Aug 2016 04:06:02 +0530 Subject: [Koha-devel] GoogleIndicTransliteration must die? Message-ID: Hi all, Koha uses the deprecated (in 2011) Google Indic Transliterate v1 API to power the GoogleIndicTransliterate syspref. It seems internally the API calls to Googe Inputtools service over http! So again we are back in MIXED CONTENT mess and nothing much we can do about it, except move out of this API. transliteration.I.js:formatted:11055 Mixed Content: The page at 'https://rksmvvlibrary.in/cgi-bin/koha/opac-search.pl?q=%E0%A6%B6%E0%A7%8D%E?E%E0%A6%AE%E0%A6%BE%E0%A6%AA%E0%A7%8D%E0%A6%B0%E0%A6%B8%E0%A6%BE%E0%A6%A6+' was loaded over HTTPS, but requested an insecure script 'http://www.google.com/inputtools/request?text=shyama&ime=transliteration_en?n&num=5&cp=0&cs=0&ie=utf-8&oe=utf-8&app=jsapi&uv&cb=_callbacks_._0isi0aei6'. This request has been blocked; the content must be served over HTTPS.Q.dh @ transliteration.I.js:formatted:11055Ww @ transliteration.I.js:formatted:4578Q.Ad @ transliteration.I.js:formatted:10982(anonymous function) @ transliteration.I.js:formatted:11237(anonymous function) @ transliteration.I.js:formatted:6670(anonymous function) @ transliteration.I.js:formatted:6059nq @ transliteration.I.js:formatted:2266(anonymous function) @ transliteration.I.js:formatted:6232a @ transliteration.I.js:formatted:6074 transliteration.I.js:formatted:10923 Uncaught TypeError: Cannot read property '0' of nullQ.Bh @ transliteration.I.js:formatted:10923Q.Mi @ transliteration.I.js:formatted:10862Mt.(anonymous function).j @ transliteration.I.js:formatted:8324(anonymous function) @ transliteration.I.js:formatted:3591 I saw this http://stackoverflow.com/questions/35238710/deprecated-google-api-will-not-work-from-page-served-over-https and this https://groups.google.com/forum/?fromgroups#!topic/google-ajax-search-api/9TqbCXqQTSA Looks like its time to bid goodbye to GoogleIndicTransliteration syspref :-( -- Indranil Das Gupta L2C2 Technologies Phone : +91-98300-20971 Blog : http://blog.l2c2.co.in IRC : indradg on irc://irc.freenode.net Twitter : indradg From dcook at prosentient.com.au Wed Aug 31 00:44:03 2016 From: dcook at prosentient.com.au (David Cook) Date: Wed, 31 Aug 2016 08:44:03 +1000 Subject: [Koha-devel] Interface patterns (tabs) In-Reply-To: References: <00f201d1f6be$e09204a0$a1b60de0$@prosentient.com.au> Message-ID: <004601d20310$01cdf2c0$0569d840$@prosentient.com.au> Hi Owen, No worries. I haven't been checking my koha-devel mail for a little while, so it's quite the coincidence I found your message today! I suppose I was looking for an established guideline or a recommendation, but I appreciate your input nonetheless and it does answer the question! That is a lot of collection codes! That does seem to suggest that a dropdown menu might be better than tabs for that interface, I guess? I'm working on an interface for the OAI-PMH import at the moment, and I'm unsure how best to group the content. I don't want to make a bunch of subpages for all the different tables, especially as some are related and thus one would want to be able to rapidly switch back and forth, but I'm not sure that tabs are the answer either. But at the moment they stretch down the page pretty far if there's a lot of data. Maybe I can use tabs to group together the most related information, and then maybe make another page for the most tangentially related content. Sort of like "Stage MARC for import" and "Staged MARC management". They're different enough to warrant separate pages, I suppose. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 > -----Original Message----- > From: koha-devel-bounces at lists.koha-community.org [mailto:koha-devel- > bounces at lists.koha-community.org] On Behalf Of Owen Leonard > Sent: Wednesday, 31 August 2016 4:00 AM > To: koha-devel at lists.koha-community.org > Subject: Re: [Koha-devel] Interface patterns (tabs) > > > Do you have any documentation for preferred use of jQuery tabs? > > Hi David, > > Sorry to have let your question hang for so long! > > If I had to describe how tabs are generally used in Koha I would say that they > are used to present distinct sub-categories of information about a thing in a > way which conserves space. I consider the bibliographic detail page to be the > best example. > > There are other pages like members/pay.pl which use "static" tabs. > These are tabs which look like jQueryUI tabs but are actually links to separate > pages. That doesn't feel very consistent to me but it seems to work. > > I realized I'd forgotten it when I looked again at Bug 12773 ("Branch transfer > limits show codes instead of descriptions (item types/collections)"). The > transfer limits page uses tabs to allow for fast switching between collection > code configurations. In my library's system the tabs get a little crazy: > > http://zivotdesign.com/p/#1/14725785555465 > > Of course we may simply have too many collection codes! I think the > screenshot is a strong hint that tabs are not the correct way to be handling > that interface. I would be interested to hear if anyone has other examples of > where a tabbed interface becomes unwieldy because of a large number of > generated tabs. > > David does that answer your question? > > -- Owen > > -- > Web Developer > Athens County Public Libraries > http://www.myacpl.org > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ git : http://git.koha- > community.org/ bugs : http://bugs.koha-community.org/ From dcook at prosentient.com.au Wed Aug 31 00:58:04 2016 From: dcook at prosentient.com.au (David Cook) Date: Wed, 31 Aug 2016 08:58:04 +1000 Subject: [Koha-devel] biblioitems.marcxml & biblioitems.marc / HUGE performance issue ! In-Reply-To: References: <57851E32.5000202@biblibre.com> Message-ID: <004701d20311$f716f550$e544dff0$@prosentient.com.au> Excitement! I've been looking forward to patches for those bugs for years. I don't have the time to test them now, but hoping that someone does soon! Of course, moving biblioitems.marcxml will break a lot of SQL reports I'm sure, but I think it's worth it. The broken reports should generate noisy errors, so it's not like they would silently fail. And there comes a time where you just need to move on to something new. I'm curious about our "table","deletedtable" methodology. I suppose by moving data to "deletedtable" for historical purposes, we're theoretically improving performance for "table" and making queries simple for active records. I think we lose data integrity sometimes (more so with issues and reserves than records) by moving data to historical tables. I suppose we're entrenched in that design now though, so discussion would be a bit moot. Anyway, hurray for Jonathan adding patches! David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 > -----Original Message----- > From: koha-devel-bounces at lists.koha-community.org [mailto:koha-devel- > bounces at lists.koha-community.org] On Behalf Of Jonathan Druart > Sent: Friday, 26 August 2016 7:39 PM > To: koha-devel at lists.koha-community.org > Subject: Re: [Koha-devel] biblioitems.marcxml & biblioitems.marc / HUGE > performance issue ! > > For the record: > Bug 10455 - remove redundant 'biblioitems.marc' field and > Bug 17196 - Move marcxml out of the biblioitems table have now patches! > > 2016-07-12 17:43 GMT+01:00 Paul Poulain : > > Hi all, > > > > Those days, we're working on a pretty large DB ( >1M biblio), for a > > customer that want to do many statistics on some fields. > > We discovered that something "simple" like: > > SELECT publicationyear, count(publicationyear) FROM biblioitems GROUP > > BY publicationyear; > > > > was giving no result in 10mn. > > This is a test DB, not optimized, but we were surprised by the results. > > After investigating we had the idea to create a biblioitems2 table > > with the same structure EXCEPT MARCXML and MARC fields > > > > launch the same SQL query : result in 3seconds ! > > This could be reproduced on any query (on fields without index). > > > > I think it's because the innoDB is storing each line in one "object", > > so, even if you need only one column, you have to read everything. > > In our case, that was 12GB+ of data to read. > > biblioitems2 is just a few dozen MB. > > (all caching values are minimum and there's no index, so not involved > > in the > > results) > > > > MY CONCLUSIONS: > > * the biblioitems.marc field must be removed quickly: it's useless > > since years, and is only resulting in slowing things > > * the bilbioitems.marcxml field should be moved outside from this table. > > Something like biblio_blob, with biblionumber, biblioitemnumber and > marcxml. > > When we need it, just join the tables. > > > > I'm almost sure it would have an important impact on Koha, as > > biblioitems table is called and used "everywhere". > > > > any opinion ? > > > > -- > > Paul Poulain, Associ?-g?rant / co-owner BibLibre, Services en > > logiciels libres pour les biblioth?ques BibLibre, Open Source software > > and services for libraries > > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ git : > > http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ git : http://git.koha- > community.org/ bugs : http://bugs.koha-community.org/ From dcook at prosentient.com.au Wed Aug 31 01:19:20 2016 From: dcook at prosentient.com.au (David Cook) Date: Wed, 31 Aug 2016 09:19:20 +1000 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: References: <57BB471A.6070701@inlibro.com> Message-ID: <004801d20314$ef8e37f0$ceaaa7d0$@prosentient.com.au> If I had more time, I?d patch Zebra? It?s been a while since I looked at this. Is the problem that no facets are returned if one of them would return zero, or that no facets are returned if one of the fields specified in the facet browse is an empty index/un-indexed field in Zebra? If it?s the latter, I wonder if using the Zebra specific search extension ?14? in the main query would help? (see ?Specifies whether un-indexed fields should be ignored. A zero value (default) throws a diagnostic when an un-indexed field is specified. A non-zero value makes it return 0 hits.? http://www.indexdata.com/zebra/doc/zebra.pdf). I suppose Adam at IndexData has been busy with the FOLIO project, so I doubt he has time to work on Zebra these days, even if we did have a patch available. Is ElasticSearch usable with Koha at this point? I heard a lot in 2015, but after Robin left I haven?t heard a word other than rumours that the patches had been pushed? David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: koha-devel-bounces at lists.koha-community.org [mailto:koha-devel-bounces at lists.koha-community.org] On Behalf Of Tomas Cohen Arazi Sent: Tuesday, 23 August 2016 11:42 PM To: Barton Chittenden ; Jonathan Druart Cc: koha-devel at lists.koha-community.org Subject: Re: [Koha-devel] 16.05, zebra and jessie I have seen use_zebra_facets=1 cause no facets rendered when GRS-1 configuration files are kept during upgrades up to where GRS-1 got deprecated (3.20?). Is it the case? What does the About > System information page says about your config? The slowliness is not in zebra per se, but in the way we retrieve the facets from it (so Koha/Perl side). We retrieve each facet at a time instead of fetching them all in one call. And they come in XML format, so need to be parsed. So, if anyone is willing to improve it, just need to optimize this function (read the TODO): sub _get_facet_from_result_set { my $facet_idx = shift; my $rs = shift; my $sep = shift; my $internal_sep = '<*>'; my $facetMaxCount = C4::Context->preference('FacetMaxCount') // 20; return if ( ! defined $facet_idx || ! defined $rs ); # zebra's facet element, untokenized index my $facet_element = 'zebra::facet::' . $facet_idx . ':0:' . $facetMaxCount; # configure zebra results for retrieving the desired facet $rs->option( elementSetName => $facet_element ); # get the facet record from result set my $facet = $rs->record( 0 )->raw; # if the facet has no restuls... return if !defined $facet; # TODO: benchmark DOM vs. SAX performance my $facet_dom = XML::LibXML->load_xml( string => ($facet) ); my @terms = $facet_dom->getElementsByTagName('term'); return if ! @terms; my $facets = {}; foreach my $term ( @terms ) { my $facet_value = $term->textContent; $facet_value =~ s/\Q$internal_sep\E/$sep/ if defined $sep; $facets->{ $facet_value } = $term->getAttribute( 'occur' ); } return $facets; } Another option would be to make _get_facets_from_zebra build the element set containing all facets so they are read in one call (comma-separate all elements). The problem is that Zebra returns zero if one of the elements is empty. Jared proposed to create a ghost record with all facet fields. I didn't manage to make it work. Another option is to patch Zebra. I started that, but abandoned once the ES code got pushed. So, if use_zebra_facets=0 is good enough, maybe it should be recommended. Problem is it is not a real facet, but the sole extraction of the fields from the first x records. As I said, it could be good enough anyway. Regards El mar., 23 ago. 2016 a las 10:21, Barton Chittenden ( >) escribi?: Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to disk (~16G/query on large libraries). Bug 13665 mentions that searches could be taking upwards of 40 seconds to complete -- I think that we've seen searches time out and return no results at about 1 minute. Is it possible to tune Zebra's space/time optimizations in any way so that it doesn't write such large files to disk? On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart > wrote: See bug 13665 - Retrieve facets from zebra is slow To understand why and when use_zebra_facet=1 is slow 2016-08-22 21:31 GMT+01:00 Barton Chittenden >: > I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I > have run into this, when using ICU-Chains: > > Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 > URL : https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 > Priority : P5 - low > Urgency : enhancement > Status : NEW > > I also know that when use_zebra_facets was first introduced, it was *very* > slow -- I can't find any bugs about that though. It's possible that it got > so slow under idzebra-2.0 2.61 that the searches are timing out. > > It should be possible to set the logging for zebra so that you can see the > PQF queries: > > See > > Bug 15714 : Remove zebra.log from debian scripts and add optional log levels > URL : https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714 > Priority : P5 - low > Urgency : enhancement > Status : RESOLVED > > For setting the log levels > > And http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html > > For a general discussion of how to use them. > > ... This should give you some idea of what's failing, both in terms of the > dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. > > My general feeling is that 2.0.59 is irredeemably broken by bug 16581, and > we need at least 2.0.60, but I don't have any experience with zebra facets. > > --Barton > > > > On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett > wrote: >> >> Greetings, >> >> Similar problem. I hope someone has a better solution than setting it to >> 0. >> >> GPML, >> Mark Tompsett >> >> -----Original Message----- >> From: Philippe Blouin >> Sent: Monday, August 22, 2016 2:40 PM >> To: koha-devel at lists.koha-community.org >> Subject: [Koha-devel] 16.05, zebra and jessie >> >> Hello! >> >> We're trying to find the correction combination. We're new on Jessie, >> so we still have some tweaking needed... >> >> - By default, we get zebra 2.00.59 installed on Jessie through the >> packages. >> - On 16.05, we get some very bad results in the search when the itemtype >> contains an hyphen (-), like 'A-DOC'. >> - So we installed zebra 2.00.62. This fixes the search... >> - But now we do not have facets. >> - So we set 0 >> - And now we have facets. But this feels... wrong? >> >> My dummy question: what is the supposedly correct version of Zebra on >> Jessie ? >> And we're we correct in setting the config to 0 ? >> >> Thanks >> Blou >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ > > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ _______________________________________________ Koha-devel mailing list Koha-devel at lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcook at prosentient.com.au Wed Aug 31 01:21:19 2016 From: dcook at prosentient.com.au (David Cook) Date: Wed, 31 Aug 2016 09:21:19 +1000 Subject: [Koha-devel] What's on in Koha devel #3 (RESTful API) In-Reply-To: References: Message-ID: <004f01d20315$36c4f640$a44ee2c0$@prosentient.com.au> +1 The REST API is just a public interface. It should use the same internal APIs as the rest of Koha. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 From: koha-devel-bounces at lists.koha-community.org [mailto:koha-devel-bounces at lists.koha-community.org] On Behalf Of Kyle Hall Sent: Wednesday, 17 August 2016 4:31 AM To: Marcel de Rooy Cc: koha-devel at lists.koha-community.org Subject: Re: [Koha-devel] What's on in Koha devel #3 (RESTful API) In my mind, we should be moving all appropriate logic to Koha/Objects with Koha/REST being just an api wrapper around the appropriate object. That will keep us from duplicating code and reduce bugs. Kyle http://www.kylehall.info ByWater Solutions ( http://bywatersolutions.com ) Meadville Public Library ( http://www.meadvillelibrary.org ) Crawford County Federated Library System ( http://www.ccfls.org ) On Mon, Aug 15, 2016 at 3:28 AM, Marcel de Rooy > wrote: > = RESTful API = > I also would like to know who is willing to maintain this code? For instance on bug 16330 (REST API: add routes to add, update and delete > patrons) the delete route is calling > # check if loans, reservations, debarrment, etc. before deletion! > my $res = $patron->delete > So no check is done from the API. > But if bug 16907 (Move DelMember and HandleDelBorrower to > Koha::Patron) is pushed, $patron->delete will take care of the holds, etc. My question is simple: are we expecting from devs, signoffers and > QAers to update and maintain these routes? > Or a dedicated person will update them before each release? Just formally, I would say that if the community agreed to push this code, the community should maintain it too. We should keep the code base in a consistent state. Similarly, we should prevent redundant code: the checks for deleting a patron in one place, etc. If we are on a road now creating a lot of redundant code between Koha/REST and Koha/Objects, we should re-evaluate first and not just keep pushing more code. Is this mainly a communication issue? Which goals do we want to achieve? Should our scripts go via REST some day and should all business logic remain in Objects? Etc. etc. Marcel _______________________________________________ Koha-devel mailing list Koha-devel at lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisc at catalyst.net.nz Wed Aug 31 01:27:46 2016 From: chrisc at catalyst.net.nz (Chris Cormack) Date: Wed, 31 Aug 2016 11:27:46 +1200 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: <004801d20314$ef8e37f0$ceaaa7d0$@prosentient.com.au> References: <57BB471A.6070701@inlibro.com> <004801d20314$ef8e37f0$ceaaa7d0$@prosentient.com.au> Message-ID: <20160830232746.sjyidue6fjkwt3ld@rorohiko.wgtn.cat-it.co.nz> * David Cook (dcook at prosentient.com.au) wrote: > > > > I suppose Adam at IndexData has been busy with the FOLIO project, so I doubt he > has time to work on Zebra these days, even if we did have a patch available. > > > > Is ElasticSearch usable with Koha at this point? I heard a lot in 2015, but > after Robin left I haven?t heard a word other than rumours that the patches had > been pushed? You missed tons of words then :) Yes, it is in 16.05, marked expiremental, it works, mostly. But it will only get better with more people using and fixing it. The next task is to update the version it works with to Elastic 2. That isn't a huge amount of work, but everyone has their own priorities and a lot of us have to work on what users ask for (not what users need ;)) Chris > > Subject: Re: [Koha-devel] 16.05, zebra and jessie > > > > I have seen use_zebra_facets=1 cause no facets rendered when GRS-1 > configuration files are kept during upgrades up to where GRS-1 got deprecated > (3.20?). Is it the case? What does the About > System information page says > about your config? > > > > The slowliness is not in zebra per se, but in the way we retrieve the facets > from it (so Koha/Perl side). We retrieve each facet at a time instead of > fetching them all in one call. And they come in XML format, so need to be > parsed. So, if anyone is willing to improve it, just need to optimize this > function (read the TODO): > > > > sub _get_facet_from_result_set { > > > > my $facet_idx = shift; > > my $rs = shift; > > my $sep = shift; > > > > my $internal_sep = '<*>'; > > my $facetMaxCount = C4::Context->preference('FacetMaxCount') // 20; > > > > return if ( ! defined $facet_idx || ! defined $rs ); > > # zebra's facet element, untokenized index > > my $facet_element = 'zebra::facet::' . $facet_idx . ':0:' . $facetMaxCount; > > # configure zebra results for retrieving the desired facet > > $rs->option( elementSetName => $facet_element ); > > # get the facet record from result set > > my $facet = $rs->record( 0 )->raw; > > # if the facet has no restuls... > > return if !defined $facet; > > # TODO: benchmark DOM vs. SAX performance > > my $facet_dom = XML::LibXML->load_xml( > > string => ($facet) > > ); > > my @terms = $facet_dom->getElementsByTagName('term'); > > return if ! @terms; > > > > my $facets = {}; > > foreach my $term ( @terms ) { > > my $facet_value = $term->textContent; > > $facet_value =~ s/\Q$internal_sep\E/$sep/ if defined $sep; > > $facets->{ $facet_value } = $term->getAttribute( 'occur' ); > > } > > > > return $facets; > > } > > > > Another option would be to make _get_facets_from_zebra build the element set > containing all facets so they are read in one call (comma-separate all > elements). The problem is that Zebra returns zero if one of the elements is > empty. Jared proposed to create a ghost record with all facet fields. I didn't > manage to make it work. Another option is to patch Zebra. I started that, but > abandoned once the ES code got pushed. > > > > So, if use_zebra_facets=0 is good enough, maybe it should be recommended. > Problem is it is not a real facet, but the sole extraction of the fields from > the first x records. > > As I said, it could be good enough anyway. > > > > Regards > > > > > > El mar., 23 ago. 2016 a las 10:21, Barton Chittenden (< > barton at bywatersolutions.com>) escribi?: > > Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to > disk (~16G/query on large libraries). Bug 13665 mentions that searches > could be taking upwards of 40 seconds to complete -- I think that we've > seen searches time out and return no results at about 1 minute. > > > > Is it possible to tune Zebra's space/time optimizations in any way so that > it doesn't write such large files to disk? > > > > On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart < > jonathan.druart at bugs.koha-community.org> wrote: > > See bug 13665 - Retrieve facets from zebra is slow > To understand why and when use_zebra_facet=1 is slow > > > 2016-08-22 21:31 GMT+01:00 Barton Chittenden < > barton at bywatersolutions.com>: > > I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, > but I > > have run into this, when using ICU-Chains: > > > > Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 > > URL : https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id > =16581 > > Priority : P5 - low > > Urgency : enhancement > > Status : NEW > > > > I also know that when use_zebra_facets was first introduced, it was > *very* > > slow -- I can't find any bugs about that though. It's possible that > it got > > so slow under idzebra-2.0 2.61 that the searches are timing out. > > > > It should be possible to set the logging for zebra so that you can > see the > > PQF queries: > > > > See > > > > Bug 15714 : Remove zebra.log from debian scripts and add optional log > levels > > URL : https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id > =15714 > > Priority : P5 - low > > Urgency : enhancement > > Status : RESOLVED > > > > For setting the log levels > > > > And http://koha.1045719.n5.nabble.com/ > Improving-Zebra-logging-td5861827.html > > > > For a general discussion of how to use them. > > > > ... This should give you some idea of what's failing, both in terms > of the > > dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. > > > > My general feeling is that 2.0.59 is irredeemably broken by bug > 16581, and > > we need at least 2.0.60, but I don't have any experience with zebra > facets. > > > > --Barton > > > > > > > > On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett > wrote: > >> > >> Greetings, > >> > >> Similar problem. I hope someone has a better solution than setting > it to > >> 0. > >> > >> GPML, > >> Mark Tompsett > >> > >> -----Original Message----- > >> From: Philippe Blouin > >> Sent: Monday, August 22, 2016 2:40 PM > >> To: koha-devel at lists.koha-community.org > >> Subject: [Koha-devel] 16.05, zebra and jessie > >> > >> Hello! > >> > >> We're trying to find the correction combination. We're new on > Jessie, > >> so we still have some tweaking needed... > >> > >> - By default, we get zebra 2.00.59 installed on Jessie through the > >> packages. > >> - On 16.05, we get some very bad results in the search when the > itemtype > >> contains an hyphen (-), like 'A-DOC'. > >> - So we installed zebra 2.00.62. This fixes the search... > >> - But now we do not have facets. > >> - So we set 0 > >> - And now we have facets. But this feels... wrong? > >> > >> My dummy question: what is the supposedly correct version of Zebra > on > >> Jessie ? > >> And we're we correct in setting the config to 0 ? > >> > >> Thanks > >> Blou > >> _______________________________________________ > >> Koha-devel mailing list > >> Koha-devel at lists.koha-community.org > >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > >> website : http://www.koha-community.org/ > >> git : http://git.koha-community.org/ > >> bugs : http://bugs.koha-community.org/ > >> > >> _______________________________________________ > >> Koha-devel mailing list > >> Koha-devel at lists.koha-community.org > >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > >> website : http://www.koha-community.org/ > >> git : http://git.koha-community.org/ > >> bugs : http://bugs.koha-community.org/ > > > > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ > > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > > -- > > Tom?s Cohen Arazi > > Theke Solutions (https://theke.io) > ? +54 9351 3513384 > GPG: B2F3C15F > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ -- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: not available URL: From dcook at prosentient.com.au Wed Aug 31 01:42:52 2016 From: dcook at prosentient.com.au (David Cook) Date: Wed, 31 Aug 2016 09:42:52 +1000 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: <20160830232746.sjyidue6fjkwt3ld@rorohiko.wgtn.cat-it.co.nz> References: <57BB471A.6070701@inlibro.com> <004801d20314$ef8e37f0$ceaaa7d0$@prosentient.com.au> <20160830232746.sjyidue6fjkwt3ld@rorohiko.wgtn.cat-it.co.nz> Message-ID: <005b01d20318$39457f40$abd07dc0$@prosentient.com.au> I don't doubt that I've missed a ton of words! Between baby and competing projects, I haven't had as much time to keep up with the bleeding edge. I want to be one of those people using and fixing it, although - as you say - everyone has their own priorities. It's the search engine that Koha deserves, but I reckon many of us are busy right now :/. Thanks for the update though, Chris. David Cook Systems Librarian Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Direct: 02 8005 0595 > -----Original Message----- > From: Chris Cormack [mailto:chrisc at catalyst.net.nz] > Sent: Wednesday, 31 August 2016 9:28 AM > To: David Cook > Cc: 'Tomas Cohen Arazi' ; 'Barton Chittenden' > ; 'Jonathan Druart' > ; koha-devel at lists.koha- > community.org > Subject: Re: [Koha-devel] 16.05, zebra and jessie > > * David Cook (dcook at prosentient.com.au) wrote: > > > > > > > > I suppose Adam at IndexData has been busy with the FOLIO project, so I > > doubt he has time to work on Zebra these days, even if we did have a > patch available. > > > > > > > > Is ElasticSearch usable with Koha at this point? I heard a lot in > > 2015, but after Robin left I haven?t heard a word other than rumours > > that the patches had been pushed? > > You missed tons of words then :) > > Yes, it is in 16.05, marked expiremental, it works, mostly. But it will only get > better with more people using and fixing it. > The next task is to update the version it works with to Elastic 2. That isn't a > huge amount of work, but everyone has their own priorities and a lot of us > have to work on what users ask for (not what users need ;)) > > Chris > > > > > Subject: Re: [Koha-devel] 16.05, zebra and jessie > > > > > > > > I have seen use_zebra_facets=1 cause no facets rendered when GRS-1 > > configuration files are kept during upgrades up to where GRS-1 got > > deprecated (3.20?). Is it the case? What does the About > System > > information page says about your config? > > > > > > > > The slowliness is not in zebra per se, but in the way we retrieve the > > facets from it (so Koha/Perl side). We retrieve each facet at a time > > instead of fetching them all in one call. And they come in XML format, > > so need to be parsed. So, if anyone is willing to improve it, just > > need to optimize this function (read the TODO): > > > > > > > > sub _get_facet_from_result_set { > > > > > > > > my $facet_idx = shift; > > > > my $rs = shift; > > > > my $sep = shift; > > > > > > > > my $internal_sep = '<*>'; > > > > my $facetMaxCount = C4::Context->preference('FacetMaxCount') // > > 20; > > > > > > > > return if ( ! defined $facet_idx || ! defined $rs ); > > > > # zebra's facet element, untokenized index > > > > my $facet_element = 'zebra::facet::' . $facet_idx . ':0:' . > > $facetMaxCount; > > > > # configure zebra results for retrieving the desired facet > > > > $rs->option( elementSetName => $facet_element ); > > > > # get the facet record from result set > > > > my $facet = $rs->record( 0 )->raw; > > > > # if the facet has no restuls... > > > > return if !defined $facet; > > > > # TODO: benchmark DOM vs. SAX performance > > > > my $facet_dom = XML::LibXML->load_xml( > > > > string => ($facet) > > > > ); > > > > my @terms = $facet_dom->getElementsByTagName('term'); > > > > return if ! @terms; > > > > > > > > my $facets = {}; > > > > foreach my $term ( @terms ) { > > > > my $facet_value = $term->textContent; > > > > $facet_value =~ s/\Q$internal_sep\E/$sep/ if defined $sep; > > > > $facets->{ $facet_value } = $term->getAttribute( 'occur' ); > > > > } > > > > > > > > return $facets; > > > > } > > > > > > > > Another option would be to make _get_facets_from_zebra build the > > element set containing all facets so they are read in one call > > (comma-separate all elements). The problem is that Zebra returns zero > > if one of the elements is empty. Jared proposed to create a ghost > > record with all facet fields. I didn't manage to make it work. Another > > option is to patch Zebra. I started that, but abandoned once the ES code > got pushed. > > > > > > > > So, if use_zebra_facets=0 is good enough, maybe it should be > recommended. > > Problem is it is not a real facet, but the sole extraction of the > > fields from the first x records. > > > > As I said, it could be good enough anyway. > > > > > > > > Regards > > > > > > > > > > > > El mar., 23 ago. 2016 a las 10:21, Barton Chittenden (< > > barton at bywatersolutions.com>) escribi?: > > > > Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to > > disk (~16G/query on large libraries). Bug 13665 mentions that searches > > could be taking upwards of 40 seconds to complete -- I think that we've > > seen searches time out and return no results at about 1 minute. > > > > > > > > Is it possible to tune Zebra's space/time optimizations in any way so that > > it doesn't write such large files to disk? > > > > > > > > On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart < > > jonathan.druart at bugs.koha-community.org> wrote: > > > > See bug 13665 - Retrieve facets from zebra is slow > > To understand why and when use_zebra_facet=1 is slow > > > > > > 2016-08-22 21:31 GMT+01:00 Barton Chittenden < > > barton at bywatersolutions.com>: > > > I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, > > but I > > > have run into this, when using ICU-Chains: > > > > > > Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 > > > URL : https://bugs.koha- > community.org/bugzilla3/show_bug.cgi?id > > =16581 > > > Priority : P5 - low > > > Urgency : enhancement > > > Status : NEW > > > > > > I also know that when use_zebra_facets was first introduced, it was > > *very* > > > slow -- I can't find any bugs about that though. It's possible that > > it got > > > so slow under idzebra-2.0 2.61 that the searches are timing out. > > > > > > It should be possible to set the logging for zebra so that you can > > see the > > > PQF queries: > > > > > > See > > > > > > Bug 15714 : Remove zebra.log from debian scripts and add optional > log > > levels > > > URL : https://bugs.koha- > community.org/bugzilla3/show_bug.cgi?id > > =15714 > > > Priority : P5 - low > > > Urgency : enhancement > > > Status : RESOLVED > > > > > > For setting the log levels > > > > > > And http://koha.1045719.n5.nabble.com/ > > Improving-Zebra-logging-td5861827.html > > > > > > For a general discussion of how to use them. > > > > > > ... This should give you some idea of what's failing, both in terms > > of the > > > dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. > > > > > > My general feeling is that 2.0.59 is irredeemably broken by bug > > 16581, and > > > we need at least 2.0.60, but I don't have any experience with zebra > > facets. > > > > > > --Barton > > > > > > > > > > > > On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett > > > wrote: > > >> > > >> Greetings, > > >> > > >> Similar problem. I hope someone has a better solution than setting > > it to > > >> 0. > > >> > > >> GPML, > > >> Mark Tompsett > > >> > > >> -----Original Message----- > > >> From: Philippe Blouin > > >> Sent: Monday, August 22, 2016 2:40 PM > > >> To: koha-devel at lists.koha-community.org > > >> Subject: [Koha-devel] 16.05, zebra and jessie > > >> > > >> Hello! > > >> > > >> We're trying to find the correction combination. We're new on > > Jessie, > > >> so we still have some tweaking needed... > > >> > > >> - By default, we get zebra 2.00.59 installed on Jessie through the > > >> packages. > > >> - On 16.05, we get some very bad results in the search when the > > itemtype > > >> contains an hyphen (-), like 'A-DOC'. > > >> - So we installed zebra 2.00.62. This fixes the search... > > >> - But now we do not have facets. > > >> - So we set 0 > > >> - And now we have facets. But this feels... wrong? > > >> > > >> My dummy question: what is the supposedly correct version of > Zebra > > on > > >> Jessie ? > > >> And we're we correct in setting the config to 0 ? > > >> > > >> Thanks > > >> Blou > > >> _______________________________________________ > > >> Koha-devel mailing list > > >> Koha-devel at lists.koha-community.org > > >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha- > devel > > >> website : http://www.koha-community.org/ > > >> git : http://git.koha-community.org/ > > >> bugs : http://bugs.koha-community.org/ > > >> > > >> _______________________________________________ > > >> Koha-devel mailing list > > >> Koha-devel at lists.koha-community.org > > >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha- > devel > > >> website : http://www.koha-community.org/ > > >> git : http://git.koha-community.org/ > > >> bugs : http://bugs.koha-community.org/ > > > > > > > > > > > > _______________________________________________ > > > Koha-devel mailing list > > > Koha-devel at lists.koha-community.org > > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > > website : http://www.koha-community.org/ > > > git : http://git.koha-community.org/ > > > bugs : http://bugs.koha-community.org/ > > > > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ > > > > -- > > > > Tom?s Cohen Arazi > > > > Theke Solutions (https://theke.io) > > ? +54 9351 3513384 > > GPG: B2F3C15F > > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ git : > > http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ > > > -- > Chris Cormack > Catalyst IT Ltd. > +64 4 803 2238 > PO Box 11-053, Manners St, Wellington 6142, New Zealand From jonathan.druart at bugs.koha-community.org Wed Aug 31 09:41:20 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Wed, 31 Aug 2016 08:41:20 +0100 Subject: [Koha-devel] GoogleIndicTransliteration must die? In-Reply-To: References: Message-ID: See bug 17137 - Omnibus: Always use SSL when referencing external sources 2016-08-30 23:36 GMT+01:00 Indranil Das Gupta : > Hi all, > > Koha uses the deprecated (in 2011) Google Indic Transliterate v1 API > to power the GoogleIndicTransliterate syspref. > > It seems internally the API calls to Googe Inputtools service over > http! So again we are back in MIXED CONTENT mess and nothing much we > can do about it, except move out of this API. > > transliteration.I.js:formatted:11055 Mixed Content: The page at > 'https://rksmvvlibrary.in/cgi-bin/koha/opac-search.pl?q=%E0%A6%B6%E0%A7%8D%E?E%E0%A6%AE%E0%A6%BE%E0%A6%AA%E0%A7%8D%E0%A6%B0%E0%A6%B8%E0%A6%BE%E0%A6%A6+' > was loaded over HTTPS, but requested an insecure script > 'http://www.google.com/inputtools/request?text=shyama&ime=transliteration_en?n&num=5&cp=0&cs=0&ie=utf-8&oe=utf-8&app=jsapi&uv&cb=_callbacks_._0isi0aei6'. > This request has been blocked; the content must be served over > HTTPS.Q.dh @ transliteration.I.js:formatted:11055Ww @ > transliteration.I.js:formatted:4578Q.Ad @ > transliteration.I.js:formatted:10982(anonymous function) @ > transliteration.I.js:formatted:11237(anonymous function) @ > transliteration.I.js:formatted:6670(anonymous function) @ > transliteration.I.js:formatted:6059nq @ > transliteration.I.js:formatted:2266(anonymous function) @ > transliteration.I.js:formatted:6232a @ > transliteration.I.js:formatted:6074 > transliteration.I.js:formatted:10923 Uncaught TypeError: Cannot read > property '0' of nullQ.Bh @ transliteration.I.js:formatted:10923Q.Mi @ > transliteration.I.js:formatted:10862Mt.(anonymous function).j @ > transliteration.I.js:formatted:8324(anonymous function) @ > transliteration.I.js:formatted:3591 > > > I saw this http://stackoverflow.com/questions/35238710/deprecated-google-api-will-not-work-from-page-served-over-https > and this https://groups.google.com/forum/?fromgroups#!topic/google-ajax-search-api/9TqbCXqQTSA > > Looks like its time to bid goodbye to GoogleIndicTransliteration syspref :-( > > -- > Indranil Das Gupta > L2C2 Technologies > > Phone : +91-98300-20971 > Blog : http://blog.l2c2.co.in > IRC : indradg on irc://irc.freenode.net > Twitter : indradg > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From nicolas.legrand at bulac.fr Wed Aug 31 09:46:34 2016 From: nicolas.legrand at bulac.fr (Nicolas Legrand) Date: Wed, 31 Aug 2016 09:46:34 +0200 Subject: [Koha-devel] biblioitems.marcxml & biblioitems.marc / HUGE performance issue ! In-Reply-To: <004701d20311$f716f550$e544dff0$@prosentient.com.au> References: <57851E32.5000202@biblibre.com> <004701d20311$f716f550$e544dff0$@prosentient.com.au> Message-ID: also, as a more than 1M biblios library, hurray for Jonathan :) 2016-08-31 0:58 GMT+02:00 David Cook : > Excitement! I've been looking forward to patches for those bugs for years. > I don't have the time to test them now, but hoping that someone does soon! > > Of course, moving biblioitems.marcxml will break a lot of SQL reports I'm > sure, but I think it's worth it. The broken reports should generate noisy > errors, so it's not like they would silently fail. And there comes a time > where you just need to move on to something new. > > I'm curious about our "table","deletedtable" methodology. I suppose by > moving data to "deletedtable" for historical purposes, we're theoretically > improving performance for "table" and making queries simple for active > records. I think we lose data integrity sometimes (more so with issues and > reserves than records) by moving data to historical tables. I suppose we're > entrenched in that design now though, so discussion would be a bit moot. > > Anyway, hurray for Jonathan adding patches! > > David Cook > Systems Librarian > Prosentient Systems > 72/330 Wattle St > Ultimo, NSW 2007 > Australia > > Office: 02 9212 0899 > Direct: 02 8005 0595 > > > > -----Original Message----- > > From: koha-devel-bounces at lists.koha-community.org [mailto:koha-devel- > > bounces at lists.koha-community.org] On Behalf Of Jonathan Druart > > Sent: Friday, 26 August 2016 7:39 PM > > To: koha-devel at lists.koha-community.org > > Subject: Re: [Koha-devel] biblioitems.marcxml & biblioitems.marc / HUGE > > performance issue ! > > > > For the record: > > Bug 10455 - remove redundant 'biblioitems.marc' field and > > Bug 17196 - Move marcxml out of the biblioitems table have now patches! > > > > 2016-07-12 17:43 GMT+01:00 Paul Poulain : > > > Hi all, > > > > > > Those days, we're working on a pretty large DB ( >1M biblio), for a > > > customer that want to do many statistics on some fields. > > > We discovered that something "simple" like: > > > SELECT publicationyear, count(publicationyear) FROM biblioitems GROUP > > > BY publicationyear; > > > > > > was giving no result in 10mn. > > > This is a test DB, not optimized, but we were surprised by the results. > > > After investigating we had the idea to create a biblioitems2 table > > > with the same structure EXCEPT MARCXML and MARC fields > > > > > > launch the same SQL query : result in 3seconds ! > > > This could be reproduced on any query (on fields without index). > > > > > > I think it's because the innoDB is storing each line in one "object", > > > so, even if you need only one column, you have to read everything. > > > In our case, that was 12GB+ of data to read. > > > biblioitems2 is just a few dozen MB. > > > (all caching values are minimum and there's no index, so not involved > > > in the > > > results) > > > > > > MY CONCLUSIONS: > > > * the biblioitems.marc field must be removed quickly: it's useless > > > since years, and is only resulting in slowing things > > > * the bilbioitems.marcxml field should be moved outside from this > table. > > > Something like biblio_blob, with biblionumber, biblioitemnumber and > > marcxml. > > > When we need it, just join the tables. > > > > > > I'm almost sure it would have an important impact on Koha, as > > > biblioitems table is called and used "everywhere". > > > > > > any opinion ? > > > > > > -- > > > Paul Poulain, Associ?-g?rant / co-owner BibLibre, Services en > > > logiciels libres pour les biblioth?ques BibLibre, Open Source software > > > and services for libraries > > > > > > > > > _______________________________________________ > > > Koha-devel mailing list > > > Koha-devel at lists.koha-community.org > > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > > website : http://www.koha-community.org/ git : > > > http://git.koha-community.org/ bugs : http://bugs.koha-community.org/ > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ git : http://git.koha- > > community.org/ bugs : http://bugs.koha-community.org/ > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > -- *Nicolas Legrand* Administration technique et d?veloppements du syst?me de gestion de la biblioth?que [image: Logo BULAC] Biblioth?que universitaire des langues et civilisations 65 rue des Grands Moulins F-75013 PARIS T +33 1 81 69 *18 22* www.bulac.fr -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathan.druart at bugs.koha-community.org Wed Aug 31 09:55:16 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Wed, 31 Aug 2016 08:55:16 +0100 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: <004801d20314$ef8e37f0$ceaaa7d0$@prosentient.com.au> References: <57BB471A.6070701@inlibro.com> <004801d20314$ef8e37f0$ceaaa7d0$@prosentient.com.au> Message-ID: 2016-08-31 0:19 GMT+01:00 David Cook : > Is ElasticSearch usable with Koha at this point? I heard a lot in 2015, but > after Robin left I haven?t heard a word other than rumours that the patches > had been pushed? It's pushed and testable. It does not work for new installs (bug 16838) and is broken if OpacSuppression is set (bug 16660). Nobody seems to take care of these bugs (I guess I should). Improvements are provided but don't get lot of attention from signoffers Bug 14567 - Browse interface for OPAC Bug 14899 - Mapping configuration page for Elastic search Cheers, Jonathan > David Cook > > Systems Librarian > > Prosentient Systems > > 72/330 Wattle St > > Ultimo, NSW 2007 > > Australia > > > > Office: 02 9212 0899 > > Direct: 02 8005 0595 > > > > From: koha-devel-bounces at lists.koha-community.org > [mailto:koha-devel-bounces at lists.koha-community.org] On Behalf Of Tomas > Cohen Arazi > Sent: Tuesday, 23 August 2016 11:42 PM > To: Barton Chittenden ; Jonathan Druart > > Cc: koha-devel at lists.koha-community.org > Subject: Re: [Koha-devel] 16.05, zebra and jessie > > > > I have seen use_zebra_facets=1 cause no facets rendered when GRS-1 > configuration files are kept during upgrades up to where GRS-1 got > deprecated (3.20?). Is it the case? What does the About > System information > page says about your config? > > > > The slowliness is not in zebra per se, but in the way we retrieve the facets > from it (so Koha/Perl side). We retrieve each facet at a time instead of > fetching them all in one call. And they come in XML format, so need to be > parsed. So, if anyone is willing to improve it, just need to optimize this > function (read the TODO): > > > > sub _get_facet_from_result_set { > > > > my $facet_idx = shift; > > my $rs = shift; > > my $sep = shift; > > > > my $internal_sep = '<*>'; > > my $facetMaxCount = C4::Context->preference('FacetMaxCount') // 20; > > > > return if ( ! defined $facet_idx || ! defined $rs ); > > # zebra's facet element, untokenized index > > my $facet_element = 'zebra::facet::' . $facet_idx . ':0:' . > $facetMaxCount; > > # configure zebra results for retrieving the desired facet > > $rs->option( elementSetName => $facet_element ); > > # get the facet record from result set > > my $facet = $rs->record( 0 )->raw; > > # if the facet has no restuls... > > return if !defined $facet; > > # TODO: benchmark DOM vs. SAX performance > > my $facet_dom = XML::LibXML->load_xml( > > string => ($facet) > > ); > > my @terms = $facet_dom->getElementsByTagName('term'); > > return if ! @terms; > > > > my $facets = {}; > > foreach my $term ( @terms ) { > > my $facet_value = $term->textContent; > > $facet_value =~ s/\Q$internal_sep\E/$sep/ if defined $sep; > > $facets->{ $facet_value } = $term->getAttribute( 'occur' ); > > } > > > > return $facets; > > } > > > > Another option would be to make _get_facets_from_zebra build the element set > containing all facets so they are read in one call (comma-separate all > elements). The problem is that Zebra returns zero if one of the elements is > empty. Jared proposed to create a ghost record with all facet fields. I > didn't manage to make it work. Another option is to patch Zebra. I started > that, but abandoned once the ES code got pushed. > > > > So, if use_zebra_facets=0 is good enough, maybe it should be recommended. > Problem is it is not a real facet, but the sole extraction of the fields > from the first x records. > > As I said, it could be good enough anyway. > > > > Regards > > > > > > El mar., 23 ago. 2016 a las 10:21, Barton Chittenden > () escribi?: > > Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to > disk (~16G/query on large libraries). Bug 13665 mentions that searches could > be taking upwards of 40 seconds to complete -- I think that we've seen > searches time out and return no results at about 1 minute. > > > > Is it possible to tune Zebra's space/time optimizations in any way so that > it doesn't write such large files to disk? > > > > On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart > wrote: > > See bug 13665 - Retrieve facets from zebra is slow > To understand why and when use_zebra_facet=1 is slow > > > 2016-08-22 21:31 GMT+01:00 Barton Chittenden : >> I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I >> have run into this, when using ICU-Chains: >> >> Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 >> URL : >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 >> Priority : P5 - low >> Urgency : enhancement >> Status : NEW >> >> I also know that when use_zebra_facets was first introduced, it was *very* >> slow -- I can't find any bugs about that though. It's possible that it got >> so slow under idzebra-2.0 2.61 that the searches are timing out. >> >> It should be possible to set the logging for zebra so that you can see the >> PQF queries: >> >> See >> >> Bug 15714 : Remove zebra.log from debian scripts and add optional log >> levels >> URL : >> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714 >> Priority : P5 - low >> Urgency : enhancement >> Status : RESOLVED >> >> For setting the log levels >> >> And >> http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html >> >> For a general discussion of how to use them. >> >> ... This should give you some idea of what's failing, both in terms of the >> dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. >> >> My general feeling is that 2.0.59 is irredeemably broken by bug 16581, and >> we need at least 2.0.60, but I don't have any experience with zebra >> facets. >> >> --Barton >> >> >> >> On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett >> wrote: >>> >>> Greetings, >>> >>> Similar problem. I hope someone has a better solution than setting it to >>> 0. >>> >>> GPML, >>> Mark Tompsett >>> >>> -----Original Message----- >>> From: Philippe Blouin >>> Sent: Monday, August 22, 2016 2:40 PM >>> To: koha-devel at lists.koha-community.org >>> Subject: [Koha-devel] 16.05, zebra and jessie >>> >>> Hello! >>> >>> We're trying to find the correction combination. We're new on Jessie, >>> so we still have some tweaking needed... >>> >>> - By default, we get zebra 2.00.59 installed on Jessie through the >>> packages. >>> - On 16.05, we get some very bad results in the search when the itemtype >>> contains an hyphen (-), like 'A-DOC'. >>> - So we installed zebra 2.00.62. This fixes the search... >>> - But now we do not have facets. >>> - So we set 0 >>> - And now we have facets. But this feels... wrong? >>> >>> My dummy question: what is the supposedly correct version of Zebra on >>> Jessie ? >>> And we're we correct in setting the config to 0 ? >>> >>> Thanks >>> Blou >>> _______________________________________________ >>> Koha-devel mailing list >>> Koha-devel at lists.koha-community.org >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>> website : http://www.koha-community.org/ >>> git : http://git.koha-community.org/ >>> bugs : http://bugs.koha-community.org/ >>> >>> _______________________________________________ >>> Koha-devel mailing list >>> Koha-devel at lists.koha-community.org >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>> website : http://www.koha-community.org/ >>> git : http://git.koha-community.org/ >>> bugs : http://bugs.koha-community.org/ >> >> >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ > > > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ > > -- > > Tom?s Cohen Arazi > > Theke Solutions (https://theke.io) > ? +54 9351 3513384 > GPG: B2F3C15F From jonathan.druart at bugs.koha-community.org Wed Aug 31 10:24:38 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Wed, 31 Aug 2016 09:24:38 +0100 Subject: [Koha-devel] Koha::* business objects authorization check In-Reply-To: References: Message-ID: Hi devs, Not sure I will manage to explain it better than you... Basically we all agree that the permission check should not be done from the controller. It would be good to let the Koha::Object-object know if a given user can interact with it. The problem is that the permissions can be different depending on the action: Add/update/delete/list Who is able to add a patron? => You need borrowers perm Who is able to edit detail for this patron? => You need borrowers perm or you are logged in with this user or the guarantor Who is able to delete this patron? => same as above (unless you cannot delete yourself) Who is able to list patrons? It depends on IndependentBranches: we want to filter on the search, not after the search (perf issues and awful design). And this is the easy example. After the discussion of yesterday, Tom?s was more willing to override ->store and raise an exception if the perms are not enough. IMO ->store should store if we call it, it's the responsibility of the developers to test if the logged in user is allowed to update/insert the stuff: if ( $patron->can_be_updated( [$logged_in_user]) ) { $patron->store; } else { # Raise an exception } We already have 2 examples of perms checking: % git grep can_be Koha/Virtualshelf.pm sub can_be_viewed { sub can_be_deleted { sub can_be_managed { And, not pushed yet: Bug 15758 - Move the C4::Branch related code to Koha::Libraries - part 4 https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=54864 +sub search_filtered { + my ( $self, $params, $attributes ) = @_; + + if ( C4::Context->preference('IndependentBranches') + and C4::Context->userenv + and not C4::Context->IsSuperLibrarian() + and C4::Context->userenv->{branch} + ) { + $params->{branchcode} = C4::Context->userenv->{branch}; + } + + return $self->SUPER::search( $params, $attributes ); +} Hope that adds confusion ;) Jonathan 2016-08-30 21:11 GMT+01:00 Tomas Cohen Arazi : > Hi everyone, adding to the discussion about Koha::Objects I have a subject > that needs broad (an urgent) discussion. > > The REST api patches are shaping up, and I'd like to focus on an important > one: 14868 [1]. > > This bug introduces a way of specifying on the Swagger files, the required > privileges to do stuff on objects. For example: > > + "x-koha-authorization": { > + "allow-owner": true, > + "allow-guarantor": true, > + "permissions": { > + "borrowers": "1" > + } > } > > specifies that to get a patron object you need to either have the > 'borrowers' permission, be the object owner, or the guarantor of the > object's owner. > > This is really nice, because permissions definition are now higher level > entities (just a configuration thing), and they are part of the API > documentation. There's even a bug with patches that introduce required > permissions in the error messages. > > The current implementation introduces checks for each object class on the > controller code. > > My feeling is that each business object should be responsible for such a > check. The first step I made was to extend Koha::Object so it exposes an > ->owner method that returns the Koha::Patron object that owns the object > (because we can get the guarantees from there too). > I did this to eliminate the need of class-specific checks on the controller. > But at the end of the day, this still didn't feel it was good enough. I > belive such a business matter should not be so tied to the > controller/routing code and it is still there. > > Talking with Jonathan we thought we could make Koha::Objects->search accept > more parameters (current userid, required permissions) and have the code > checking authorization there, basically creating a filter on the search. The > main problem is how to propagate the swagger-specified required permissions > down to the Koha::Object in a clean way. > > I'd like to hear how people belive we should handle on this new+fresh code > permissions check, so we have good foundations for the future of the > project. Cleaning untested business logic from controller scripts is a good > start! And this is the time to do it! > > Thanks in advance > > PS. I'm sure Jonathan will explain the discussion better once he comes back > from the pub :-D > > [1] https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14868 > -- > Tom?s Cohen Arazi > Theke Solutions (https://theke.io) > ? +54 9351 3513384 > GPG: B2F3C15F > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From jonathan.druart at bugs.koha-community.org Wed Aug 31 10:32:39 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Wed, 31 Aug 2016 09:32:39 +0100 Subject: [Koha-devel] Koha::* business objects authorization check In-Reply-To: <20160830202305.tples2og7k2ljc6l@rorohiko.wgtn.cat-it.co.nz> References: <20160830202305.tples2og7k2ljc6l@rorohiko.wgtn.cat-it.co.nz> Message-ID: 2016-08-30 21:23 GMT+01:00 Chris Cormack : > * Tomas Cohen Arazi (tomascohen at gmail.com) wrote: >> Hi everyone, adding to the discussion about Koha::Objects I have a subject that >> needs broad (an urgent) discussion. >> >> The REST api patches are shaping up, and I'd like to focus on an important one: >> 14868 [1]. >> >> This bug introduces a way of specifying on the Swagger files, the required >> privileges to do stuff on objects. For example: >> >> >> + "x-koha-authorization": { >> + "allow-owner": true, >> + "allow-guarantor": true, >> + "permissions": { >> + "borrowers": "1" >> + } >> } >> >> specifies that to get a patron object you need to either have the 'borrowers' >> permission, be the object owner, or the guarantor of the object's owner. >> >> This is really nice, because permissions definition are now higher level >> entities (just a configuration thing), and they are part of the API >> documentation. There's even a bug with patches that introduce required >> permissions in the error messages. >> >> The current implementation introduces checks for each object class on the >> controller code. >> >> My feeling is that each business object should be responsible for such a check. >> The first step I made was to extend Koha::Object so it exposes an ->owner >> method that returns the Koha::Patron object that owns the object (because we >> can get the guarantees from there too). > > I like it, it is only when we do things like this that the Objects become useful, > otherwise they are a shim over db access (and a slow shim at that). > Once they do more things like this, they will be a lot more useful. > > Chris I think it's also quite useful when we manage to migrate our legacy code to OO quite easily (reducing the number of lines and adding test coverage). Note that Koha::Object[s] does not reduce speed compare to using DBIx::Class directly, I have already proved that previously. Your turn to prove the contrary if you think you are right :) Cheers, Jonathan > >> I did this to eliminate the need of class-specific checks on the controller. >> But at the end of the day, this still didn't feel it was good enough. I belive >> such a business matter should not be so tied to the controller/routing code and >> it is still there. >> >> Talking with Jonathan we thought we could make Koha::Objects->search accept >> more parameters (current userid, required permissions) and have the code >> checking authorization there, basically creating a filter on the search. The >> main problem is how to propagate the swagger-specified required permissions >> down to the Koha::Object in a clean way. >> >> I'd like to hear how people belive we should handle on this new+fresh code >> permissions check, so we have good foundations for the future of the project. >> Cleaning untested business logic from controller scripts is a good start! And >> this is the time to do it! >> >> Thanks in advance >> >> PS. I'm sure Jonathan will explain the discussion better once he comes back >> from the pub :-D >> >> [1] https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14868 >> -- >> Tom?s Cohen Arazi >> Theke Solutions (https://theke.io) >> ? +54 9351 3513384 >> GPG: B2F3C15F > >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ > > > -- > Chris Cormack > Catalyst IT Ltd. > +64 4 803 2238 > PO Box 11-053, Manners St, Wellington 6142, New Zealand > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From jonathan.druart at bugs.koha-community.org Wed Aug 31 10:48:34 2016 From: jonathan.druart at bugs.koha-community.org (Jonathan Druart) Date: Wed, 31 Aug 2016 09:48:34 +0100 Subject: [Koha-devel] Sessions terminated at random under Plack In-Reply-To: References: Message-ID: 2016-08-30 19:40 GMT+01:00 Tomas Cohen Arazi : > Magnus, there's been a lot of movement on the caching layer, and some of > that work has been backported to the stable releases. Also, the > RestrictSessionByIP setting was getting in the middle > (https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=17050). > > There's also a problem with memcached initialization in C4::Context that > makes sessionStorage=memcache fail to persist sessions. Please confirm and open a bug report if it's still the case. I don't recreate it. > My suggestion would be to set sessionStorage to 'mysql' and disable the > RestricSessionByIP syspref. > > Regards > > > > El mar., 30 ago. 2016 a las 8:24, Magnus Enger () > escribi?: >> >> Dear Community, >> >> I am seeing a strange problem, and I'm not sure where to start digging. >> >> I have a (big) server with 30 odd Koha instances. One of these >> instances has been running under Plack for quite some time now, >> without any problems. >> >> Two new instances have a problem where librarians get kicked out of >> the intranet more or less frequently, with a message like "Your >> session has expired, please log in again". >> >> We had this problem on 3.22.x. >> After upgrading to 16.05.02 it went away. >> After upgrading to 16.05.03 last night it came back again. >> >> It is not consistent. Earlier today it looked like you could log in, >> click on a link, get thrown out, login again and then things would >> work. Later users got kicked out every single time they clicked on a >> link in the intranet. >> >> All sites are running off the official Debian packages, on the same >> server. >> Memcached is installed, enabled and used for storing sessions. >> Switching SessionStorage to the DB does not stop the problem. >> Stopping and disabling Plack for these two instances makes the problem go >> away. >> >> I have not found anything interesting in Plack or Apache logs. >> >> Anyone got a hunch what might be causing this? Or where to start digging? >> >> Best regards, >> Magnus >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ > > -- > Tom?s Cohen Arazi > Theke Solutions (https://theke.io) > ? +54 9351 3513384 > GPG: B2F3C15F > > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ From philippe.blouin at inlibro.com Wed Aug 31 14:17:16 2016 From: philippe.blouin at inlibro.com (Philippe Blouin) Date: Wed, 31 Aug 2016 08:17:16 -0400 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: References: <57BB471A.6070701@inlibro.com> <004801d20314$ef8e37f0$ceaaa7d0$@prosentient.com.au> Message-ID: <57C6CACC.7050605@inlibro.com> We had time and a resources available for a while. We got Bouzid to learn Elastic Search, learn about its integration in Koha, then wanted him to contribute. But been told twice to ... wait. We certainly have no time anymore. We'll be back, but probably not until 2017 unless sponsored. Now, may I suggest something? Documentation. Not user friendly, but at least dev-enlightening. "You" guys put time in there, as with plack and memcache and whatnot, but if you're on the outside, you have no way of getting a head start beside reading some 1200 bugzilla comments in chronological order to know what's the status, how to get it to work on a git, etc... (my ref to plack is not accidental). You can put tens of hours alone on the next bug and then wonder why nobody comes in when you call for testers, or invest a few hours on documenting a quick way for dummasses like me to catch up. *Now*, everyone is constrained by time, and I apologize if the comment above is frustrating to read. I don't want to be rude, and I appreciate all the hours everyone put to improve the product while I don't or can't. Please see that just as my opinion on why different big devs don't get as many helpers as deserved. Philippe Blouin, Responsable du d?veloppement informatique T?l. : (888) 604-2627 philippe.blouin at inLibro.com inLibro | pour esprit libre | www.inLibro.com On 08/31/2016 03:55 AM, Jonathan Druart wrote: > 2016-08-31 0:19 GMT+01:00 David Cook : >> Is ElasticSearch usable with Koha at this point? I heard a lot in 2015, but >> after Robin left I haven?t heard a word other than rumours that the patches >> had been pushed? > It's pushed and testable. > It does not work for new installs (bug 16838) and is broken if > OpacSuppression is set (bug 16660). Nobody seems to take care of these > bugs (I guess I should). > Improvements are provided but don't get lot of attention from signoffers > Bug 14567 - Browse interface for OPAC > Bug 14899 - Mapping configuration page for Elastic search > > Cheers, > Jonathan > >> David Cook >> >> Systems Librarian >> >> Prosentient Systems >> >> 72/330 Wattle St >> >> Ultimo, NSW 2007 >> >> Australia >> >> >> >> Office: 02 9212 0899 >> >> Direct: 02 8005 0595 >> >> >> >> From: koha-devel-bounces at lists.koha-community.org >> [mailto:koha-devel-bounces at lists.koha-community.org] On Behalf Of Tomas >> Cohen Arazi >> Sent: Tuesday, 23 August 2016 11:42 PM >> To: Barton Chittenden ; Jonathan Druart >> >> Cc: koha-devel at lists.koha-community.org >> Subject: Re: [Koha-devel] 16.05, zebra and jessie >> >> >> >> I have seen use_zebra_facets=1 cause no facets rendered when GRS-1 >> configuration files are kept during upgrades up to where GRS-1 got >> deprecated (3.20?). Is it the case? What does the About > System information >> page says about your config? >> >> >> >> The slowliness is not in zebra per se, but in the way we retrieve the facets >> from it (so Koha/Perl side). We retrieve each facet at a time instead of >> fetching them all in one call. And they come in XML format, so need to be >> parsed. So, if anyone is willing to improve it, just need to optimize this >> function (read the TODO): >> >> >> >> sub _get_facet_from_result_set { >> >> >> >> my $facet_idx = shift; >> >> my $rs = shift; >> >> my $sep = shift; >> >> >> >> my $internal_sep = '<*>'; >> >> my $facetMaxCount = C4::Context->preference('FacetMaxCount') // 20; >> >> >> >> return if ( ! defined $facet_idx || ! defined $rs ); >> >> # zebra's facet element, untokenized index >> >> my $facet_element = 'zebra::facet::' . $facet_idx . ':0:' . >> $facetMaxCount; >> >> # configure zebra results for retrieving the desired facet >> >> $rs->option( elementSetName => $facet_element ); >> >> # get the facet record from result set >> >> my $facet = $rs->record( 0 )->raw; >> >> # if the facet has no restuls... >> >> return if !defined $facet; >> >> # TODO: benchmark DOM vs. SAX performance >> >> my $facet_dom = XML::LibXML->load_xml( >> >> string => ($facet) >> >> ); >> >> my @terms = $facet_dom->getElementsByTagName('term'); >> >> return if ! @terms; >> >> >> >> my $facets = {}; >> >> foreach my $term ( @terms ) { >> >> my $facet_value = $term->textContent; >> >> $facet_value =~ s/\Q$internal_sep\E/$sep/ if defined $sep; >> >> $facets->{ $facet_value } = $term->getAttribute( 'occur' ); >> >> } >> >> >> >> return $facets; >> >> } >> >> >> >> Another option would be to make _get_facets_from_zebra build the element set >> containing all facets so they are read in one call (comma-separate all >> elements). The problem is that Zebra returns zero if one of the elements is >> empty. Jared proposed to create a ghost record with all facet fields. I >> didn't manage to make it work. Another option is to patch Zebra. I started >> that, but abandoned once the ES code got pushed. >> >> >> >> So, if use_zebra_facets=0 is good enough, maybe it should be recommended. >> Problem is it is not a real facet, but the sole extraction of the fields >> from the first x records. >> >> As I said, it could be good enough anyway. >> >> >> >> Regards >> >> >> >> >> >> El mar., 23 ago. 2016 a las 10:21, Barton Chittenden >> () escribi?: >> >> Zebra tends to be I/O bound -- we've seen it write enormous .zrs files to >> disk (~16G/query on large libraries). Bug 13665 mentions that searches could >> be taking upwards of 40 seconds to complete -- I think that we've seen >> searches time out and return no results at about 1 minute. >> >> >> >> Is it possible to tune Zebra's space/time optimizations in any way so that >> it doesn't write such large files to disk? >> >> >> >> On Tue, Aug 23, 2016 at 5:38 AM, Jonathan Druart >> wrote: >> >> See bug 13665 - Retrieve facets from zebra is slow >> To understand why and when use_zebra_facet=1 is slow >> >> >> 2016-08-22 21:31 GMT+01:00 Barton Chittenden : >>> I haven't run into the issue with the dashes in idzebra-2.0 2.0.59, but I >>> have run into this, when using ICU-Chains: >>> >>> Bug 16581 : ICU tokenization bug in idzebra-2.0 2.0.59-1 >>> URL : >>> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16581 >>> Priority : P5 - low >>> Urgency : enhancement >>> Status : NEW >>> >>> I also know that when use_zebra_facets was first introduced, it was *very* >>> slow -- I can't find any bugs about that though. It's possible that it got >>> so slow under idzebra-2.0 2.61 that the searches are timing out. >>> >>> It should be possible to set the logging for zebra so that you can see the >>> PQF queries: >>> >>> See >>> >>> Bug 15714 : Remove zebra.log from debian scripts and add optional log >>> levels >>> URL : >>> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=15714 >>> Priority : P5 - low >>> Urgency : enhancement >>> Status : RESOLVED >>> >>> For setting the log levels >>> >>> And >>> http://koha.1045719.n5.nabble.com/Improving-Zebra-logging-td5861827.html >>> >>> For a general discussion of how to use them. >>> >>> ... This should give you some idea of what's failing, both in terms of the >>> dashes in 2.0.59 and the non-functional zebra facets under 2.0.61. >>> >>> My general feeling is that 2.0.59 is irredeemably broken by bug 16581, and >>> we need at least 2.0.60, but I don't have any experience with zebra >>> facets. >>> >>> --Barton >>> >>> >>> >>> On Mon, Aug 22, 2016 at 2:49 PM, Mark Tompsett >>> wrote: >>>> Greetings, >>>> >>>> Similar problem. I hope someone has a better solution than setting it to >>>> 0. >>>> >>>> GPML, >>>> Mark Tompsett >>>> >>>> -----Original Message----- >>>> From: Philippe Blouin >>>> Sent: Monday, August 22, 2016 2:40 PM >>>> To: koha-devel at lists.koha-community.org >>>> Subject: [Koha-devel] 16.05, zebra and jessie >>>> >>>> Hello! >>>> >>>> We're trying to find the correction combination. We're new on Jessie, >>>> so we still have some tweaking needed... >>>> >>>> - By default, we get zebra 2.00.59 installed on Jessie through the >>>> packages. >>>> - On 16.05, we get some very bad results in the search when the itemtype >>>> contains an hyphen (-), like 'A-DOC'. >>>> - So we installed zebra 2.00.62. This fixes the search... >>>> - But now we do not have facets. >>>> - So we set 0 >>>> - And now we have facets. But this feels... wrong? >>>> >>>> My dummy question: what is the supposedly correct version of Zebra on >>>> Jessie ? >>>> And we're we correct in setting the config to 0 ? >>>> >>>> Thanks >>>> Blou >>>> _______________________________________________ >>>> Koha-devel mailing list >>>> Koha-devel at lists.koha-community.org >>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>>> website : http://www.koha-community.org/ >>>> git : http://git.koha-community.org/ >>>> bugs : http://bugs.koha-community.org/ >>>> >>>> _______________________________________________ >>>> Koha-devel mailing list >>>> Koha-devel at lists.koha-community.org >>>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>>> website : http://www.koha-community.org/ >>>> git : http://git.koha-community.org/ >>>> bugs : http://bugs.koha-community.org/ >>> >>> >>> _______________________________________________ >>> Koha-devel mailing list >>> Koha-devel at lists.koha-community.org >>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >>> website : http://www.koha-community.org/ >>> git : http://git.koha-community.org/ >>> bugs : http://bugs.koha-community.org/ >> >> >> _______________________________________________ >> Koha-devel mailing list >> Koha-devel at lists.koha-community.org >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ >> >> -- >> >> Tom?s Cohen Arazi >> >> Theke Solutions (https://theke.io) >> ? +54 9351 3513384 >> GPG: B2F3C15F > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mtompset at hotmail.com Wed Aug 31 15:12:35 2016 From: mtompset at hotmail.com (Mark Tompsett) Date: Wed, 31 Aug 2016 13:12:35 +0000 Subject: [Koha-devel] Serious problem Message-ID: Greetings, Take the following steps: check out, check in, restart mysql server, check out, check in... old_issues bug! This is because we delete from issues (among other tables) when we move it to old_issues and "InnoDb forgets its highest auto_increment after server restart". Any ideas on the best way to solve it? Should I resurrect my https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10454 ? GPML, Mark Tompsett From oleonard at myacpl.org Wed Aug 31 15:27:03 2016 From: oleonard at myacpl.org (Owen Leonard) Date: Wed, 31 Aug 2016 09:27:03 -0400 Subject: [Koha-devel] Serious problem In-Reply-To: References: Message-ID: > check out, check in, restart mysql server, check out, check in... old_issues > bug! What exactly is the bug? -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org From mtompset at hotmail.com Wed Aug 31 15:41:57 2016 From: mtompset at hotmail.com (Mark Tompsett) Date: Wed, 31 Aug 2016 13:41:57 +0000 Subject: [Koha-devel] Serious problem In-Reply-To: References: Message-ID: Greetings, >> check out, check in, restart mysql server, check out, check in... >> old_issues bug! > What exactly is the bug? Loss of history. Unless you do this after the server restart: use koha_library; insert into issues (borrowernumber) values (1); -- this will fill the gap, and should restore it. actually, whatever the mysql is to set the auto_increment to max(issue_id)+1 from old_issues would work too. GPML, Mark Tompsett From tomascohen at gmail.com Wed Aug 31 16:26:33 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Wed, 31 Aug 2016 14:26:33 +0000 Subject: [Koha-devel] Koha::* business objects authorization check In-Reply-To: References: <20160830202305.tples2og7k2ljc6l@rorohiko.wgtn.cat-it.co.nz> Message-ID: Is there some consensus on having the required permissions defined in the swagger file? If that was the case, no matter how we check them at Koha::Object level, we need to make the permissions schema available at that level. So we could need to parse the swagger file and cache it. I like it for its simplicity and maintainability. but if we don't agree on making the swagger-specified permissions globally available, we would then duplicate the logic, and we could make mistakes or have them divert. If we think of the REST api alone this is not an issue. Because we have the Swagger plugin. El mi?., 31 ago. 2016 a las 5:32, Jonathan Druart (< jonathan.druart at bugs.koha-community.org>) escribi?: > 2016-08-30 21:23 GMT+01:00 Chris Cormack : > > * Tomas Cohen Arazi (tomascohen at gmail.com) wrote: > >> Hi everyone, adding to the discussion about Koha::Objects I have a > subject that > >> needs broad (an urgent) discussion. > >> > >> The REST api patches are shaping up, and I'd like to focus on an > important one: > >> 14868 [1]. > >> > >> This bug introduces a way of specifying on the Swagger files, the > required > >> privileges to do stuff on objects. For example: > >> > >> > >> + "x-koha-authorization": { > >> + "allow-owner": true, > >> + "allow-guarantor": true, > >> + "permissions": { > >> + "borrowers": "1" > >> + } > >> } > >> > >> specifies that to get a patron object you need to either have the > 'borrowers' > >> permission, be the object owner, or the guarantor of the object's owner. > >> > >> This is really nice, because permissions definition are now higher level > >> entities (just a configuration thing), and they are part of the API > >> documentation. There's even a bug with patches that introduce required > >> permissions in the error messages. > >> > >> The current implementation introduces checks for each object class on > the > >> controller code. > >> > >> My feeling is that each business object should be responsible for such > a check. > >> The first step I made was to extend Koha::Object so it exposes an > ->owner > >> method that returns the Koha::Patron object that owns the object > (because we > >> can get the guarantees from there too). > > > > I like it, it is only when we do things like this that the Objects > become useful, > > otherwise they are a shim over db access (and a slow shim at that). > > Once they do more things like this, they will be a lot more useful. > > > > Chris > > I think it's also quite useful when we manage to migrate our legacy > code to OO quite easily (reducing the number of lines and adding test > coverage). > Note that Koha::Object[s] does not reduce speed compare to using > DBIx::Class directly, I have already proved that previously. Your turn > to prove the contrary if you think you are right :) > > Cheers, > Jonathan > > > > >> I did this to eliminate the need of class-specific checks on the > controller. > >> But at the end of the day, this still didn't feel it was good enough. I > belive > >> such a business matter should not be so tied to the controller/routing > code and > >> it is still there. > >> > >> Talking with Jonathan we thought we could make Koha::Objects->search > accept > >> more parameters (current userid, required permissions) and have the code > >> checking authorization there, basically creating a filter on the > search. The > >> main problem is how to propagate the swagger-specified required > permissions > >> down to the Koha::Object in a clean way. > >> > >> I'd like to hear how people belive we should handle on this new+fresh > code > >> permissions check, so we have good foundations for the future of the > project. > >> Cleaning untested business logic from controller scripts is a good > start! And > >> this is the time to do it! > >> > >> Thanks in advance > >> > >> PS. I'm sure Jonathan will explain the discussion better once he comes > back > >> from the pub :-D > >> > >> [1] https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=14868 > >> -- > >> Tom?s Cohen Arazi > >> Theke Solutions (https://theke.io) > >> ? +54 9351 3513384 > >> GPG: B2F3C15F > > > >> _______________________________________________ > >> Koha-devel mailing list > >> Koha-devel at lists.koha-community.org > >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > >> website : http://www.koha-community.org/ > >> git : http://git.koha-community.org/ > >> bugs : http://bugs.koha-community.org/ > > > > > > -- > > Chris Cormack > > Catalyst IT Ltd. > > +64 4 803 2238 > > PO Box 11-053, Manners St, Wellington 6142, New Zealand > > > > _______________________________________________ > > Koha-devel mailing list > > Koha-devel at lists.koha-community.org > > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > > website : http://www.koha-community.org/ > > git : http://git.koha-community.org/ > > bugs : http://bugs.koha-community.org/ > _______________________________________________ > Koha-devel mailing list > Koha-devel at lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomascohen at gmail.com Wed Aug 31 17:59:57 2016 From: tomascohen at gmail.com (Tomas Cohen Arazi) Date: Wed, 31 Aug 2016 15:59:57 +0000 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: <57C6CACC.7050605@inlibro.com> References: <57BB471A.6070701@inlibro.com> <004801d20314$ef8e37f0$ceaaa7d0$@prosentient.com.au> <57C6CACC.7050605@inlibro.com> Message-ID: El mi?., 31 ago. 2016 a las 9:17, Philippe Blouin (< philippe.blouin at inlibro.com>) escribi?: > We had time and a resources available for a while. We got Bouzid to learn > Elastic Search, learn about its integration in Koha, then wanted him to > contribute. But been told twice to ... wait. > I don't think that's the common behaviour in the dev group. Everyone does whatever they want and tries to get attention on what they did, synergy. Sometimes when one is doing some dev and others pretend to do it again, you tell'em to hold on, so there's no overlap. But anyone willing to improve the project is usually encouraged. Maybe a communication issue. > We certainly have no time anymore. We'll be back, but probably not until > 2017 unless sponsored. > Well, at least you will be able soon :-D > Now, may I suggest something? Documentation. Not user friendly, but at > least dev-enlightening. > > "You" guys put time in there, as with plack and memcache and whatnot, but > if you're on the outside, you have no way of getting a head start beside > reading some 1200 bugzilla comments in chronological order to know what's > the status, how to get it to work on a git, etc... (my ref to plack is not > accidental). > As a non-Plack-expert (but got involved on integrating it and fixing stuff) I agree the conversation about Plack was difficult. Some problems are really tricky and difficult to explain. You can put tens of hours alone on the next bug and then wonder why nobody > comes in when you call for testers, or invest a few hours on documenting a > quick way for dummasses like me to catch up. > The best option is asking for help to get involved. I tend to translate silence<->lack of interest. But maybe that's just my problem or a different-culture problem. I remember I sent several emails when trying to integrate Plack into the packages and didn't get much attention. Maybe the question about the design decisions we need to make about permission checking gets more attention, because it is more a theoretical question. Regards -- Tom?s Cohen Arazi Theke Solutions (https://theke.io ) ? +54 9351 3513384 GPG: B2F3C15F -------------- next part -------------- An HTML attachment was scrubbed... URL: From barton at bywatersolutions.com Wed Aug 31 19:49:33 2016 From: barton at bywatersolutions.com (Barton Chittenden) Date: Wed, 31 Aug 2016 13:49:33 -0400 Subject: [Koha-devel] 16.05, zebra and jessie In-Reply-To: <57C6CACC.7050605@inlibro.com> References: <57BB471A.6070701@inlibro.com> <004801d20314$ef8e37f0$ceaaa7d0$@prosentient.com.au> <57C6CACC.7050605@inlibro.com> Message-ID: On Wed, Aug 31, 2016 at 8:17 AM, Philippe Blouin < philippe.blouin at inlibro.com> wrote: > Now, may I suggest something? Documentation. Not user friendly, but at > least dev-enlightening. > > "You" guys put time in there, as with plack and memcache and whatnot, but > if you're on the outside, you have no way of getting a head start beside > reading some 1200 bugzilla comments in chronological order to know what's > the status, how to get it to work on a git, etc... (my ref to plack is not > accidental). > > You can put tens of hours alone on the next bug and then wonder why nobody > comes in when you call for testers, or invest a few hours on documenting a > quick way for dummasses like me to catch up. > > *Now*, everyone is constrained by time, and I apologize if the comment > above is frustrating to read. I don't want to be rude, and I appreciate > all the hours everyone put to improve the product while I don't or can't. > Please see that just as my opinion on why different big devs don't get as > many helpers as deserved. > I think this is actually a very cogent observation, and deserves its own thread. I've definitely been in the situation where I've wanted a better view of large developments, but was overwhelmed by the comment threads in bugzilla. I've got some ideas, but I don't want to hijack the discussion of 16.05, zebra and jessie. -------------- next part -------------- An HTML attachment was scrubbed... URL: