[Bug 11081] New: Port Koha::Contrib::Tamil indexer into Koha code base
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Bug ID: 11081 Summary: Port Koha::Contrib::Tamil indexer into Koha code base Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Command-line Utilities Assignee: gmcharlt@gmail.com Reporter: frederic@tamil.fr QA Contact: testopia@bugs.koha-community.org -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Assignee|gmcharlt@gmail.com |frederic@tamil.fr --- Comment #1 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 22081 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22081&action=edit Proposed patch Port Koha::Contrib::Tamil indexer into Koha code base Add two scripts for indexing: koha-index and koha-index-daemon. Documentation: perldoc koha-index perldoc koha-index-daemon New dependencies: MooseX::Getopt MooseX::RW AnyEvent::Processor Available as a Debian package: libmoosex-getopt-perl Test plan: - Apply the patch - Install dependencies, if necessary - Reindex a whole biblio catalog: koha-index --verbose - Reindex a whole authorities catalog: koha-index --source authority --verbose - Modify 1 biblio record, and index it: koha-index --select queue - Modifiy 1 authority record, and index it: koha-index --select queue --source authority - Run indexing daemon, with a 15s timeout: koha-index-daemon --timeout 15 - Modify a biblio record: wait 15s, and see - Modify an authority record: wait 15s, and see -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 M. Tompsett <mtompset@hotmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtompset@hotmail.com --- Comment #2 from M. Tompsett <mtompset@hotmail.com> --- How does this compare to bug 6435? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 --- Comment #3 from Frédéric Demians <frederic@tamil.fr> ---
How does this compare to bug 6435?
From my perspective, rebuild_zebra.pl has grown too far, doing to much
It's a new implementation of indexing: lightweight, surclassable in order to accommodate SolR, not mixing indexing and data sanitizing, quicker (equivalent to -xml option). It's a true event driven script, not sleep(ed) one. It may also have to do with Koha code base refactoring. things. Take a look at: http://wiki.koha-community.org/wiki/Background_indexing_with_Zebra See also bug 8773 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |8773 -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 I'm just a bot <gitbot@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |gitbot@bugs.koha-community. | |org When did the bot| |2013-10-30 last check this| | --- Comment #4 from I'm just a bot <gitbot@bugs.koha-community.org> --- Patch applied cleanly, go forth and signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 wajasu <matted-34813@mypacks.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |matted-34813@mypacks.net --- Comment #5 from wajasu <matted-34813@mypacks.net> --- I tested your indexer a couple of years back. I guess if Moose related deps are around now, this can get in. I liked running what I had back then when testing. It was nice to catalogue and have things show up quickly. I wonder if any of the issues relating to Bug 11078 would be handled by this implementation. Also, if your implementaion is a long running daemon, do you have any stats on the longest uptime of your production site. Maybe Robin can comment about if this implementation is package friendly. Multiple database instances handled, etc. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 --- Comment #6 from Frédéric Demians <frederic@tamil.fr> ---
I tested your indexer a couple of years back. I guess if Moose related deps are around now, this can get in. I liked running what I had back then when testing. It was nice to catalogue and have things show up quickly.
rebuild_zebra.pl daemon mode provide that today. See my response to M. Tompsett to find reasons why this enhancement may be of interest in the medium term.
I wonder if any of the issues relating to Bug 11078 would be handled by this implementation.
For me bug 11078 is a non-issue. If you do a full re-indexing, you HAVE to stop the indexing daemon, or you have to do it when nobody will update biblio/authortiy records.
Also, if your implementaion is a long running daemon, do you have any stats on the longest uptime of your production site.
~1 year. It is stopped when Koha instances are upgraded.
Maybe Robin can comment about if this implementation is package friendly. Multiple database instances handled, etc.
See bug 8773. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bgkriegel@gmail.com Status|Needs Signoff |Patch doesn't apply --- Comment #7 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Trying to test Some conflicts, dependencies installed. When runnin I found: misc/bin/koha-index --verbose Exporting records to update Can't call method "config" on an undefined value at /home/bgkriegel/kohaclone/C4/Context.pm line 640. Am I doing something wrong? Some strange POD errors FAIL Koha/Indexer/Daemon.pm OK critic OK forbidden patterns FAIL pod *** ERROR: Unknown command 'attr' in file Koha/Indexer/Daemon.pm OK valid FAIL Koha/Indexer/Indexing.pm OK critic OK forbidden patterns FAIL pod *** ERROR: Unknown command 'method' in file Koha/Indexer/Indexing.pm *** ERROR: Unknown command 'HEAD1' in file Koha/Indexer/Indexing.pm OK valid -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #22081|0 |1 is obsolete| | --- Comment #8 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 39274 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=39274&action=edit Port Koha::Contrib::Tamil indexer into Koha code base Add two scripts for indexing: koha-index and koha-index-daemon. Documentation: perldoc koha-index perldoc koha-index-daemon New dependencies: MooseX::Getopt MooseX::RW AnyEvent::Processor Available as a Debian package: libmoosex-getopt-perl Test plan: - Apply the patch - Install dependencies, if necessary - Reindex a whole biblio catalog: koha-index --verbose - Reindex a whole authorities catalog: koha-index --source authority --verbose - Modify 1 biblio record, and index it: koha-index --select queue - Modifiy 1 authority record, and index it: koha-index --select queue --source authority - Run indexing daemon, with a 15s timeout: koha-index-daemon --timeout 15 - Modify a biblio record: wait 15s, and see - Modify an authority record: wait 15s, and see -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 --- Comment #9 from Frédéric Demians <frederic@tamil.fr> --- (In reply to Bernardo Gonzalez Kriegel from comment #7)
Trying to test
Thanks!
Some conflicts, dependencies installed.
Fixed on the new attached patch.
When runnin I found: misc/bin/koha-index --verbose Exporting records to update Can't call method "config" on an undefined value at /home/bgkriegel/kohaclone/C4/Context.pm line 640.
Fixed.
Am I doing something wrong?
Some strange POD errors
Fixed. I stay available for fixing any issue you may encounter. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #39274|0 |1 is obsolete| | --- Comment #10 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 39277 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=39277&action=edit [SIGNED-OFF] Bug 11081: Port Koha::Contrib::Tamil indexer into Koha code base Add two scripts for indexing: koha-index and koha-index-daemon. Documentation: perldoc koha-index perldoc koha-index-daemon New dependencies: MooseX::Getopt MooseX::RW AnyEvent::Processor Available as a Debian package: libmoosex-getopt-perl Test plan: - Apply the patch - Install dependencies, if necessary - Reindex a whole biblio catalog: koha-index --verbose - Reindex a whole authorities catalog: koha-index --source authority --verbose - Modify 1 biblio record, and index it: koha-index --select queue - Modifiy 1 authority record, and index it: koha-index --select queue --source authority - Run indexing daemon, with a 15s timeout: koha-index-daemon --timeout 15 - Modify a biblio record: wait 15s, and see - Modify an authority record: wait 15s, and see Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Works well, followed test plan without problems One comment: perhaps an option for koha-index-daemon to work quietly, without printing information. No koha-qa errors, but new files don't have license, fixed in followup -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 --- Comment #11 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 39278 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=39278&action=edit Bug 11081: (followup) Add license information -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #39278|0 |1 is obsolete| | --- Comment #12 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 39320 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=39320&action=edit [SIGNED-OFF] Bug 11081: (followup) Add license information Signed-off-by: Frederic Demians <f.demians@tamil.fr> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #39277|0 |1 is obsolete| | Attachment #39320|0 |1 is obsolete| | --- Comment #13 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 39419 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=39419&action=edit [PASSED QA] Bug 11081: Port Koha::Contrib::Tamil indexer into Koha code base Add two scripts for indexing: koha-index and koha-index-daemon. Documentation: perldoc koha-index perldoc koha-index-daemon New dependencies: MooseX::Getopt MooseX::RW AnyEvent::Processor Available as a Debian package: libmoosex-getopt-perl Test plan: - Apply the patch - Install dependencies, if necessary - Reindex a whole biblio catalog: koha-index --verbose - Reindex a whole authorities catalog: koha-index --source authority --verbose - Modify 1 biblio record, and index it: koha-index --select queue - Modifiy 1 authority record, and index it: koha-index --select queue --source authority - Run indexing daemon, with a 15s timeout: koha-index-daemon --timeout 15 - Modify a biblio record: wait 15s, and see - Modify an authority record: wait 15s, and see Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Works well, followed test plan without problems One comment: perhaps an option for koha-index-daemon to work quietly, without printing information. No koha-qa errors, but new files don't have license, fixed in followup Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 --- Comment #14 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 39420 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=39420&action=edit [PASSED QA] Bug 11081: (followup) Add license information Signed-off-by: Frederic Demians <f.demians@tamil.fr> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Heather Braum <hbraum@nekls.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hbraum@nekls.org -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Brendan Gallagher <brendan@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |brendan@bywatersolutions.co | |m Status|Passed QA |Failed QA --- Comment #15 from Brendan Gallagher <brendan@bywatersolutions.com> --- Didn't apply cleanly for me. Can you please rebase? -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Passed QA -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #39419|0 |1 is obsolete| | --- Comment #16 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 46176 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=46176&action=edit Bug 11081: Port Koha::Contrib::Tamil indexer into Koha code base Add two scripts for indexing: koha-index and koha-index-daemon. Documentation: perldoc koha-index perldoc koha-index-daemon New dependencies: MooseX::Getopt MooseX::RW AnyEvent::Processor Available as a Debian package: libmoosex-getopt-perl Test plan: - Apply the patch - Install dependencies, if necessary - Reindex a whole biblio catalog: koha-index --verbose - Reindex a whole authorities catalog: koha-index --source authority --verbose - Modify 1 biblio record, and index it: koha-index --select queue - Modifiy 1 authority record, and index it: koha-index --select queue --source authority - Run indexing daemon, with a 15s timeout: koha-index-daemon --timeout 15 - Modify a biblio record: wait 15s, and see - Modify an authority record: wait 15s, and see Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Works well, followed test plan without problems One comment: perhaps an option for koha-index-daemon to work quietly, without printing information. No koha-qa errors, but new files don't have license, fixed in followup Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Frédéric Demians <frederic@tamil.fr> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #39420|0 |1 is obsolete| | --- Comment #17 from Frédéric Demians <frederic@tamil.fr> --- Created attachment 46177 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=46177&action=edit Bug 11081: (followup) Add license information Signed-off-by: Frédéric Demians <f.demians@tamil.fr> Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Frédéric Demians <f.demians@tamil.fr> -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 --- Comment #18 from Frédéric Demians <frederic@tamil.fr> --- Rebased, and revert to Passed QA status. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11081 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |gmcharlt@gmail.com, | |katrin.fischer@bsz-bw.de Keywords| |dependency -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org