https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |nick@bywatersolutions.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Fridolin SOMERS from comment #2)

Very nice.

Why add "use Try::Tiny" in misc/search_tools/rebuild_elasticsearch.pl ?

Otherwise the try/catch doesn't work with the values returned from Indexer.pm -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Björn Nylén <bjorn.nylen@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #109626|0 |1 is obsolete| | CC| |bjorn.nylen@ub.lu.se --- Comment #5 from Björn Nylén <bjorn.nylen@ub.lu.se> --- Created attachment 113501 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=113501&action=edit 109626: Bug 26312: Catch ES Client errors, log, and continue indexing when error encountered This didn't apply cleanly, fixed the conflict in Koha/SearchEngine/Elasticsearch/Indexer.pm . We will try to test later as we're having timeout issues as well. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #7 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 114229 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=114229&action=edit Bug 26312: (follow-up) Reset buffers even if commit fails -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kyle@bywatersolutions.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |major -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #114229|0 |1 is obsolete| | --- Comment #11 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 117619 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=117619&action=edit Bug 26312: (follow-up) Reset buffers even if commit fails Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ere.maijala@helsinki.fi -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #13 from Björn Nylén <bjorn.nylen@ub.lu.se> --- (In reply to Martin Renvoize from comment #9)

Did you have a chance to test this Björn?

Sorry, didn't figure out a good way to force a timeout in my testingenv. Worked around the timeouts in other ways so it got forgotten. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Joonas Kylmälä <joonas.kylmala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |joonas.kylmala@helsinki.fi CC| |joonas.kylmala@helsinki.fi -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #16 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- (In reply to Joonas Kylmälä from comment #15)

This seems like a great improvement. Looking into this right now. In the future we could also add retry mechanism when encountering a timeout in indexing.

I'd love to see retry and things in the future.. This is a great first step though, thanks for taking a look -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Joonas Kylmälä <joonas.kylmala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #18 from Nick Clemens <nick@bywatersolutions.com> --- Created attachment 119014 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=119014&action=edit Bug 26312: (follow-up) Fix whitespace and missing semicolon -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #20 from Nick Clemens <nick@bywatersolutions.com> --- (In reply to Joonas Kylmälä from comment #19)

Now it gives the following warnings:

...

Use of uninitialized value in string eq at /kohadevbox/koha/misc/search_tools/rebuild_elasticsearch.pl line 352.

and

...

[8012] Use of uninitialized value $msg in concatenation (.) or string at /kohadevbox/koha/misc/search_tools/rebuild_elasticsearch.pl line 379.

Hmmm, because of the last error/warning I think the intended error message for timeout doesn't come up?

Hi Joonas, can you detail how you are triggering the error/simulating a timeout? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |victor@tuxayo.net --- Comment #23 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Tested the thing with koha-elasticsearch -v -p 5 --commit 1000 --rebuild kohadev and running docker restart koha_es_1 when the process was at step "Processing slice 5 of 5" It still died even though the output was different: [995] Processing slice 5 of 5 [995] Indexing biblios [796] Committing final records... [992] Committing final records... [Bad response received when submitting request to Elasticsearch][Bad response received when submitting request to Elasticsearch][995] Committing final records... [994] Committing final records... Something went wrong rebuilding indexes for kohadev root@kohadevbox:koha(testing-SO-QA)$ [Bad response received when submitting request to Elasticsearch][Bad response received when submitting request to Elasticsearch][993] Committing final records... [Bad response received when submitting request to Elasticsearch] note that shell prompt appeared when processing was still happening in the background and additional error message where outputted after. Did I miss something? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #25 from Victor Grousset/tuxayo <victor@tuxayo.net> --- (In reply to Joonas Kylmälä from comment #21)

you can download and import for example https://ia600208.us.archive.org/22/items/marc_western_washington_univ/ wwu_bibs.mrc_revrev.mrc to kohadevbox and after few thousand of those are in start indexing with:

...

koha-elasticsearch -v -p 5 --commit 1000 --rebuild kohadev

Upload worked but I couldn't complete the staging so couldn't try to import. The process took all my free ram and swap (6.5 GiB) and I killed it before it would cause a system freeze or other processes to be killed. Should one just have more that 6.5 G free ram+swap to be able to complete the staging or did I went in the wrong direction? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #27 from Joonas Kylmälä <joonas.kylmala@helsinki.fi> --- (In reply to Victor Grousset/tuxayo from comment #26)

Each time the reindex went without issue. Are there additional steps?

No other steps (except don't have too fast computer I guess) And it required the biblios to be imported to work. The importing works on command line, I used bulkmarcimport.pl if I don't remember wrong. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Joonas Kylmälä <joonas.kylmala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #117618|0 |1 is obsolete| | Attachment #117619|0 |1 is obsolete| | Attachment #119014|0 |1 is obsolete| | Attachment #119079|0 |1 is obsolete| | --- Comment #28 from Joonas Kylmälä <joonas.kylmala@helsinki.fi> --- Created attachment 119612 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=119612&action=edit Bug 26312: Catch ES Client errors, log, and continue indexing when error encountered This catches a timeout response from the ES server, logs this, and continues the indexing To test: 1 - perl misc/search_tools/rebuild_elasticsearch.pl 2 - Make the ES server timeout (I don't have good instruction yet) 3 - Watch the job crash 4 - Apply patches 5 - perl misc/search_tools/rebuild_elasticsearch.pl 6 - Make the server timeout 7 - Note the job reports failed commit, and continues Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #30 from Joonas Kylmälä <joonas.kylmala@helsinki.fi> --- Created attachment 119614 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=119614&action=edit Bug 26312: (follow-up) Fix whitespace and missing semicolon Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #32 from Joonas Kylmälä <joonas.kylmala@helsinki.fi> --- The error is handled now nicely:

[30266] Elasticsearch exception thrown: Timeout

and this QA tool error seems to be false positive:

FAIL pod coverage POD is missing for 'description'

Passing QA. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #34 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 119700 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=119700&action=edit Bug 26312: (follow-up) Fix passing of exception messages 'error' has special meaning in exceptions so naming the fields: type, details Rather than only dealing with a single exception type, we generically get the ES exception info and pass it up. I could not recreate timeout still, however, I simply restarted the ES docker during commit stage to cause NoNodes exceptions Signed-off-by: Joonas Kylmälä <joonas.kylmala@helsinki.fi> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |21.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|21.05.00 |21.05.00,20.11.05 released in| | Status|Pushed to master |Pushed to stable --- Comment #37 from Fridolin Somers <fridolin.somers@biblibre.com> --- Pushed to 20.11.x for 20.11.05 -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 --- Comment #39 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Hi, trying to backport to 19.11.x. I had a conflict but I'm not very sure about it's resolution. Here it is. == The conflict == 3 parts of the conflict view: <<<<<<< HEAD THIS IS 19.11 before the patch ||||||| THIS IS 20.05 before the patch ======= THIS IS 20.05 after the patch

...

...

...

...

...

>

↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ <<<<<<< HEAD $response = $elasticsearch->bulk( index => $conf->{index_name}, type => 'data', # is just hard coded in Indexer.pm? body => \@body ); ||||||| parent of a60a3b26d7 (Bug 26312: Catch ES Client errors, log, and continue indexing when error encountered) my $elasticsearch = $self->get_elasticsearch(); $response = $elasticsearch->bulk( index => $self->index_name, type => 'data', # is just hard coded in Indexer.pm? body => \@body ); if ($response->{errors}) { carp "One or more ElasticSearch errors occurred when indexing documents"; } ======= try{ my $elasticsearch = $self->get_elasticsearch(); $response = $elasticsearch->bulk( index => $self->index_name, type => 'data', # is just hard coded in Indexer.pm? body => \@body ); if ($response->{errors}) { carp "One or more ElasticSearch errors occurred when indexing documents"; } } catch { if( ref $_ eq 'Search::Elasticsearch::Error::Timeout' ){ Koha::Exceptions::Elasticsearch::BadResponse->throw( error => "Record commit failed.", details => "Timeout", ); } };

...

...

...

...

...

> a60a3b26d7 (Bug 26312: Catch ES Client errors, log, and continue indexing when error encountered)

== The resolution == try{ $response = $elasticsearch->bulk( index => $conf->{index_name}, type => 'data', # is just hard coded in Indexer.pm? body => \@body ); } catch { if ( ref $_ eq 'Search::Elasticsearch::Error::Timeout' ) { Koha::Exceptions::Elasticsearch::BadResponse->throw( error => "Record commit failed.", details => "Timeout", ); } }; But I'm not sure not to have caused a side effect when filtering the changes to keep between 19.11 and 20.05 There are also two other minor conflicts. Not worried about these. == Test == misc/search_tools/rebuild_elasticsearch.pl -v -d -c 1 # PLUS docker restart koha_es_1 # during the process # It did show the expected difference with and without the patch But still worried about an eventual side effect. If someone would like to have this in 19.11.x some review is need. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=26312 Andrew Nugged <nugged@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nugged@gmail.com -- You are receiving this mail because: You are watching all bug changes.