[Bug 39104] New: ElasticSearch indexing crashes with exceptio in catch block
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 Bug ID: 39104 Summary: ElasticSearch indexing crashes with exceptio in catch block Change sponsored?: --- Product: Koha Version: 24.11 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: simon.hohl@dainst.org QA Contact: testopia@bugs.koha-community.org With ElasticSearch 8 I get the following error while indexing: ``` Can't locate object method "type" via package "UTF-8 "\xC3" does not map to Unicode at /usr/share/perl5/MARC/File/Encode.pm line 35. " (perhaps you forgot to load "UTF-8 "\xC3" does not map to Unicode at /usr/share/perl5/MARC/File/Encode.pm line 35. "?) at /usr/share/koha/bin/search_tools/rebuild_elasticsearch.pl line 321. Something went wrong rebuilding indexes for ... ``` I am not used to perl, at first I thought I had some weird encoding in my record, but now I think this is actually a bug in the indexing script (rebuild_elasticsearch.pl line 321), which looks like this: (...) while ( my $record = $next->() ) { my $id = $record->id // $record->authid; my $record = $record->record; $count++; if ( $verbose == 1 ) { _log( 1, "$count records processed\n" ) if ( $count % 1000 == 0); } else { _log( 2, "$id\n" ); } push @id_buffer, $id; push @commit_buffer, $record; if ( !( --$commit_count ) ) { _log( 1, "Committing $commit records...\n" ); my $response; try{ $response = $indexer->update_index( \@id_buffer, \@commit_buffer ); _handle_response($response); _log( 1, "Commit complete\n" ); } catch { _log(1,"Elasticsearch exception thrown: ".$_->type."\n"); _log(2,"Details: ".$_->details."\n"); }; $commit_count = $commit; @id_buffer = (); @commit_buffer = (); } } (...) So the catch log is blowing up the indexing process. I can get the script to process most records by changing those two logs in the catch-block, but I am unsure how to fix the logging to see what the underlying issue with my records is. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 Simon Hohl <simon.hohl@dainst.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|ElasticSearch indexing |ElasticSearch indexing |crashes with exceptio in |crashes with exception in |catch block |catch block -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au Summary|ElasticSearch indexing |Elasticsearch indexing |crashes with exception in |crashes with exception in |catch block |catch block -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |38913 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38913 [Bug 38913] Elasticsearch indexing explodes with some oversized records with UTF-8 characters -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 Janusz Kaczmarek <januszop@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |januszop@gmail.com --- Comment #1 from Janusz Kaczmarek <januszop@gmail.com> --- At fist glance it looks very similar to the issue solved by Bug 38913. Could you please verify and maybe confirm that the above mentioned patch fixes your issue? -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on|38913 | See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=38913 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=38913 [Bug 38913] Elasticsearch indexing explodes with some oversized records with UTF-8 characters -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- Yeah this sounds like it could be a duplicate of bug 38913 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 --- Comment #3 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- bug 38913 will be in the not yet released 24.11.02 -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 --- Comment #4 from Simon Hohl <simon.hohl@dainst.org> --- Yes, this is a duplicate. Thank you for pointing me toward the other ticket. :) -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=39104 Simon Hohl <simon.hohl@dainst.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |DUPLICATE --- Comment #5 from Simon Hohl <simon.hohl@dainst.org> --- *** This bug has been marked as a duplicate of bug 38913 *** -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org