https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=44022 Bug ID: 44022 Summary: Elasticsearch: product check workaround is never restored, one failed request breaks the client for the rest of the process Initiative type: --- Sponsorship --- status: Product: Koha Version: Main Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: Searching - Elasticsearch Assignee: koha-bugs@lists.koha-community.org Reporter: johan@imcode.com QA Contact: testopia@bugs.koha-community.org Target Milestone: --- Bug 38345 added a monkey patch of Search::Elasticsearch::Role::Cxn::process_response in Koha::SearchEngine::Elasticsearch. When a response has no x-elastic-product header, it assigns $Search::Elasticsearch::Role::Cxn::PRODUCT_CHECK_VALUE = ''; to skip the product check (needed for OpenSearch). The assignment is permanent. A client side failure (timeout, refused or reset connection) is reported by HTTP::Tiny as a synthetic 599 response without headers, so it triggers the assignment too. From then on the product check expects an empty product header, and every response from a healthy Elasticsearch node, which does send x-elastic-product: Elasticsearch, is rejected with ProductCheck. The node is marked dead and every following request fails with "No nodes are available" (NoNodes), although Elasticsearch is reachable again. A fresh process works, so this only affects long running processes: es_indexer_daemon.pl and Plack workers. They stay broken until restarted. With several nodes configured, failover fails the same way: a failed first node poisons the process and the healthy second node is then rejected. To reproduce: 1. Start a long running process that talks to Elasticsearch, for example misc/workers/es_indexer_daemon.pl, or a script that creates a Koha::SearchEngine::Indexer in a loop and calls update_index(). 2. Stop Elasticsearch for about ten seconds, then start it again. Result: the process keeps failing with NoNodes after Elasticsearch is back, while a new process can reach it. Expected: the process recovers as soon as Elasticsearch answers. Fix: use local, so that the product check is skipped for that one response only. This keeps the OpenSearch behaviour of bug 38345. The patch adds t/Koha/SearchEngine/Elasticsearch/ProductCheck.t, which needs no running Elasticsearch. It fails on the current code and passes with the patch. This is also a prerequisite for recovering from an outage in a long running process (see the follow-up bugs about es_indexer_daemon.pl). -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.