https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29921 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29921 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #129697|0 |1 is obsolete| | --- Comment #2 from Julian Maurice <julian.maurice@biblibre.com> --- Created attachment 130594 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130594&action=edit Bug 29921: Allow to index in ES all records modified since a given date This patch adds an option `--modified-since` to rebuild_elasticsearch.pl If used, all records modified since the given date will be indexed. For biblios it uses biblio_metadata.timestamp and items.timestamp For authorities it uses auth_header.modification_time. Test plan: 1. Create several biblio records and items at different times (or update manually the columns specified above) 2. Look at the records modification time: SELECT MAX(GREATEST(bm.timestamp, COALESCE(i.timestamp, 0))) ts FROM biblio_metadata bm LEFT JOIN items i ON (bm.biblionumber = i.biblionumber) GROUP BY bm.biblionumber ORDER BY ts 3. Run misc/search_tools/rebuild_elasticsearch.pl -b -v --modified-since "$DATE" $DATE can be a date (YYYY-MM-DD) or a datetime (YYYY-MM-DD HH:MM[:SS]) 4. Verify that the correct number of records have been indexed (do the same for authorities:) 5. Create several authority records at different times (or update manually the columns specified above) 6. Look at the records modification time: SELECT modification_time FROM auth_header ORDER BY modification_time 7. Run misc/search_tools/rebuild_elasticsearch.pl -a -v --modified-since "$DATE" $DATE can be a date (YYYY-MM-DD) or a datetime (YYYY-MM-DD HH:MM[:SS]) 8. Verify that the correct number of records have been indexed -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29921 Stéphane Delaune <stephane.delaune@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |stephane.delaune@biblibre.c | |om --- Comment #3 from Stéphane Delaune <stephane.delaune@biblibre.com> --- I just tested (see test plan below): it works, thank you test plan : koha@test-koha:~$ mysql MariaDB [koha]> select max(timestamp) from items; +---------------------+ | max(timestamp) | +---------------------+ | 2022-02-02 10:02:20 | +---------------------+ 1 row in set (0.001 sec) MariaDB [koha]> select max(timestamp) from biblio_metadata; +---------------------+ | max(timestamp) | +---------------------+ | 2022-02-02 10:40:16 | +---------------------+ 1 row in set (0.000 sec) MariaDB [koha]> select max(modification_time) from auth_header; +------------------------+ | max(modification_time) | +------------------------+ | 2022-01-31 16:22:58 | +------------------------+ MariaDB [koha]> select biblionumber from items where itemnumber=842565; +--------------+ | biblionumber | +--------------+ | 509975 | +--------------+ 1 row in set (0.001 sec) MariaDB [koha]> update items set timestamp='2022-02-14 10:02:20' where itemnumber=842565; Query OK, 0 rows affected (0.001 sec) Rows matched: 1 Changed: 0 Warnings: 0 MariaDB [koha]> update biblio_metadata set timestamp='2022-02-14 10:02:20' where biblionumber=509974; Query OK, 0 rows affected (0.001 sec) Rows matched: 1 Changed: 0 Warnings: 0 MariaDB [koha]> update auth_header set modification_time='2022-02-14 10:02:20' where authid=426547; Query OK, 0 rows affected (0.001 sec) Rows matched: 1 Changed: 0 Warnings: 0 MariaDB [koha]> exit; Bye koha@test-koha:~$ ~/src/misc/search_tools/rebuild_elasticsearch.pl -b -v --modified-since '2022-02-14 10:02:19' [16518] Checking state of biblios index [16518] Indexing biblios [16518] Committing final records... [16518] Total 2 records indexed koha@test-koha:~$ ~/src/misc/search_tools/rebuild_elasticsearch.pl -a -v --modified-since '2022-02-14 10:02:19' [16529] Checking state of authorities index [16529] Indexing authorities [16529] Committing final records... [16529] Total 1 records indexed -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29921 solene.ngamga@inlibro.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |solene.ngamga@inlibro.com Status|Needs Signoff |Patch doesn't apply --- Comment #5 from solene.ngamga@inlibro.com --- Bug 29921 - Allow to index in Elasticsearch all records modified since a given date 130594 - Bug 29921: Allow to index in ES all records modified since a given date Apply? [(y)es, (n)o, (i)nteractive] y Applying: Bug 29921: Allow to index in ES all records modified since a given date Using index info to reconstruct a base tree... M Koha/BiblioUtils.pm Falling back to patching base and 3-way merge... Auto-merging Koha/BiblioUtils.pm CONFLICT (content): Merge conflict in Koha/BiblioUtils.pm error: Failed to merge in the changes. Patch failed at 0001 Bug 29921: Allow to index in ES all records modified since a given date hint: Use 'git am --show-current-patch=diff' to see the failed patch When you have resolved this problem run "git bz apply --continue". If you would prefer to skip this patch, instead run "git bz apply --skip". To restore the original branch and stop patching run "git bz apply --abort". Patch left in /tmp/Bug-29921-Allow-to-index-in-ES-all-records-modifie-ZvCefZ.patch -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29921 --- Comment #7 from Fridolin Somers <fridolin.somers@biblibre.com> --- Note Bug 35345 has been pushed -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29921 Mengu Yazicioglu <mengu@devinim.com.tr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mengu@devinim.com.tr -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29921 --- Comment #9 from Julian Maurice <julian.maurice@biblibre.com> --- Can we add conditions on the items and biblio_metadatas tables with the --where option ? What this patch does essentially is: WHERE biblio_metadatas.timestamp >= ? OR items.timestamp >= ? -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=29921 Thomas Klausner <domm@plix.at> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Patch doesn't apply |RESOLVED Resolution|--- |DUPLICATE --- Comment #11 from Thomas Klausner <domm@plix.at> --- *** This bug has been marked as a duplicate of bug 35345 *** -- You are receiving this mail because: You are watching all bug changes.