[koha-commits] main Koha release repository branch master updated. v16.11.00-822-g0868b70
Git repo owner
gitmaster at git.koha-community.org
Thu Apr 13 14:54:16 CEST 2017
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "main Koha release repository".
The branch, master has been updated
via 0868b709811078405a834114e779e0a35184d54d (commit)
via 630896330b0b5249c1a53342cdd1458e5677f7ee (commit)
via dea93375dcc147280e93153b0a7e3b90ce66b4a0 (commit)
via d49e8bd5e22a0b39e5628b24a033badc44a07bba (commit)
via 7d536702792e2ff4d6a083b5ce5aa670fc88e0f9 (commit)
via f953cc26e6e9569f6674e6186efc3e37b5c721b2 (commit)
via 04263912c7e413c5dbf13fe61dcdfa4b4912ab56 (commit)
via 06647d29a7c02b6483f27f6c46e9d0e639236d6e (commit)
via 4c762ba69c0a85f3c5a3add40f8f4c16081965fe (commit)
via 2bb870b3f0cadef851497ab1d4ad8783f47ac8f2 (commit)
via 25c87a754203f38267eba219d34f86b5922be298 (commit)
via 5cc310238558ceee83cb340c0246d47d880ccf57 (commit)
via a26ec1b47f3834b529b75c79d3975be0d16f2ad4 (commit)
via 7effc34cd251dd6e3841c365015ba8c9d4e68b46 (commit)
via 7d762b1d8f1487a86c96e46e4fa03857ce0f3fa8 (commit)
from 6727a61578ec2b0ed3ebed182715a3f85c0b33d6 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 0868b709811078405a834114e779e0a35184d54d
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Tue Apr 11 14:58:41 2017 +0200
Bug 9988: [QA Follow-up] Satisfy QA issues
[1] See comment102. Moved sub reporting_tag_xml to MergeRequest.pm.
Adjusted t/db_dependent/Koha/Authorities.t accordingly.
This resolves the C3 inconsistent hierarchy errors.
[2] Removed empty POD section Instance Methods from MergeRequests.
This resolves the POD error in comment102 point 2.
[3] Include a tag 100 for UNIMARC in reporting_tag_xml to resolve an error
on encoding in MARC::File::XML. Subtest for oldmarc and subtest for
reporting_tag_xml adjusted accordingly.
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit 630896330b0b5249c1a53342cdd1458e5677f7ee
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Thu Feb 23 12:23:36 2017 +0100
Bug 9988: Add the cronjob and print an upgrade message
The Debian cron file and the misc crontab example are updated.
A message is printed when upgrading.
Note: At this moment the merge cron job is run once a day. This is imo a
good starting point. The load for this job greatly depends on the value of
pref AuthorityMergeLimit. Of course you can schedule the job more often,
and if this need is felt more globally, we can adjust it later.
Test plan:
[1] Run the dbrev and see the message.
[2] Read the changes to the cron files.
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit dea93375dcc147280e93153b0a7e3b90ce66b4a0
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Wed Feb 22 16:51:58 2017 +0100
Bug 9988: Refactor the cron script
The cron job is moved from migration tools to cronjobs. And renamed to
a plural form. The script is now based on Koha objects. It does no longer
include the code to merge one record. This can be done via the interface,
and will be added to a maintenance script on bug 18071. Should not be part
of this cron job.
Adding a cron_cleanup method to MergeRequests; this method is called from
the cron script to reset older entries still marked in progress and to
also remove old processed entries. Tested in a separate unit test.
Test plan:
[1] Run t/db_dependent/Authorities/MergeRequests.t
[2] Set AuthorityMergeLimit to 0. (All merges are postponed.)
[3] Modify an authority linked to a few records.
[4] Delete an authority linked to a few records with batch delete tool.
[5] And select two auth records with linked records.
Merge these two records with authority/merge.pl.
Note: Do not select Default. See also bug 17380.
[6] Check the need_merge_authorities table for inserted records.
[7] Run misc/cronjobs/merge_authorities.pl -b and inspect the linked
records and the record status in need_merge_authorities.
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit d49e8bd5e22a0b39e5628b24a033badc44a07bba
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Thu Feb 23 12:46:52 2017 +0100
Bug 9988: Few subtle changes for postponed merge
The fails in the previous test showed that we need the first three
changes here. Some final polishing in points 4 to 6.
[1] Sub merge: Refine the condition for initializing $tags_new.
A postponed 'modify'-merge (A to B) makes that $authtypefrom is not
defined when running merge later. When crossing the type boundary, we
need a new field too.
[2] Sub merge: Add condition for an empty @record_to array.
This indicates also that a field should be removed, since we should
otherwise only add a $9 subfield.
[3] Sub merge: Adjust initializing @record_from.
This change is tested by verifying a cleared subfield in the test.
[4] DelAuthority: Adding a skipmerge parameter to allow the call from
authorities/merge.pl to skip an unneeded merge.
This also prevents that the 'delete-merge' would precede the
'modify-merge' under a hypothetical race condition.
[5] DelAuthority: There is actually no need to call GetAuthority.
The subfields of the old record are not relevant in this case.
[6] Added a few POD lines to merge.
[7] Removed a trailing space in a comment line in merge.
Test plan:
[1] Run t/db_dependent/Authorities/Merge.t.
The last subtest should no longer fail now.
[2] See test plan of next patch.
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit 7d536702792e2ff4d6a083b5ce5aa670fc88e0f9
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Thu Feb 23 12:46:52 2017 +0100
Bug 9988: Add a subtest for specific postponed merge issues
This subtest shows that we need a few little tweaks to make merge
work in some specific postponed merge situations.
Test plan:
[1] Run t/db_dependent/Authorities/Merge.t.
The last subtest should fail.
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit f953cc26e6e9569f6674e6186efc3e37b5c721b2
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Wed Feb 22 10:53:03 2017 +0100
Bug 9988: Remove further references to dontmerge
[1] The preference was sent to HEA. We can now send both AuthorityMergeMode
as well as AuthorityMergeLimit.
[2] A comment in authorities/merge.pl is removed. Note that a subsequent
patch will modify and test the cron job.
[3] Script misc/batchRebuildItemsTables.pl temporarily enabled dontmerge.
This is equivalent to setting the mergelimit to zero.
The function defnonull is no longer needed. (If the pref was NULL,
we restore that value. Sub merge won't mind.)
Test plan:
[1] Run t/db_dependent/UsageStats.t
[2] Run misc/batchRebuildItemsTables.pl -t
This just ensures you it still compiles; the changes speak for itself.
[3] Now git grep on dontmerge.
You should only find hits in atomicupdate and misc/translator/po.
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit 04263912c7e413c5dbf13fe61dcdfa4b4912ab56
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Tue Feb 21 11:54:23 2017 +0100
Bug 9988: Check the merge limit in sub merge
At this point, we are replacing dontmerge functionality by the new
AuthorityMergeLimit logic. Instead of doing this check before calling
merge, we just call merge and check it there. In order to let the cron
job do the larger (postponed) merges, we add a parameter override_limit.
A subtest is added in Merge.t to test the 'postponed merge' feature. Since
merge now also calls get_usage_count, an additional mock is added. All
references to dontmerge are removed.
In merge two lines, initializing $dbh and $counteditbiblios, are moved.
The dontmerge test in DelAuthority and ModAuthority is removed. Since this
did not leave much in ModAuthority, I fixed the whitespace on the remaining
lines rightaway (yes, I know).
A minimal set of changes is applied to the cron script; it will get further
attention on a next patch.
Test plan:
[1] Run t/db_dependent/Authorities/Merge.t
[2] Set AuthorityMergeLimit to 2. Modify an authority with two linked
biblios. Check that the merge was done immediately.
[3] Now modify an authority with more than 2 linked records.
Verify that the merge was postponed; a record must be inserted in
the need_merge_authorities table.
[4] Testing of the merge cron job is *postponed* to a next patch.
Note: I tested a modification, but the script just needs more
attention.
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit 06647d29a7c02b6483f27f6c46e9d0e639236d6e
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Tue Feb 21 09:22:59 2017 +0100
Bug 9988: Remove the Zebra code from sub merge
Since we can now call linked_biblionumbers, we can now remove all Zebra
related code from merge. We also add a parameter biblionumbers; we use it
in the test now, but it may be handy too later in the maintenance script
when we want to trigger a merge for specific biblionumber(s). See bug
report 18071.
All mocks for ZOOM, Context::Zconn, Search::new_record_for_zebra in the
merge test can now be replaced by one mock for linked_biblionumbers. Note
that we test the biblionumbers parameter in the last subtest without that
mock.
Remove unused vars $countunmodifiedbiblio, $counterrors from merge.
Renamed zebrarecords to linkedrecords in the Merge test.
Test plan:
[1] Run t/db_dependent/Authorities/Merge.t
[2] Modify an authority record. Check the linked biblio records.
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit 4c762ba69c0a85f3c5a3add40f8f4c16081965fe
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Tue Feb 21 08:39:24 2017 +0100
Bug 9988: Merge should have a parameter hash
We will need a few additional parameters for merge later on. This patch
puts the original parameters in a parameter hash.
For the same reason DelAuthority gets a parameter hash here.
Note: We remove the second parameter from the DelAuthority call in
authorities/authorities-home.pl here. It was not used and could have
presented problems in the future.
Test plan:
[1] Run t/db_dependent/AuthoritiesMarc.t.
[2] Run t/db_dependent/Authorities/Merge.t.
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit 2bb870b3f0cadef851497ab1d4ad8783f47ac8f2
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Wed Feb 15 15:48:17 2017 +0100
Bug 9988: Add get_usage_count and linked_biblionumbers to Koha::Authority
When replacing the Zebra code in sub merge, we actually need CountUsage
as well as the results it gets from SearchEngine.
In order to get count and/or results, we now create:
[1] instance methods get_usage_count and linked_biblionumbers in
Koha::Authority,
[2] class methods of the same name in Koha::Authorities.
The instance method calls the class method. The class method can be used
for deleted authority records, and in 'legacy calls' before refactoring.
Note: On BZ 18149 we will replace all CountUsage calls.
Test plan:
Run t/db_dependent/Koha/Authorities.t
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit 25c87a754203f38267eba219d34f86b5922be298
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Wed Feb 15 15:48:17 2017 +0100
Bug 9988: Add extract_biblionumber to Koha/SearchEngine
When we will replace the Zebra code in sub merge, we will call SearchEngine
to pass records and we need a routine to extract a biblionumber from
a search result record. A record from Zebra still must be converted to
MARC::Record. This is no longer needed for a ElasticSearch record.
Test plan:
Run t/db_dependent/Koha/SearchEngine/Search.t
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit 5cc310238558ceee83cb340c0246d47d880ccf57
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Sun Jan 1 11:50:44 2017 +0100
Bug 9988: Add Koha objects for table need_merge_authorities
This patch adds two Koha objects: MergeRequest(s).
MergeRequest has a new method and an oldmarc method.
A class method reporting_tag_xml is added to MergeRequests.pm.
All new routines are tested in Authorities.t.
Removes a few unneeded modules from Koha::Authority.
Test plan:
Run t/db_dependent/Koha/Authorities.t
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit a26ec1b47f3834b529b75c79d3975be0d16f2ad4
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Tue Feb 14 09:07:27 2017 +0100
Bug 9988: DBIx schema changes for need_merge_authorities
As generated by misc/devel/update_dbix_class_files.pl.
No test plan.
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit 7effc34cd251dd6e3841c365015ba8c9d4e68b46
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Tue Feb 14 08:42:04 2017 +0100
Bug 9988: Table revision for need_merge_authorities
In order to let merge do exactly the same when called from the merge
cron job, we need three additional fields in the table. The use of these
fields will be explained in a follow-up.
Test plan:
Run the dbrev (updatestructure step in webinstaller).
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Also tested running kohastructure.sql for this table.
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
commit 7d762b1d8f1487a86c96e46e4fa03857ce0f3fa8
Author: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Date: Tue Jan 10 11:39:31 2017 +0100
Bug 9988: Database revision for AuthorityMergeLimit
Introduce the new preference and remove dontmerge.
If dontmerge was enabled, the new pref will be set to zero and we will
print a message about raising this value now.
Test plan:
Run the database revision or a new install.
In case of an upgrade, check the new pref value. Should be 0 when
dontmerge was enabled, otherwise 50.
Signed-off-by: Marcel de Rooy <m.de.rooy at rijksmuseum.nl>
Also tested sysprefs.sql on a new database.
Signed-off-by: Marc Véron <veron at veron.ch>
Signed-off-by: Jacek Ablewicz <abl at biblos.pk.edu.pl>
Signed-off-by: Julian Maurice <julian.maurice at biblibre.com>
Signed-off-by: Kyle M Hall <kyle at bywatersolutions.com>
-----------------------------------------------------------------------
Summary of changes:
C4/AuthoritiesMarc.pm | 185 ++++++++++----------
C4/ImportBatch.pm | 2 +-
C4/UsageStats.pm | 3 +-
Koha/Authorities.pm | 57 ++++++
Koha/Authority.pm | 38 +++-
Koha/Authority/MergeRequest.pm | 141 +++++++++++++++
Koha/Authority/MergeRequests.pm | 109 ++++++++++++
Koha/Schema/Result/NeedMergeAuthority.pm | 32 +++-
Koha/SearchEngine/Elasticsearch/Search.pm | 16 ++
Koha/SearchEngine/Search.pm | 23 ++-
Koha/SearchEngine/Zebra/Search.pm | 17 ++
authorities/authorities-home.pl | 2 +-
authorities/authorities.pl | 2 +-
authorities/merge.pl | 8 +-
debian/koha-common.cron.daily | 1 +
installer/data/mysql/atomicupdate/bug_9988a.perl | 19 ++
.../data/mysql/atomicupdate/bug_9988b_table.perl | 14 ++
.../data/mysql/atomicupdate/bug_9988c_cron.perl | 6 +
installer/data/mysql/kohastructure.sql | 11 +-
installer/data/mysql/sysprefs.sql | 2 +-
.../en/modules/admin/preferences/authorities.pref | 10 +-
misc/batchRebuildItemsTables.pl | 14 +-
misc/cronjobs/crontab.example | 3 +
misc/cronjobs/merge_authorities.pl | 89 ++++++++++
misc/migration_tools/merge_authority.pl | 103 -----------
misc/migration_tools/remove_unused_authorities.pl | 2 +-
t/db_dependent/Authorities/Merge.t | 149 +++++++++++-----
t/db_dependent/Authorities/MergeRequests.t | 44 +++++
t/db_dependent/AuthoritiesMarc.t | 2 +-
t/db_dependent/Koha/Authorities.t | 169 +++++++++++++++++-
t/db_dependent/Koha/SearchEngine/Search.t | 63 +++++++
t/db_dependent/UsageStats.t | 3 +-
tools/batch_delete_records.pl | 2 +-
33 files changed, 1050 insertions(+), 291 deletions(-)
create mode 100644 Koha/Authority/MergeRequest.pm
create mode 100644 Koha/Authority/MergeRequests.pm
create mode 100644 installer/data/mysql/atomicupdate/bug_9988a.perl
create mode 100644 installer/data/mysql/atomicupdate/bug_9988b_table.perl
create mode 100644 installer/data/mysql/atomicupdate/bug_9988c_cron.perl
create mode 100755 misc/cronjobs/merge_authorities.pl
delete mode 100755 misc/migration_tools/merge_authority.pl
create mode 100644 t/db_dependent/Authorities/MergeRequests.t
create mode 100644 t/db_dependent/Koha/SearchEngine/Search.t
hooks/post-receive
--
main Koha release repository
More information about the koha-commits
mailing list