https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #1 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- It would be great if this could get some attention as it can create data loss. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Nick Clemens <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com, | |martin.renvoize@ptfs-europe | |.com, | |nick@bywatersolutions.com, | |tomascohen@gmail.com --- Comment #3 from Nick Clemens <nick@bywatersolutions.com> --- So, the way it seems to work is, if there are any rules for the context, then we don't get the '*' rules i.e. if you are going to define rules for z3950, then you must define all the rules you want - but if you don't define any rules for a specific filter, then the '*' rules are used Is this the intention, Tomas? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |glasklas@gmail.com --- Comment #4 from David Gustafsson <glasklas@gmail.com> --- Yes, even though this behaviour seems be confusing users, that was the intention. '*' rules are never "merged in" into another rule set, they just serve as defaults where no rules matching the current context are found. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #6 from David Gustafsson <glasklas@gmail.com> --- Context is module and filter, so if rules are applied with either z3950 or batchmod, '*' rules are ignored. '*' rules are only applied if there are no rules defined for the current context (module + filter). I think I decided on this so that each context would have a clean slate and though that would be easier to maintain, not having to undo wildcard rules for a new context, but I can see how it's not the most intuitive. For our library for example, we want to protect quite a lot of fields on imports, but want everything to be overwritable in the staff interface. So we define the "import" rules in the '*' context, and can just add one tag wildcard rule with for the staff interface to make everything overwritable again. I think I assumed that would be the most common use case. Merging in '*' would complicate things for us (but perhaps make things easier for others). Another reason avoided this was that it complicates the code quite a bit, but I think it would still be doable. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #8 from David Gustafsson <glasklas@gmail.com> --- To summarize how the rules would be applied in a less convoluted way, for a specific tag and context: First all the rules for the matching module filter combination (context) are evaluated, if there is a match, that rule applies. If no rule was matched then rules for the wildcard filter are evaluated. There is another issue to address, and that is module priority. Say we have two or more matching context with different modules, which one applies? I think this is an oversight in the current implementation, and we have to make sure we loop through modules in a specific predefined order. Right now we just loop though the hash keys of "context" provided to the function merge_record in effect resulting in random order. I will have a go at patch addressing both the merging issue, and priority of modules. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |In Discussion --- Comment #10 from David Gustafsson <glasklas@gmail.com> --- The attached patch needs some more work. Have done some manual testing and I think it's probably ok, but need to add tests before ready for review. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #12 from David Gustafsson <glasklas@gmail.com> --- Now added some tests, it would probably be a good idea to add a test for multiple contexts with regards to module precedence. Besides from that it should be ready for review. Also have not run the qa-script yet. Don't know how much more time I currently have to spend on this though, it should more or less be ready for review. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #14 from David Gustafsson <glasklas@gmail.com> --- Created attachment 159058 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=159058&action=edit Bug 33268: Add tests for context module priority -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |glasklas@gmail.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #159004|0 |1 is obsolete| | --- Comment #16 from David Nind <david@davidnind.com> --- Created attachment 159160 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=159160&action=edit Bug 33268: Overlay rules don't work correctly when source is set to * Fallback to overlay rules with wildcard filter if no match found for exact filter match. To test: 1) Run tests t/db_dependent/Biblio/MarcOverlayRules.t Sponsored-by: Gothenburg University Library Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #18 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- WARN Koha/MarcOverlayRules.pm WARN tidiness The file is less tidy than before (bad/messy lines before: 114, now: 128) WARN t/db_dependent/Biblio/MarcOverlayRules.t WARN tidiness The file is less tidy than before (bad/messy lines before: 226, now: 285) Please tidy inline. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #20 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- We are keen on the feature, but not on the right version yet. I know of at least one big library that will be using it in production very soon. I found the current behaviour with * confusing and we also had others reporting about it. But I haven't been able to follow the latest developments here to give an opinion about the planned changes yet. It looks like we are missing a test plan that highlights the change. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #23 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 159519 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=159519&action=edit Bug 33268: Add tests for context module priority Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Edit: tidied inline (tcohen) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #159518|0 |1 is obsolete| | --- Comment #25 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 160017 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=160017&action=edit Bug 33268: Overlay rules don't work correctly when source is set to * Fallback to overlay rules with wildcard filter if no match found for exact filter match. To test: 1) Run tests t/db_dependent/Biblio/MarcOverlayRules.t Sponsored-by: Gothenburg University Library Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Edit: tidied inline (tcohen) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #27 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Great teamwork here.. QA scripts happy and functionality now works more as described. Passing QA -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #29 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- *ping* This is waiting now waiting for feedback for more than 2 weeks. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Ray Delahunty <r.delahunty@arts.ac.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |r.delahunty@arts.ac.uk -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff --- Comment #32 from David Gustafsson <glasklas@gmail.com> --- @Katrin Fischer Sorry about the late reply. 1) I don't know if I can summarize it better than the current commit message, perhaps I could add "instead of falling back to the default rule (overwrite), which was the previous behavior. Previously the wildcard filter rules where only applied if no other filter value would match. The added tests should also be equivalent to performing manual testing, so don't really see why we would need both, but I guess I could add some simple case to be performed in the staff interface. 2) Gave this some thought and I think there is a relatively simple way to add rules depending on existing rules that would result in this patch not at all affecting behavior for current used defined rules. Will add a database migration for this. 3) Haven't done this for any other patch, so a little bit uncertain what it should contain. About your rules the wildcard filter rule will be merged in to the rule-sets of both the other rules. If the filter value is z3950 rule 2 and 4 will be applied, if batchmod 4 and five. If instead of 245 you had *, the wildcard filter rule (for tag 300) would not be included since the rule for z3950 and batchmod would always match, and rules with a specific filter value take precedence over wildcard filter rules if they match. (As summarized with "Fallback to overlay rules with wildcard filter if no match found for exact filter match." in the commit message. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162764|0 |1 is obsolete| | --- Comment #34 from David Gustafsson <glasklas@gmail.com> --- Created attachment 162766 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162766&action=edit Bug 33268: If rules exists add rules sure to preserve current behaviour -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #36 from David Gustafsson <glasklas@gmail.com> --- 2) Should now be fixed -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162767|0 |1 is obsolete| | --- Comment #38 from David Gustafsson <glasklas@gmail.com> --- Created attachment 162785 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162785&action=edit Bug 33268: If rules exists add rules to preserve current behaviour -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #40 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 162796 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162796&action=edit Bug 33268: Add tests Sponsored-by: Gothenburg University Library Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #160017|0 |1 is obsolete| | --- Comment #42 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 162798 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162798&action=edit Bug 33268: Overlay rules don't work correctly when source is set to * Fallback to overlay rules with wildcard filter if no match found for exact filter match. To test: 1) Run tests t/db_dependent/Biblio/MarcOverlayRules.t Sponsored-by: Gothenburg University Library Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Edit: tidied inline (tcohen) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #44 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 162800 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162800&action=edit Bug 33268: (QA follow-up): tidy up code -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162796|0 |1 is obsolete| | --- Comment #46 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 162801 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162801&action=edit Bug 33268: Add tests Sponsored-by: Gothenburg University Library Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162798|0 |1 is obsolete| | --- Comment #48 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 162803 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162803&action=edit Bug 33268: Overlay rules don't work correctly when source is set to * Fallback to overlay rules with wildcard filter if no match found for exact filter match. To test: 1) Run tests t/db_dependent/Biblio/MarcOverlayRules.t Sponsored-by: Gothenburg University Library Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Edit: tidied inline (tcohen) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162800|0 |1 is obsolete| | --- Comment #50 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 162805 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162805&action=edit Bug 33268: (QA follow-up): tidy up code -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #52 from David Gustafsson <glasklas@gmail.com> --- About the tests there is no difference testing manually vs the test cases with the exception if there is an incorrect overlay_context parameter for ModBiblio somewhere, and that has already been manually testing and would also probably have been discovered in production if where the case. Of course there could also be some testing scenario not covered by the tests (and most should be), but in that case we should just add that test in the test suite instead. I'm not able to reproduce your test case, for me it seems to work. Tried both with the same rules, and some different permutations. I will try again with your version of the patch. Also made a script to print out the resulting rule-set, and it produces the expected result. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #54 from Victor Grousset/tuxayo <victor@tuxayo.net> ---

I'm not able to reproduce your test case, for me it seems to work. [...] Works for me also with the latest version of the patch

So you followed the steps and at the 11th you got the subfield protected as expected? :D (vs at the 6th were it got changed) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #56 from Victor Grousset/tuxayo <victor@tuxayo.net> --- (In reply to David Gustafsson from comment #55)

Yes, exactly.

I retried and it works!!! :D I might have messed something because I should have been blocked at step:

8. Change the values of the two subfields to something else

Because the * overlay rules also include the editor. And back then I was able to change the values with the editor and now as expected, I can't. So likely had mess the rules. (You likely hit that too with my bogus test plan and that was an hint that I did something wrong) Anyway, I'm fixing the test plan and putting it in the patch commit message. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162802|0 |1 is obsolete| | --- Comment #58 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 162937 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162937&action=edit Bug 33268: Add tests for context module priority Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Edit: tidied inline (tcohen) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162804|0 |1 is obsolete| | --- Comment #60 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 162939 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=162939&action=edit Bug 33268: If rules exists add rules to preserve current behavior Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #62 from Victor Grousset/tuxayo <victor@tuxayo.net> --- It works :) -- @QA here is an example of the changes from the DB upgrade on the data used in test plan: +----+-----+--------+----------+-----+--------+--------+--------+ | id | tag | module | filter | add | append | remove | delete | +----+-----+--------+----------+-----+--------+--------+--------+ | 2 | 245 | source | z3950 | 0 | 0 | 0 | 0 | | 3 | 300 | source | * | 0 | 0 | 0 | 0 | | 4 | 245 | source | batchmod | 0 | 0 | 0 | 0 | +----+-----+--------+----------+-----+--------+--------+--------+ ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ +----+-----+--------+----------+-----+--------+--------+--------+ | id | tag | module | filter | add | append | remove | delete | +----+-----+--------+----------+-----+--------+--------+--------+ | 2 | 245 | source | z3950 | 0 | 0 | 0 | 0 | | 3 | 300 | source | * | 0 | 0 | 0 | 0 | | 4 | 245 | source | batchmod | 0 | 0 | 0 | 0 | | 12 | * | source | z3950 | 1 | 1 | 1 | 1 | | 13 | * | source | batchmod | 1 | 1 | 1 | 1 | +----+-----+--------+----------+-----+--------+--------+--------+ --- @David G. to address Comment 28 's point 3 (release notes) you can look at latest release announcement: https://koha-community.org/koha-23-11-released/ Some tickets have grey boxes with text. That's from the "Text to go in the release notes" field here. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #63 from David Gustafsson <glasklas@gmail.com> --- @Victor Grousset/tuxayo Ok, thanks for the feedback and test plan. I now submitted a release note. It's not that easy to explain the change in clear terms, so suggestions are welcome if there is room for improvements. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the|This enhancement changes |This enhancement changes release notes|how marc overlay rules are |how marc overlay rules are |evaluated. Before the |evaluated. Before the |change rules with filter |change rules with filter |set to '*' (wildcard) where |set to '*' (wildcard) where |only applied when no other |only applied when no other |rules had been defined for |rules had been defined for |a specific filter value, |a specific filter value, |regardless of if any of the |regardless of if any of the |rules with that filter |rules with that filter |value matched or not. |value matched or not. | | |With |With |this change we fallback to |this change we fallback to |the wildcard filter rules |the wildcard filter rules |even though rules with a |even though rules with a |matching filter value do |matching filter value do |exists, if no rule for that |exists, if no rule for that |filter value matches. This |filter value matches. This |resolves the issue of |resolves the issue of |having to repeat the same |having to repeat the same |default rules for each |default rules for each |filter value rule set. If |filter value rule set. If |for some filter value the |for some filter value the |wildcard filter rules |wildcard filter rules |should be overridden, a |should be overridden, a |wildcard tag rule for that |wildcard tag rule for that |filter value can be defined |filter value can be defined |which will have higher |which will have higher |precedence and override all |precedence and override all |the filter wildcard rules. |the filter wildcard rules. | | |In summary, the rules will |In summary, the rules will |applied as follows: |applied as follows: | | |- A |- A |matching rule is looked for |matching rule is looked for |based on context (module |based on context (module |and filter value), if |and filter value), if |multiple contexts matches |multiple contexts matches |rules for the module with |rules for the module with |highest priority are used. |highest priority are used. |The module priority is |The module priority is |(from highest to lowest) |(from highest to lowest) |"User name", "Patron |"User name", "Patron |category" and "Source" |category" and "Source". | | |- |- |If no matching rule is |If no matching rule is |found, we fallback to the |found, we fallback to the |wildcard filter rules of |wildcard filter rules of |the current context. |the current context module. | | |For |For Koha installations |Koha installations where |where marc overlay rules is |marc overlay rules is in |in use a database migration |use a database migration |will run that creates new |will run that creates new |rules if necessary to |rules if necessary to |preserve the current |preserve the current |behavior, so no user action |behavior, so no user action |needs to be taken in order |needs to be taken in order |to correct for this change. |to correct for this change. | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the|This enhancement changes | release notes|how marc overlay rules are | |evaluated. Before the | |change rules with filter | |set to '*' (wildcard) where | |only applied when no other | |rules had been defined for | |a specific filter value, | |regardless of if any of the | |rules with that filter | |value matched or not. | | | |With | |this change we fallback to | |the wildcard filter rules | |even though rules with a | |matching filter value do | |exists, if no rule for that | |filter value matches. This | |resolves the issue of | |having to repeat the same | |default rules for each | |filter value rule set. If | |for some filter value the | |wildcard filter rules | |should be overridden, a | |wildcard tag rule for that | |filter value can be defined | |which will have higher | |precedence and override all | |the filter wildcard rules. | | | |In summary, the rules will | |applied as follows: | | | |- A | |matching rule is looked for | |based on context (module | |and filter value), if | |multiple contexts matches | |rules for the module with | |highest priority are used. | |The module priority is | |(from highest to lowest) | |"User name", "Patron | |category" and "Source". | | | |- | |If no matching rule is | |found, we fallback to the | |wildcard filter rules of | |the current context module. | | | |For Koha installations | |where marc overlay rules is | |in use a database migration | |will run that creates new | |rules if necessary to | |preserve the current | |behavior, so no user action | |needs to be taken in order | |to correct for this change. | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 David Gustafsson <glasklas@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This enhancement changes release notes| |how marc overlay rules are | |evaluated. Before the | |change rules with filter | |set to '*' (wildcard) where | |only applied when no other | |rules had been defined for | |a specific filter value, | |regardless of if any of the | |rules with that filter | |value matched or not. With | |this change we fallback to | |the wildcard filter rules | |even though rules with a | |matching filter value do | |exists, if no rule for that | |filter value matches. This | |resolves the issue of | |having to repeat the same | |default rules for each | |filter value rule set. If | |for some filter value the | |wildcard filter rules | |should be overridden, a | |wildcard tag rule for that | |filter value can be defined | |which will have higher | |precedence and override all | |the filter wildcard rules. | |In summary, the rules will | |applied as follows: - A | |matching rule is looked for | |based on context (module | |and filter value), if | |multiple contexts matches | |rules for the module with | |highest priority are used. | |The module priority is | |(from highest to lowest) | |"User name", "Patron | |category" and "Source". - | |If no matching rule is | |found, we fallback to the | |wildcard filter rules of | |the current context module. | |For Koha installations | |where marc overlay rules is | |in use a database migration | |will run that creates new | |rules if necessary to | |preserve the current | |behavior, so no user action | |needs to be taken in order | |to correct for this change. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the|This enhancement changes |This enhancement changes release notes|how marc overlay rules are |how marc overlay rules are |evaluated. Before the |evaluated. |change rules with filter | |set to '*' (wildcard) where |Before the |only applied when no other |change rules with filter |rules had been defined for |set to '*' (wildcard) where |a specific filter value, |only applied when no other |regardless of if any of the |rules had been defined for |rules with that filter |a specific filter value, |value matched or not. With |regardless of if any of the |this change we fallback to |rules with that filter |the wildcard filter rules |value matched or not. |even though rules with a | |matching filter value do |With |exists, if no rule for that |this change we fallback to |filter value matches. This |the wildcard filter rules |resolves the issue of |even though rules with a |having to repeat the same |matching filter value do |default rules for each |exists, if no rule for that |filter value rule set. If |filter value matches. This |for some filter value the |resolves the issue of |wildcard filter rules |having to repeat the same |should be overridden, a |default rules for each |wildcard tag rule for that |filter value rule set. If |filter value can be defined |for some filter value the |which will have higher |wildcard filter rules |precedence and override all |should be overridden, a |the filter wildcard rules. |wildcard tag rule for that |In summary, the rules will |filter value can be defined |applied as follows: - A |which will have higher |matching rule is looked for |precedence and override all |based on context (module |the filter wildcard rules. |and filter value), if | |multiple contexts matches |In summary, the rules will |rules for the module with |applied as follows: |highest priority are used. | |The module priority is |- A |(from highest to lowest) |matching rule is looked for |"User name", "Patron |based on context (module |category" and "Source". - |and filter value), if |If no matching rule is |multiple contexts matches |found, we fallback to the |rules for the module with |wildcard filter rules of |highest priority are used. |the current context module. |The module priority is |For Koha installations |(from highest to lowest) |where marc overlay rules is |"User name", "Patron |in use a database migration |category" and "Source". |will run that creates new | |rules if necessary to |- |preserve the current |If no matching rule is |behavior, so no user action |found, we fallback to the |needs to be taken in order |wildcard filter rules of |to correct for this change. |the current context module. | | | |For Koha installations | |where marc overlay rules is | |in use a database migration | |will run that creates new | |rules if necessary to | |preserve the current | |behavior, so no user action | |needs to be taken in order | |to correct for this change. --- Comment #66 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Oh sorry, no idea what happened! :o I didn't even do another real change that day on the ticket. The "Removed" column for my action on the 2024-03-12 in the ticket history makes absolutely no sense with what was previously in the field that got emptied. Like all the lines are in random order. Anyway, thanks for reverting. In the process, the line breaks were lost, I reintroduced them, hopefully nothing weird will happen again ^^" -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA --- Comment #69 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- I didn't get the expected results (mostly) following the test plan Instead of disabling overlay rules to change the record - I updated the MARC modificatoin template for the 300 to set value 'FAILFAILFAILFAIL' After applying, updating, restarting all, I batch modified and the 300 was still updated. My rules match what Victor posts after the update -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #71 from Victor Grousset/tuxayo <victor@tuxayo.net> --- UI: The MARCOverlayRules preference is not set, don't forget to enable it for rules to take effect. No, wait, I just missed that. Retesting. (good, bisecting that would have been tedious) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #73 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Can't believe we still haven't fixed this :( -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Lisette Scheer <lisette@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|release-notes-needed | CC| |lisette@bywatersolutions.co | |m -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Jan Kissig <bibliothek@th-wildau.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162937|0 |1 is obsolete| | --- Comment #75 from Jan Kissig <bibliothek@th-wildau.de> --- Created attachment 175660 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=175660&action=edit Bug 33268: Add tests for context module priority Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Edit: tidied inline (tcohen) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Jan Kissig <bibliothek@th-wildau.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #162939|0 |1 is obsolete| | --- Comment #77 from Jan Kissig <bibliothek@th-wildau.de> --- Created attachment 175662 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=175662&action=edit Bug 33268: If rules exists add rules to preserve current behavior Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #79 from Jan Kissig <bibliothek@th-wildau.de> --- I used the same overlay rules +----+-----+--------+----------+---------+ | id | tag | module | filter | preset | +----+-----+--------+----------+---------+ | 2 | 245 | source | z3950 | protect | | 3 | 300 | source | * | protect | | 4 | 245 | source | batchmod | protect | +----+-----+--------+----------+---------+ And tested according to the test plan - applying the patch - batch modification (245 -> CATS and 300 -> CATS) - 245a and 300a remained the same as before So I would confirm, that rule id=3 acts as expected even when doing a batch modification -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #175659|0 |1 is obsolete| | --- Comment #81 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 175939 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=175939&action=edit Bug 33268: Add tests Sponsored-by: Gothenburg University Library Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #175661|0 |1 is obsolete| | --- Comment #83 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 175941 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=175941&action=edit Bug 33268: Overlay rules don't work correctly when source is set to * Fallback to overlay rules with wildcard filter if no match found for exact filter match. To test: 1. Administration => record overlay rules 2. Add this: Module Filter Tag Preset Added Appended Removed Deleted Source Z39.50 245 Protect Skip Skip Skip Skip Source * 300 Protect Skip Skip Skip Skip Source Batch record modification 245 Protect Skip Skip Skip Skip 3. Add MARC modification templates: Update existing or add new field 245$a with value CATSCATSCATSCATS Update existing or add new field 300$a with value CATSCATSCATSCATS 4. Find a record that has those fields (likely any record would) 5. Use batch record modification on the record 6. 300$a should have been catified (expected, that's the bug) 7. 245$a is preserved 8. Change the value of the 300$a to something else. Temporarly disable MARCOverlayRules to be able to do so. And reenable it. 9. Apply the patch, restart services 10. Use batch record modification on the record 11. 300$a should be preserved, cat protection should have worked 12. Run tests: prove t/db_dependent/Biblio/MarcOverlayRules.t 13. Celebrate! :D Sponsored-by: Gothenburg University Library Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Edit: tidied inline (tcohen) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #175663|0 |1 is obsolete| | --- Comment #85 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 175943 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=175943&action=edit Bug 33268: (QA follow-up): tidy up code Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #175939|0 |1 is obsolete| | --- Comment #87 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Created attachment 176201 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=176201&action=edit Bug 33268: Add tests Sponsored-by: Gothenburg University Library Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #175941|0 |1 is obsolete| | --- Comment #89 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Created attachment 176203 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=176203&action=edit Bug 33268: Overlay rules don't work correctly when source is set to * Fallback to overlay rules with wildcard filter if no match found for exact filter match. To test: 1. Administration => record overlay rules 2. Add this: Module Filter Tag Preset Added Appended Removed Deleted Source Z39.50 245 Protect Skip Skip Skip Skip Source * 300 Protect Skip Skip Skip Skip Source Batch record modification 245 Protect Skip Skip Skip Skip 3. Add MARC modification templates: Update existing or add new field 245$a with value CATSCATSCATSCATS Update existing or add new field 300$a with value CATSCATSCATSCATS 4. Find a record that has those fields (likely any record would) 5. Use batch record modification on the record 6. 300$a should have been catified (expected, that's the bug) 7. 245$a is preserved 8. Change the value of the 300$a to something else. Temporarly disable MARCOverlayRules to be able to do so. And reenable it. 9. Apply the patch, restart services 10. Use batch record modification on the record 11. 300$a should be preserved, cat protection should have worked 12. Run tests: prove t/db_dependent/Biblio/MarcOverlayRules.t 13. Celebrate! :D Sponsored-by: Gothenburg University Library Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Edit: tidied inline (tcohen) Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #175943|0 |1 is obsolete| | --- Comment #91 from Martin Renvoize (ashimema) <martin.renvoize@ptfs-europe.com> --- Created attachment 176205 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=176205&action=edit Bug 33268: (QA follow-up): tidy up code Signed-off-by: Jan Kissig <bibliothek@th-wildau.de> Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #92 from Nick Clemens (kidclamp) <nick@bywatersolutions.com> --- Martin beat me by a few minutes, I also tested, works as expected, the DB upgrade preserves the existing behavior, seems great -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |25.05.00 released in| | Status|Passed QA |Pushed to main -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Paul Derscheid <paul.derscheid@lmscloud.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to main |Pushed to stable Version(s)|25.05.00 |25.05.00,24.11.04 released in| | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 --- Comment #96 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- *** Bug 34191 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33268 Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to stable |Needs documenting CC| |wainuiwitikapark@catalyst.n | |et.nz --- Comment #97 from Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> --- Not backporting to 22.11.x as it wasn't backported to 24.05.x -- You are receiving this mail because: You are watching all bug changes.