http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle.m.hall@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff Assignee|henridamien@koha-fr.org |kyle.m.hall@gmail.com -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle.m.hall@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #9329|0 |1 is obsolete| | --- Comment #3 from Kyle M Hall <kyle.m.hall@gmail.com> --- Created attachment 9330 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=9330&action=edit Bug 8015 - Add MARC Modifications Templates The MARC Modification Templates system gives Koha users the power to make alterations to MARC records automatically while staging MARC records for import. This tool is useful for altering MARC records from various venders work with your MARC framework. The system essentially allows one to create a basic script using actions to Copy, Move, Add, Update and Delete fields. Each action can also have an optional condition to check the value or existance of another field. The Copy & Move actions also support Regular Expressions, which can be used to automatically modify field values during the copy/move. An example would be to strip out the '$' character in field 020$c. Furthermore, the value for an update can include variables that change each time the template is used. Currently, the system supports two variables, __BRANCHCODE__ which is replaced with the branchcode of the library currently using the template, and __CURRENTDATE__ which is replaced with the current date in ISO format ( YYYY-MM-DD ). At its simplist, it can perform functions such as: Copy field 092$a to 952$c At its most complex it can run actions like: Copy field 020$c to 020$c using RegEx s/\$// if 020$c equals RegEx m/^\$/ -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle.m.hall@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #9331|0 |1 is obsolete| | --- Comment #5 from Kyle M Hall <kyle.m.hall@gmail.com> --- Created attachment 10089 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=10089&action=edit Bug 8015 - Add MARC Modifications Templates The MARC Modification Templates system gives Koha users the power to make alterations to MARC records automatically while staging MARC records for import. This tool is useful for altering MARC records from various venders work with your MARC framework. The system essentially allows one to create a basic script using actions to Copy, Move, Add, Update and Delete fields. Each action can also have an optional condition to check the value or existance of another field. The Copy & Move actions also support Regular Expressions, which can be used to automatically modify field values during the copy/move. An example would be to strip out the '$' character in field 020$c. Furthermore, the value for an update can include variables that change each time the template is used. Currently, the system supports two variables, __BRANCHCODE__ which is replaced with the branchcode of the library currently using the template, and __CURRENTDATE__ which is replaced with the current date in ISO format ( YYYY-MM-DD ). At its simplist, it can perform functions such as: Copy field 092$a to 952$c At its most complex it can run actions like: Copy field 020$c to 020$c using RegEx s/\$// if 020$c equals RegEx m/^\$/ -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 delaye <stephane.delaye@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff CC| |jonathan.druart@biblibre.co | |m -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 claire.hernandez@biblibre.com <claire.hernandez@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |claire.hernandez@biblibre.c | |om -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #11607|0 |1 is obsolete| | --- Comment #10 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 11608 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11608&action=edit [SIGNED-OFF] Bug 8015: Followup: FIX cp and mv if subfields don't exist. Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 delaye <stephane.delaye@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Failed QA -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #13 from Jonathan Druart <jonathan.druart@biblibre.com> --- *Quick and dirty patch* :-/ But it could fix the issue. Kyle, what do you think about it ? It could introduces some regressions. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #12280|0 |1 is obsolete| | --- Comment #15 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 12288 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=12288&action=edit Bug 8015: Followup: On move, we cant delete existing fields Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #10089|0 |1 is obsolete| | --- Comment #17 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 12569 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=12569&action=edit Bug 8015 - Add MARC Modifications Templates The MARC Modification Templates system gives Koha users the power to make alterations to MARC records automatically while staging MARC records for import. This tool is useful for altering MARC records from various venders work with your MARC framework. The system essentially allows one to create a basic script using actions to Copy, Move, Add, Update and Delete fields. Each action can also have an optional condition to check the value or existance of another field. The Copy & Move actions also support Regular Expressions, which can be used to automatically modify field values during the copy/move. An example would be to strip out the '$' character in field 020$c. Furthermore, the value for an update can include variables that change each time the template is used. Currently, the system supports two variables, __BRANCHCODE__ which is replaced with the branchcode of the library currently using the template, and __CURRENTDATE__ which is replaced with the current date in ISO format ( YYYY-MM-DD ). At its simplist, it can perform functions such as: Copy field 092$a to 952$c At its most complex it can run actions like: Copy field 020$c to 020$c using RegEx s/\$// if 020$c equals RegEx m/^\$/ -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #12288|0 |1 is obsolete| | --- Comment #19 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 12571 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=12571&action=edit [SIGNED-OFF] Bug 8015: Followup: On move, we cant delete existing fields Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #21 from Kyle M Hall <kyle@bywatersolutions.com> --- (In reply to comment #16)

i want test the new patch in a sandbox but i have this message

The patches needed a rebase. You should be able to test it now. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #23 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 12839 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=12839&action=edit Bug 8015: Followup Add MARC Modifications Templates -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #12570|0 |1 is obsolete| | --- Comment #25 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 12857 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=12857&action=edit [SIGNED-OFF] Bug 8015: Followup: FIX cp and mv if subfields don't exist. Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #12572|0 |1 is obsolete| | --- Comment #27 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 12859 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=12859&action=edit Bug 8015 - Add MARC Modifications Templates - Followup -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Laurence Lefaucheur <laurence.lefaucheur@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |laurence.lefaucheur@biblibr | |e.com --- Comment #29 from Laurence Lefaucheur <laurence.lefaucheur@biblibre.com> --- When I try to import unimarc files with templates, I have some problems with unimarc format checking. I join a file bib-237.utf8 as an exemple : I am able to import it without template, but when a template is selected, Stage for import in /cgi-bin/koha/tools/stage-marc-import.pl page produces no job with template : - import of a iso 5426 file : not OK - export of a koha biblio in marc unicode/utf8 format and following import : not OK - export of a koha biblio in marc unicode/utf8,standard format and following import : OK In the log we found things as : errors in C4::Charset::SetUTF8Flag Field 686 must have indicators (use ' ' for empty indicators) at /home/koha/src/C4/Charset.pm line 142 As these files can be well imported without template, I think they should be as well be with a template -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #31 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 13138 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=13138&action=edit Bug 8015: Catch error in the SetUTF8Flag routine The eval avoids the interface to run endlessly if an error occurred. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #12856|0 |1 is obsolete| | Attachment #12857|0 |1 is obsolete| | Attachment #12858|0 |1 is obsolete| | Attachment #12859|0 |1 is obsolete| | Attachment #12860|0 |1 is obsolete| | Attachment #13495|0 |1 is obsolete| | --- Comment #33 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 14030 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=14030&action=edit Bug 8015 - Add MARC Modifications Templates The MARC Modification Templates system gives Koha users the power to make alterations to MARC records automatically while staging MARC records for import. This tool is useful for altering MARC records from various venders work with your MARC framework. The system essentially allows one to create a basic script using actions to Copy, Move, Add, Update and Delete fields. Each action can also have an optional condition to check the value or existance of another field. The Copy & Move actions also support Regular Expressions, which can be used to automatically modify field values during the copy/move. An example would be to strip out the '$' character in field 020$c. Furthermore, the value for an update can include variables that change each time the template is used. Currently, the system supports two variables, __BRANCHCODE__ which is replaced with the branchcode of the library currently using the template, and __CURRENTDATE__ which is replaced with the current date in ISO format ( YYYY-MM-DD ). At its simplist, it can perform functions such as: Copy field 092$a to 952$c At its most complex it can run actions like: Copy field 020$c to 020$c using RegEx s/\$// if 020$c equals RegEx m/^\$/ -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #35 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 14032 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=14032&action=edit Bug 8015: Followup: On move, we cant delete existing fields Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #37 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 14034 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=14034&action=edit Bug 8015: Followup Add MARC Modifications Templates Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #39 from Laurence Lefaucheur <laurence.lefaucheur@biblibre.com> --- Tested on sandbox8 marc template for import created import of an UTF8 file (the one joined in this bz) : OK, encoding problem solved marc modification according to template : OK -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #42 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to comment #41)

I've got a problem with the file to test import (bib-237.utf8). I'v added it to a basket from a staged file, but the price is not filled. I think the price is include in the file (947$c ?). There is no problem with quantity (10 books).

Does it work on master?

After mys first test, I've noticed a software error. - Manage staged MARC records - create a new basket - add an order from a staged file - select file - clic on "add order" (to access ) = software error

Can't call method "raw" on an undefined value at /home/koha/src/C4/Search.pm line 272.

If I dont clic on "add order", there is no software error.

I cannot reproduce, could you give me the url of the basket please? -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #44 from Jonathan Druart <jonathan.druart@biblibre.com> --- The manipulations for Bug 3087 were not made on sandboxes. Now the problem is solved. Pierre, could you check that your problem does not exist anymore please? -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #45 from Pierre Angot <tredok.pierre@gmail.com> --- Do I create a new sandbox ? Patch 8015 is already applied on Pelardon ^^ -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #47 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to comment #46)

There is no problem with software error.

But, when I've added an order with the file to test, vendor price and quantity are empty.

I think it is the current behavior in master. So there is no regression here. (The import improvement is in the patch proposed for Bug 7180). -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 claire.hernandez@biblibre.com <claire.hernandez@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Priority|P5 - low |P4 Blocks| |5725 -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #14030|0 |1 is obsolete| | Attachment #14031|0 |1 is obsolete| | Attachment #14032|0 |1 is obsolete| | Attachment #14033|0 |1 is obsolete| | Attachment #14034|0 |1 is obsolete| | Attachment #14040|0 |1 is obsolete| | --- Comment #50 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 16330 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16330&action=edit Bug 8015 - Add MARC Modifications Templates The MARC Modification Templates system gives Koha users the power to make alterations to MARC records automatically while staging MARC records for import. This tool is useful for altering MARC records from various venders work with your MARC framework. The system essentially allows one to create a basic script using actions to Copy, Move, Add, Update and Delete fields. Each action can also have an optional condition to check the value or existance of another field. The Copy & Move actions also support Regular Expressions, which can be used to automatically modify field values during the copy/move. An example would be to strip out the '$' character in field 020$c. Furthermore, the value for an update can include variables that change each time the template is used. Currently, the system supports two variables, __BRANCHCODE__ which is replaced with the branchcode of the library currently using the template, and __CURRENTDATE__ which is replaced with the current date in ISO format ( YYYY-MM-DD ). At its simplist, it can perform functions such as: Copy field 092$a to 952$c At its most complex it can run actions like: Copy field 020$c to 020$c using RegEx s/\$// if 020$c equals RegEx m/^\$/ -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #52 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 16332 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16332&action=edit Bug 8015: Followup: On move, we cant delete existing fields Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #54 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 16334 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16334&action=edit Bug 8015: Followup Add MARC Modifications Templates Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #56 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 16363 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16363&action=edit Bug 8015: Fix template capitalization amd other template issues -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #58 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 16377 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16377&action=edit Bug 8015: Fix error where field object is returned instead of field value for fields without subfields -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #60 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 16379 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16379&action=edit Bug 8015: Escape escape characters for strings -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16363|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #63 from Kyle M Hall <kyle@bywatersolutions.com> --- Point of documentation: It is possible to operate on a field without a subfield if and only if that field does not have subfields according to the MARC specification. For example, we can copy 005 to 052$b, because 005 is just data without subfields. We can also copy 952$g to 300$c, as they are data points as well. The scenario that does not work is trying to move a field with subfields to a new field. That is, for example, we cannot move field 300 to field 500. What can can do is move 300$a to 500$a, 300$b to 500%b, and so on. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16413|0 |1 is obsolete| | --- Comment #65 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 16415 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16415&action=edit Bug 8015: Fix bad parameter list for direct external call to update_field -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #67 from Kyle M Hall <kyle@bywatersolutions.com> ---

- Move 1st field 200$a to 300$a : 200$a is note moved but is deleted

Fixed!

- Update field 995$f with value bla if 995$f does not match : there is nothing.

Fixed! -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Paul Poulain <paul.poulain@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |paul.poulain@biblibre.com --- Comment #68 from Paul Poulain <paul.poulain@biblibre.com> --- Patch tested with a sandbox, by Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Koha Team AMU <koha.aixmarseille@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koha.aixmarseille@gmail.com --- Comment #70 from Koha Team AMU <koha.aixmarseille@gmail.com> --- All is right : delete, move, add, copy are ok, even with a regEx. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Fridolyn SOMERS <fridolyn.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16455|Bug 8015: Fix problem with |[SIGNED-OFF] Bug 8015: Fix description|moving existing subfield |problem with moving |value to nonexistent |existing subfield value to |field/subfield |nonexistent field/subfield -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Fridolyn SOMERS <fridolyn.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16331|0 |1 is obsolete| | --- Comment #72 from Fridolyn SOMERS <fridolyn.somers@biblibre.com> --- Created attachment 16470 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16470&action=edit [SIGNED-OFF] Bug 8015: Followup: FIX cp and mv if subfields don't exist. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Fridolyn SOMERS <fridolyn.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16333|0 |1 is obsolete| | --- Comment #74 from Fridolyn SOMERS <fridolyn.somers@biblibre.com> --- Created attachment 16472 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16472&action=edit [SIGNED-OFF] Bug 8015 - Add MARC Modifications Templates - Followup -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Fridolyn SOMERS <fridolyn.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16335|0 |1 is obsolete| | --- Comment #76 from Fridolyn SOMERS <fridolyn.somers@biblibre.com> --- Created attachment 16475 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16475&action=edit [SIGNED-OFF] Bug 8015: Catch error in the SetUTF8Flag routine -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Fridolyn SOMERS <fridolyn.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16377|0 |1 is obsolete| | --- Comment #78 from Fridolyn SOMERS <fridolyn.somers@biblibre.com> --- Created attachment 16477 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16477&action=edit Bug 8015: Fix error where field object is returned instead of field value for fields without subfields -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Fridolyn SOMERS <fridolyn.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16379|0 |1 is obsolete| | --- Comment #80 from Fridolyn SOMERS <fridolyn.somers@biblibre.com> --- Created attachment 16479 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=16479&action=edit [SIGNED-OFF] Bug 8015: Escape escape characters for strings -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |katrin.fischer@bsz-bw.de Version|master |3.14 -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #83 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 17037 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=17037&action=edit Bug 8015: FIX QA issues This patch fixes some stuffs failing qa tests: POD, indentation (tabs), perlcritic -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |ASSIGNED --- Comment #85 from Jonathan Druart <jonathan.druart@biblibre.com> --- Hi Kyle, I add some unit tests for these new routines/methods (QA requirements). One test does not pass and I think it should. I delete the internal routine (in SimpleMARC), I hope it is good for you. I am under the impression that the build of the @values array (@values = ($values[0]) x scalar( @fields ) if @values == 1;) is enough. I add a followup for QA issues (and add your name in order to keep the authorship). Could you have a look at these 2 UT files please? Feel free to add others tests if something is missing. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #87 from Jonathan Druart <jonathan.druart@biblibre.com> --- Strange, I didn't get any problem to apply all of them against master. I just push a branch on our kohac repository: http://git.biblibre.com/?p=kohac;a=shortlog;h=refs/heads/BZ8015 -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle.m.hall@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Needs Signoff --- Comment #89 from Kyle M Hall <kyle.m.hall@gmail.com> --- (In reply to comment #88)

Kyle, did you have a look at my unit tests?

These are fantastic! Everything passes except the last test for SimpleMARC.t Also, test 14 has no description. Thanks for pointing me to your git branch. ok 1 - use MARC::Field; ok 2 - use MARC::Record; ok 3 - use Koha::SimpleMARC; ok 4 - 650$a exists ok 5 - 650$b does not exist ok 6 - 650$a exists, field_exists returns the first one ok 7 - first 650$a ok 8 - second 650$a ok 9 - first 650$a bis ok 10 - second 650$a bis ok 11 - There is no 3 650$a ok 12 - After copy 245$a still exists ok 13 - 246$a is a new field ok 14 ok 15 - Copy multivalued field ok 16 - Copy first field 650$a ok 17 - Copy second field 650$a ok 18 - Copy field using regex ok 19 - Copy fields using regex on existing fields ok 20 - Copy first field using regex on existing fields without erase existing values ok 21 - Copy fields using regex on existing fields without erase existing values ok 22 - update existing subfield 952$p ok 23 - create subfield 952$p ok 24 - create subfield 952$k on existing 952 field ok 25 - update all subfields 952$p with the same value ok 26 - update all subfields 952$p with the different values ok 27 - copy 952$d into others 952 field ok 28 - The first 952$c has moved not ok 29 - Now the second 952$c exists -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #90 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to comment #89)

These are fantastic! Everything passes except the last test for SimpleMARC.t

not ok 29 - Now the second 952$c exists

Yep, it was I said in comment 85 ("One test does not pass and I think it should."). So is there a bug in move_field? Or my test is bad (but I don't think so). -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #93 from Jonathan Druart <jonathan.druart@biblibre.com> --- I amended my ut patch in order to add some tests for the move_field routine. Now all tests pass. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle.m.hall@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #96 from Jonathan Druart <jonathan.druart@biblibre.com> --- I updated the git branch just now. All patches (submitted on this report) are rebased against master (just 2 conflicts with the first patch on updatedatabase.pl and kohastructure.sql). -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|3.14 |master Severity|enhancement |new feature -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA --- Comment #99 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Ok, what I did was create 2 branches, one with all patches, and another with a squashed version to allow for easier code review. The QA script fails in several places. Bit worried about some of those, so I will wait for a follow up: QA SCRIPT testing 1 commit(s) (applied to 6fcb671 'Bug 10678 - Update addbooks help page') OK C4/Charset.pm OK pod OK forbidden patterns OK valid OK critic FAIL C4/ImportBatch.pm OK pod OK forbidden patterns FAIL valid Useless use of private variable in void context OK critic OK C4/Koha.pm OK pod OK forbidden patterns OK valid OK critic FAIL C4/MarcModificationTemplates.pm OK pod FAIL forbidden patterns forbidden pattern: Koha is now under the GPLv3 license (line 9) FAIL valid Useless use of private variable in void context OK critic FAIL Koha/SimpleMARC.pm OK pod OK forbidden patterns OK valid FAIL critic Expression form of "eval" at line 89, column 7. See page 161 of PBP. Expression form of "eval" at line 242, column 12. See page 161 of PBP. OK installer/data/mysql/updatedatabase.pl OK pod OK forbidden patterns OK valid OK critic OK t/SimpleMARC.t OK pod OK forbidden patterns OK valid OK critic OK t/db_dependent/MarcModificationTemplates.t OK pod OK forbidden patterns OK valid OK critic FAIL tools/marc_modification_templates.pl OK pod FAIL forbidden patterns forbidden pattern: Koha is now under the GPLv3 license (line 8) FAIL valid Useless use of private variable in void context OK critic FAIL tools/stage-marc-import.pl OK pod OK forbidden patterns FAIL valid Useless use of private variable in void context OK critic OK koha-tmpl/intranet-tmpl/prog/en/includes/tools-menu.inc OK forbidden patterns OK tt_valid OK valid_template OK koha-tmpl/intranet-tmpl/prog/en/modules/admin/admin-home.tt OK forbidden patterns OK tt_valid OK valid_template FAIL koha-tmpl/intranet-tmpl/prog/en/modules/tools/marc_modification_templates.tt FAIL forbidden patterns forbidden pattern: intranet-tmpl should certainly replaced with [% interface %] (line 301) forbidden pattern: intranet-tmpl should certainly replaced with [% interface %] (line 305) forbidden pattern: intranet-tmpl should certainly replaced with [% interface %] (line 293) forbidden pattern: intranet-tmpl should certainly replaced with [% interface %] (line 297) OK tt_valid OK valid_template OK koha-tmpl/intranet-tmpl/prog/en/modules/tools/stage-marc-import.tt OK forbidden patterns OK tt_valid OK valid_template OK koha-tmpl/intranet-tmpl/prog/en/modules/tools/tools-home.tt OK forbidden patterns OK tt_valid OK valid_template -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #101 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I'd better leave it to you, Fridolyn or Kyle to fix - already lots of hands on this. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #103 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Hi Kyle, I used the patches on the bug and they applied fine apart from a database related conflict in the first patch that was easy to solve. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #105 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Sorry, I can't before late tonight - it's on my laptop. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #107 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21025 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21025&action=edit Bug 8015: Followup: FIX cp and mv if subfields don't exist. Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #109 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21027 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21027&action=edit Bug 8015 - Add MARC Modifications Templates - Followup Signed-off-by: Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #111 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21029 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21029&action=edit Bug 8015: Catch error in the SetUTF8Flag routine The eval avoids the interface to run endlessly if an error occurred. Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16477|0 |1 is obsolete| | --- Comment #113 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21031 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21031&action=edit Bug 8015: Fix error where field object is returned instead of field value for fields without subfields Signed-off-by: Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #115 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21033 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21033&action=edit Bug 8015: Escape escape characters for strings Signed-off-by: Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #17036|0 |1 is obsolete| | --- Comment #117 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21035 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21035&action=edit Bug 8015: Fix problem with moving existing subfield value to nonexistent field/subfield Signed-off-by: Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #119 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21037 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21037&action=edit Bug 8015: Add unit tests for SimpleMARC and MarcModificationTemplates routines Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #121 from Jonathan Druart <jonathan.druart@biblibre.com> --- Rebased patches. I have no idea how to remove 2 others warnings. It's a Perl Critic recommendation not to eval a literal. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #123 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to Katrin Fischer from comment #122)

Hm, can someone help us out here fixing the perl critic violations?

To me, there are only 2 ways to remove theses warnings: 1/ adds a rule to the perlcriticrc file. 2/ write an unreadable code. (i.e. eval "\$value =~ s$regex"; could be written splitting the $regex var on the first character (/ or |, etc.). In both case it is not a good thing to do. Perl::Critic provides recommendations, sometimes I think we are not obliged to follow them. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16468|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16471|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16473|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16476|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #16479|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #18441|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #126 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21371 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21371&action=edit Bug 8015: Followup QA issues eval { $eval }; != eval $eval; my $r = q|/foo/fee/|; my @values = ("foo", "bar", "foobar"); for my $v ( @values ) { eval {"\$v =~ s$r"}; say $v; eval "\$v =~ s$r"; say $v; } will display: foo fee bar bar foobar feebar -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #128 from Jared Camins-Esakov <jcamins@cpbibliography.com> --- (In reply to Paul Poulain from comment #127)

Katrin, please reconsider your position = this improvement is very important for libraries, much more than a technical perlcritic violation. My opinion here is that we must decide that the user interest is more important than this single violation. (the law must serve us, we must not serve the law)

This is not a minor violation. As far as I can tell, there is unsanitized user input being run directly. Consider the following code: eval { $eval }; != eval $eval; my $r = q|/foo/fee/; print "hi";|; my @values = ("foo", "bar", "foobar"); for my $v ( @values ) { eval {"\$v =~ s$r"}; say $v; eval "\$v =~ s$r"; say $v; } Result: "hi" is printed by each eval. This example is fairly benign, but what if the regex were as follows: my $r = q|/whatever/something/; require C4::Context; $dbh = C4::Context->dbh; $dbh->do("DELETE FROM issues;")|; I was unable to test this example, since I couldn't get MARC modification templates to work during a cursory test, but I'm sure you see my point. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #130 from Jared Camins-Esakov <jcamins@cpbibliography.com> --- (In reply to Paul Poulain from comment #129)

I hadn't checked what the eval was related to, I assumed it was safe. I agree with your point : failed QA, this could probably be exploited.

Jared, would you be pleased if the parameters where sanitized, even if the eval is still here ?

I see no way to sanitize the input while using the eval. The entire point of the eval is to allow arbitrary code to be run through the regex. Maybe it would be better to have two boxes, one for the match and one for the replacement? -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #132 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to Galen Charlton from comment #131)

I think it would be better to drop the notion of using a string eval at all and just rewrite ModifyRecordWithTemplate to become a dispatcher that calls the modification subroutines directly. I say "just rewrite" because, for once, I think the work can be confined to a single routine.

Also, the patchset currently includes no tests that run ModifyRecordWithTemplate directly -- rather a lack, IMO. Resolving that can go hand-in-hand with removing the eval.

I will try to do something. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Jonathan Druart <jonathan.druart@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21371|0 |1 is obsolete| | --- Comment #133 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21484 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21484&action=edit Bug 8015: Get rid of the eval in ModifyRecordWithTemplate This patch removes the use of eval in the C4::MarcModificationTemplates::ModifyRecordWithTemplate routine. Now this routine call the wanted modification routine with the list of parameters. This call is done only if the condition is respected. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #135 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21486 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21486&action=edit Bug 8015: Get rid of the eval for substitution Before this patch, the regex substitution was contain into only one variable (e.g. my $regex = "/foo/bar/i"). Now each member of the regex is stored into a field in the marc_modification_template_actions sql table. In order to avoid a complex code, only modifiers i and g are take into account. Note: If you already add the mmta table, you have to drop it. This patch also adds a foreign key from mmta to mmt tables. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #137 from Jonathan Druart <jonathan.druart@biblibre.com> --- These 4 last patches remove the eval. All 19 patches pass qa tools. Test plan: - Retest *all* the feature with complex actions. - prove t/SimpleMarc.t - prove t/db_dependent/MarcModificationTemplates.t Have fun and good luck! -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Galen Charlton <gmcharlt@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_3_14_candidate -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #140 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to Bernardo Gonzalez Kriegel from comment #139)

Is there anything to do before running tests? No! I will fix that asap

-- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #142 from Jonathan Druart <jonathan.druart@biblibre.com> --- Created attachment 21809 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21809&action=edit Bug 8015: FIX ui issue -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #144 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Testing this! Tests now run without problems :) For me this works, I've tried add, copy, delete, move, regex find and replace (text and also changing branch in 952ab using __BRANCHNAME__) What's missing: 1) Check that template name is non empty. If you press "Create template" can create as many "no name" templates as you wish. Avoiding duplicates could be cool. 2) Help File. This is important for prospective users and DocManager job. At last a minimal one could help testing -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #146 from Jonathan Druart <jonathan.druart@biblibre.com> --- (In reply to Bernardo Gonzalez Kriegel from comment #144)

What's missing:

1) Check that template name is non empty. If you press "Create template" can create as many "no name" templates as you wish. Avoiding duplicates could be cool.

Done in the last patch.

2) Help File. This is important for prospective users and DocManager job. At last a minimal one could help testing

I don't have the time to do that. Maybe Kyle? -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21024|0 |1 is obsolete| | --- Comment #148 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 21901 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21901&action=edit Bug 8015 - Add MARC Modifications Templates The MARC Modification Templates system gives Koha users the power to make alterations to MARC records automatically while staging MARC records for import. This tool is useful for altering MARC records from various venders work with your MARC framework. The system essentially allows one to create a basic script using actions to Copy, Move, Add, Update and Delete fields. Each action can also have an optional condition to check the value or existance of another field. The Copy & Move actions also support Regular Expressions, which can be used to automatically modify field values during the copy/move. An example would be to strip out the '$' character in field 020$c. Furthermore, the value for an update can include variables that change each time the template is used. Currently, the system supports two variables, __BRANCHCODE__ which is replaced with the branchcode of the library currently using the template, and __CURRENTDATE__ which is replaced with the current date in ISO format ( YYYY-MM-DD ). At its simplist, it can perform functions such as: Copy field 092$a to 952$c At its most complex it can run actions like: Copy field 020$c to 020$c using RegEx s/\$// if 020$c equals RegEx m/^\$/ Signed-off-by: Leila <koha.aixmarseille@gmail.com> Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Works as described. No koha-qa errors One comment to rule them all (please squash this patchset!) I think I tested all possible cases: add, copy, delete, move, regex find and replace (text and also changing branch in 952ab using __BRANCHNAME__) Unit test run fine. No help file yet but not a functional requirement. Think is a good adition. -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21026|0 |1 is obsolete| | --- Comment #150 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 21903 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21903&action=edit Bug 8015: Followup: On move, we cant delete existing fields Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21028|0 |1 is obsolete| | --- Comment #152 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 21905 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21905&action=edit Bug 8015: Followup Add MARC Modifications Templates Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21030|0 |1 is obsolete| | --- Comment #154 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 21907 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21907&action=edit Bug 8015: Fix template capitalization amd other template issues Signed-off-by: Leila <koha.aixmarseille@gmail.com> Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21032|0 |1 is obsolete| | --- Comment #156 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 21909 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21909&action=edit Bug 8015: Fix bad ordering on function parameters Signed-off-by: Leila <koha.aixmarseille@gmail.com> Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21034|0 |1 is obsolete| | --- Comment #158 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 21911 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21911&action=edit Bug 8015: Fix bad parameter list for direct external call to update_field Signed-off-by: Leila <koha.aixmarseille@gmail.com> Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21036|0 |1 is obsolete| | --- Comment #160 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 21913 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21913&action=edit Bug 8015: FIX QA issues This patch fixes some stuffs failing qa tests: POD, indentation (tabs), perlcritic Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21038|0 |1 is obsolete| | --- Comment #162 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 21915 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21915&action=edit Bug 8015: Fix complains from qa tools Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21485|0 |1 is obsolete| | --- Comment #164 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 21917 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21917&action=edit Bug 8015: Get rid of eval for evaluating =~ m// Koha::SimpleMarc::field_equals uses eval in order to check if a string matches a pattern. Now this eval is removed and the "regex" variable does not contain the regex separated character (/ or |). Regression: Before this patch, the user was able to user a modifier. Now it is not possible. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #21809|0 |1 is obsolete| | --- Comment #166 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Created attachment 21919 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=21919&action=edit Bug 8015: FIX ui issue Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #168 from Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> --- Signed! 1) Could you consider squashing this into one... or two patches. This is insane :) 2) Needed follow-up: Help 3) Needed follow-up: Update copyright headers as current usage -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #170 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 22186 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22186&action=edit Bug 8015: Followup: FIX cp and mv if subfields don't exist. Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> Bug 8015: Followup: On move, we cant delete existing fields Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #172 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 22188 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22188&action=edit Bug 8015: Followup Add MARC Modifications Templates Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> Bug 8015: Catch error in the SetUTF8Flag routine The eval avoids the interface to run endlessly if an error occurred. Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #174 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 22190 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22190&action=edit Bug 8015: Add unit tests for SimpleMARC and MarcModificationTemplates routines Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Bug 8015: Fix complains from qa tools Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Bug 8015: Get rid of the eval in ModifyRecordWithTemplate This patch removes the use of eval in the C4::MarcModificationTemplates::ModifyRecordWithTemplate routine. Now this routine call the wanted modification routine with the list of parameters. This call is done only if the condition is respected. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Bug 8015: Get rid of eval for evaluating =~ m// Koha::SimpleMarc::field_equals uses eval in order to check if a string matches a pattern. Now this eval is removed and the "regex" variable does not contain the regex separated character (/ or |). Regression: Before this patch, the user was able to user a modifier. Now it is not possible. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Bug 8015: Get rid of the eval for substitution Before this patch, the regex substitution was contain into only one variable (e.g. my $regex = "/foo/bar/i"). Now each member of the regex is stored into a field in the marc_modification_template_actions sql table. In order to avoid a complex code, only modifiers i and g are take into account. Note: If you already add the mmta table, you have to drop it. This patch also adds a foreign key from mmta to mmt tables. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Bug 8015: FIX ui issue Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Bug 8015: The template name is a required field Test plan: Try to add a template with an empty string as name. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #176 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 22284 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22284&action=edit Bug 8015: Add user permissions to all languages -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #178 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 22290 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22290&action=edit Bug 8015: QA Followup * Add bug number to database update * Fix capitalization errors * Fix JavaScript string translation errors -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 --- Comment #180 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Test report 1)OK Staging, importing, reverting and cleaning using - stage_file.pl and commit_file.pl - Stage and Staged MARC record management tools 2) (normal) Saving an empty action - fil 3) (minor) Feedback about mandatory fields not quite standard 4) (major) Rule: Copy 650 to 689 - no subfields entered [Wed Oct 23 00:07:58 2013] [error...] [client 127.0.0.1] [Wed Oct 23 00:07:58 2013] stage-marc-import.pl: data() is only for control fields (generally, just tags below 010) , use subfield() at /home/katrin/koha/Koha/SimpleMARC.pm line 188, referer: http://staff.master/cgi-bin/koha/tools/stage-marc-import.pl No feedback in the interface, we should make sure people know that whole fields can't be copied. Maybe subfield should be required to, if it's not a controlfield (Note: controlfields are different for MARC21 and UNIMARC) 5) (?) Rule: Copy 650 a to 689 a Works, but 689 does not exist in my framework - should this be possible? 6) OK Rule: Add/Update 952 $a CPL Add/Update 952 $b CPL Add/Update 952 $y BK Adds items, works nicely 7) OK Rule: Copy 245 $a to 500 $a Regex: s/e/foo Intfoormfoodiatfoo Pfoorl - Works nicely. 8) (normal) Also: Lots of messages in the logs [Wed Oct 23 00:09:46 2013] [error] [client 127.0.0.1] [Wed Oct 23 00:09:46 2013] marc_modification_templates.pl: Use of uninitialized value in string eq at /home/katrin/koha/tools/marc_modification_templates.pl line 70., referer: http://staff.master/cgi-bin/koha/tools/marc_modification_templates.pl [Wed Oct 23 00:10:06 2013] [error] [client 127.0.0.1] [Wed Oct 23 00:10:06 2013] stage-marc-import.pl: Use of uninitialized value $template_id in string eq at /home/katrin/koha/C4/MarcModificationTemplates.pm line 84., referer: http://staff.master/cgi-bin/koha/tools/stage-marc-import.pl X) Possible enhancements: - Allow applying of modification templates on staged and maybe also imported records - Warn people from deleting mandatory fields and subfields - The Add/Update option should have a regex - Ex. set serial flag on LDR Other problems found: - Staging MARC records uploads without selecting a file first Note to myself: File a bug about translation issues coursed logic in templates (building sentences) Kyle, can you please take a look? *hides* -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #22186|0 |1 is obsolete| | --- Comment #182 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 22393 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22393&action=edit Bug 8015: Followup: FIX cp and mv if subfields don't exist. Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> Bug 8015: Followup: On move, we cant delete existing fields Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #22188|0 |1 is obsolete| | --- Comment #184 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 22395 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22395&action=edit Bug 8015: Followup Add MARC Modifications Templates Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> Bug 8015: Catch error in the SetUTF8Flag routine The eval avoids the interface to run endlessly if an error occurred. Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Signed-off-by: Leila <koha.aixmarseille@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #22190|0 |1 is obsolete| | --- Comment #186 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 22397 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22397&action=edit Bug 8015: Add unit tests for SimpleMARC and MarcModificationTemplates routines Signed-off-by: Kyle M Hall <kyle@bywatersolutions.com> Bug 8015: Fix complains from qa tools Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Bug 8015: Get rid of the eval in ModifyRecordWithTemplate This patch removes the use of eval in the C4::MarcModificationTemplates::ModifyRecordWithTemplate routine. Now this routine call the wanted modification routine with the list of parameters. This call is done only if the condition is respected. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Bug 8015: Get rid of eval for evaluating =~ m// Koha::SimpleMarc::field_equals uses eval in order to check if a string matches a pattern. Now this eval is removed and the "regex" variable does not contain the regex separated character (/ or |). Regression: Before this patch, the user was able to user a modifier. Now it is not possible. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Bug 8015: Get rid of the eval for substitution Before this patch, the regex substitution was contain into only one variable (e.g. my $regex = "/foo/bar/i"). Now each member of the regex is stored into a field in the marc_modification_template_actions sql table. In order to avoid a complex code, only modifiers i and g are take into account. Note: If you already add the mmta table, you have to drop it. This patch also adds a foreign key from mmta to mmt tables. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Bug 8015: FIX ui issue Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> Bug 8015: The template name is a required field Test plan: Try to add a template with an empty string as name. Signed-off-by: Bernardo Gonzalez Kriegel <bgkriegel@gmail.com> -- You are receiving this mail because: You are watching all bug changes.

http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8015 Kyle M Hall <kyle@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #22285|0 |1 is obsolete| | --- Comment #188 from Kyle M Hall <kyle@bywatersolutions.com> --- Created attachment 22399 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=22399&action=edit Bug 8015: Remove references to C4::Koha::Log -- You are receiving this mail because: You are watching all bug changes.