August 2017 - Koha-bugs - lists.koha-community.org

[Bug 10306] Koha to MARC mappings (Part 1): Allow multiple mappings per kohafield (for say 260/RDA 264)
by bugzilla-daemon＠bugs.koha-community.org 17 Aug '17

17 Aug '17

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10306 Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #66135|0 |1 is obsolete| | --- Comment #43 from Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> --- Created attachment 66142 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=66142&action=ed… Bug 10306: Adjusted online help text for koha2marclinks Test plan: Click on Help. Read the text. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 10306] Koha to MARC mappings (Part 1): Allow multiple mappings per kohafield (for say 260/RDA 264)
by bugzilla-daemon＠bugs.koha-community.org 17 Aug '17

17 Aug '17

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10306 Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #66134|0 |1 is obsolete| | --- Comment #42 from Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> --- Created attachment 66141 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=66141&action=ed… Bug 10306: Support for multiple mappings in koha2marclinks This actually refactors koha2marclinks.pl in order to support multiple mappings per kohafield. Instead of three separate mapping pages for biblio, biblioitems and items, the script now lists them together. This gives a complete overview of all mappings rightaway. Changes are applied immediately across all frameworks. Note: This report handles the Default mappings just like it did before. In this script Koha already considered them as authoritative, although other parts of Koha did not. Follow-up report 19096 makes Default mappings authoritative throughout all Koha. On each line two buttons are provided, Add and Remove, in order to add or remove an individual mapping. We do no longer provide a separate form with the names of MARC tags. Since this form is targeted for administrators, it should be enough to ask for a field tag and subfield code. Note: The mappings for biblionumber, biblioitemnumber and itemnumber are so vital that this form marks them as readonly. It is not recommended to change them. Test plan: [1] Add a mapping. Verify via Frameworks or mysql command line that the kohafield is saved to the other frameworks too. [2] Remove the mapping again. Check Frameworks or mysql cl again. [3] Test adding a second mapping. Map copyrightdate to 260c and 264c. And map biblioitems.place to 260a and 264a. [4] Edit biblio record 1: Put 1980 in 260c. Do not include 264c. Edit biblio record 2: Put 1990 in 264c. Do not include 260c. Edit biblio record 3: Put 2000 in both 260c and 264c. Put CityA in 260a and in 264a. Edit biblio record 4: Put 2010 in 260c, and 2015 in 264c (which you should refuse normally). Put CityA in 260a, and CityB in 264a. [5] Create a report that shows biblioitems.place and biblio.copyrightdate for those biblio records. Record 4 should have 2010 in copyrightdate (since TransformMarcToKoha picks the first year for copyrightdate). Record 3 should have place CityA; record 4 should have CityA | CityB. Note: The CityA | CityB example illustrates that we should add some additional handling in TransformMarcToKoha for multiple 264s. [6] Add these four biblio records to a new list. Sort by Year. With OPACXSLTListsDisplay==default, check if the order = 1,2,3,4. (The order is based on biblio.copyrightdate.) Note that (RDA) record 2 would be on top without this patch set, since copyrightdate would have been null. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 10306] Koha to MARC mappings (Part 1): Allow multiple mappings per kohafield (for say 260/RDA 264)
by bugzilla-daemon＠bugs.koha-community.org 17 Aug '17

17 Aug '17

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10306 Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #65951|0 |1 is obsolete| | --- Comment #41 from Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> --- Created attachment 66140 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=66140&action=ed… Bug 10306: Restructure TransformMarcToKoha and update some POD lines This patch makes the following changes: [1] Added POD for CountItemsIssued, GetBiblioItemData [2] Moved TransformMarcToKohaOneField closer to TransformMarcToKoha (before sub CountItemsIssued) [3] Restructured TransformMarcToKoha by extracting individual kohafields via TransformMarcToKohaOneField. The latter does no longer call the former. This small optimization avoids traversing the whole MARC record again and again. [4] Moved adjusting copyrightdate/publicationyear to separate helper routine _adjust_pubyear [5] Removed obsolete sub _get_inverted_marc_field_map. Test plan: Run t/db_dependent/Biblio/TransformMarcToKoha.t -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 10306] Koha to MARC mappings (Part 1): Allow multiple mappings per kohafield (for say 260/RDA 264)
by bugzilla-daemon＠bugs.koha-community.org 17 Aug '17

17 Aug '17

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10306 Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #65950|0 |1 is obsolete| | --- Comment #40 from Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> --- Created attachment 66139 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=66139&action=ed… Bug 10306: Add tests for module changes GetMarcSubfieldStructure: In Biblio.t we are adding a subtest that checks the structure returned by this routine. Is it a hashref pointing to arrayrefs of hashrefs? ;) In Search.t this routine was mocked. The change in the returned structure is now applied to this mock too (moving the marc tag hashes into arrayrefs). GetMarcFromKohaField: In Biblio.t we add a subtest for it. We are checking if it returns multiple mappings (per kohafield) and calling it in scalar context too. GetMarcSubfieldStructureFromKohaField: The existing subtest in Biblio.t is expanded to also test the call in list context. TransformKohaToMarc: This sub has its own test script. We are adding a subtest in TransformKohaToMarc.t for a test with multiple mappings, and for mapping to a control field in another framework. This also tests the additional framework parameter. Additionally, we add a test for the new no_split option used for items. TransformMarcToKoha: This implicitly tests its helper sub _get_inverted. This patch adds a new test script for this routine. TransformMarcToKohaOneField: A few tests are added to the previous new test script. Test plan: [1] Run t/db_dependent/Biblio.t [2] Run t/db_dependent/Biblio/TransformKohaToMarc.t [3] Run t/db_dependent/Biblio/TransformMarcToKoha.t [4] Run t/db_dependent/Search.t -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 10306] Koha to MARC mappings (Part 1): Allow multiple mappings per kohafield (for say 260/RDA 264)
by bugzilla-daemon＠bugs.koha-community.org 17 Aug '17

17 Aug '17

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10306 Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #65949|0 |1 is obsolete| | --- Comment #39 from Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> --- Created attachment 66138 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=66138&action=ed… Bug 10306: Allow controlfields in TransformKohaToMarc Since the interface allows you to connect a kohafield to a MARC controlfield, this routine should be able to handle that. Unfortunately it did not. Test plan: Change will be tested in Biblio/TransformKohaToMarc.t in the next patch. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 10306] Koha to MARC mappings (Part 1): Allow multiple mappings per kohafield (for say 260/RDA 264)
by bugzilla-daemon＠bugs.koha-community.org 17 Aug '17

17 Aug '17

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10306 Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #65948|0 |1 is obsolete| | --- Comment #38 from Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> --- Created attachment 66137 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=66137&action=ed… Bug 10306: Core module changes for multiple mappings In order to allow multiple Koha to MARC mappings (for one kohafield), we need to adjust a few key routines in C4/Biblio.pm. This results in a few changes in dependent modules too. Note: Multiple mappings also include 'alternating' mappings. Such as the case of MARC21 260 and 264: only one of both fields will be used. Sub TransformMarcToKoha will handle that just fine; the opposite transformation is harder, since we do no longer know which field was the source. In that case TransformKohaToMarc will fill both fields. We only use that operation in Koha for items (in Acquisition and Cataloging). Sub GetMarcSubfieldStructure This sub used a selectall_hashref, which is fine as long as we have only one mapping for each kohafield. But as DBI states it: If a row has the same key as an earlier row then it replaces the earlier row. In other words, we lose the first mapping if we have two. This patch uses selectall_arrayref with Slice and rearranges the output so that the returned hash returns an arrayref of hashrefs for each kohafield. In order to improve consistency, we add an order clause to the SQL statement used too. Sub GetMarcFromKohaField This sub just returned one tag and subfield, but in case of multiple mappings we should return them all now. Note: Many calls still expect just one result and will work just fine: my ($tag, $sub) = GetMarcFromKohaField(...) A possible second mapping would be silently ignored. Often the sub is called for biblionumber or itemnumber. I would not recommend the use of multiple mappings for such fields btw. In case the sub is called in scalar context, it will return only the first tag (instead of the number of tags and subfields). Sub GetMarcSubfieldStructureFromKohaField This sub previously returned the hash for one kohafield. In scalar context it will behave like before: it returns the first hashref in the arrayref that comes from GetMarcSubfieldStructure. In list context, it returns an array of all hashrefs (incl. multiple mappings). The sub is not used in C4::Ris. Removed the use statement. Sub TransformKohaToMarc This sub got a second parameter: frameworkcode. Historically, Koha more or less assumes kohafields to be defined across all frameworks (see Koha to MARC mappings). Therefore it falls back to Default when it is not passed. When going thru all mappings in building a MARC record, it also supports multiple mappings. Note that Koha uses this routine in Acquisition and in Cataloging for items. Normally the MARC record is leading however and the Koha fields are derivatives for optimization and reporting. The added third parameter allows for passing a new option no_split => 1. We use this option in C4::Items::Item2Marc; if two item fields are mapped to one kohafield but would have different values (which would be very unusual), these values are glued together. When transforming to MARC again, we do not want to duplicate the item subfields, but we keep the glued value in both subfields. This operation only affects items, since we are not doing this reverse operation for biblio and biblioitem fields. Sub _get_inverted_marc_field_map This sub is a helper routine of TransformMarcToKoha, the opposite transformation. When saving a MARC record, all kohafields are extracted including multiple mappings. Suppose that you had both 260c and 264c in your record (which you won't), than both values get saved initially into copyrightdate like A | B. The additional code for copyrightdate will extract the first year from this string. A small fix in TransformMarcToKoha makes that it only saves a value in a kohafield if it is defined and not empty. (Same for concatenation.) Sub TransformMarcToKohaOneField This sub now just calls TransformMarcToKoha and extracts the requested field. Note that since we are caching the structure, this does not result in additional database access and is therefore performance-wise insignificant. We simplify code and maintenance. Instead of modifying the passed hashref, it simply returns a value. A call in C4::Breeding is adjusted accordingly. The routine getKohaField in Koha::MetadataRecord is redirected to TransformMarcToKohaOneField. NOTE: The fourth patch restructures/optimizes TransformMarcToKoha[OneField]. Sub get_koha_field_from_marc This sub can be removed. A call is replaced by TransformMarcToKohaOneField in C4::XISBN. Note: The commented lines for sub ModZebrafiles are removed (directly under TransformMarcToKohaOneField). Test plan: For unit tests and interface tests, please see follow-ups. Run qa tools in order to verify that the modules still compile well. Read the code changes and verify that they make sense. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 13618] Prevent XSS in the Staff Client and the OPAC
by bugzilla-daemon＠bugs.koha-community.org 17 Aug '17

17 Aug '17

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13618 DEVINIM <kohadevinim(a)devinim.com.tr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kohadevinim(a)devinim.com.tr -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 10306] Koha to MARC mappings (Part 1): Allow multiple mappings per kohafield (for say 260/RDA 264)
by bugzilla-daemon＠bugs.koha-community.org 17 Aug '17

17 Aug '17

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10306 --- Comment #37 from Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> --- Resolved a "Odd number of elements in anonymous hash" warning in koha2marclinks. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 10306] Koha to MARC mappings (Part 1): Allow multiple mappings per kohafield (for say 260/RDA 264)
by bugzilla-daemon＠bugs.koha-community.org 17 Aug '17

17 Aug '17

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10306 Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #65953|0 |1 is obsolete| | --- Comment #36 from Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> --- Created attachment 66135 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=66135&action=ed… Bug 10306: Adjusted online help text for koha2marclinks Test plan: Click on Help. Read the text. -- You are receiving this mail because: You are watching all bug changes.

1 0

[Bug 10306] Koha to MARC mappings (Part 1): Allow multiple mappings per kohafield (for say 260/RDA 264)
by bugzilla-daemon＠bugs.koha-community.org 17 Aug '17

17 Aug '17

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=10306 Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #65952|0 |1 is obsolete| | --- Comment #35 from Marcel de Rooy <m.de.rooy(a)rijksmuseum.nl> --- Created attachment 66134 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=66134&action=ed… Bug 10306: Support for multiple mappings in koha2marclinks This actually refactors koha2marclinks.pl in order to support multiple mappings per kohafield. Instead of three separate mapping pages for biblio, biblioitems and items, the script now lists them together. This gives a complete overview of all mappings rightaway. Changes are applied immediately across all frameworks. Note: This report handles the Default mappings just like it did before. In this script Koha already considered them as authoritative, although other parts of Koha did not. Follow-up report 19096 makes Default mappings authoritative throughout all Koha. On each line two buttons are provided, Add and Remove, in order to add or remove an individual mapping. We do no longer provide a separate form with the names of MARC tags. Since this form is targeted for administrators, it should be enough to ask for a field tag and subfield code. Note: The mappings for biblionumber, biblioitemnumber and itemnumber are so vital that this form marks them as readonly. It is not recommended to change them. Test plan: [1] Add a mapping. Verify via Frameworks or mysql command line that the kohafield is saved to the other frameworks too. [2] Remove the mapping again. Check Frameworks or mysql cl again. [3] Test adding a second mapping. Map copyrightdate to 260c and 264c. And map biblioitems.place to 260a and 264a. [4] Edit biblio record 1: Put 1980 in 260c. Do not include 264c. Edit biblio record 2: Put 1990 in 264c. Do not include 260c. Edit biblio record 3: Put 2000 in both 260c and 264c. Put CityA in 260a and in 264a. Edit biblio record 4: Put 2010 in 260c, and 2015 in 264c (which you should refuse normally). Put CityA in 260a, and CityB in 264a. [5] Create a report that shows biblioitems.place and biblio.copyrightdate for those biblio records. Record 4 should have 2010 in copyrightdate (since TransformMarcToKoha picks the first year for copyrightdate). Record 3 should have place CityA; record 4 should have CityA | CityB. Note: The CityA | CityB example illustrates that we should add some additional handling in TransformMarcToKoha for multiple 264s. [6] Add these four biblio records to a new list. Sort by Year. With OPACXSLTListsDisplay==default, check if the order = 1,2,3,4. (The order is based on biblio.copyrightdate.) Note that (RDA) record 2 would be on top without this patch set, since copyrightdate would have been null. -- You are receiving this mail because: You are watching all bug changes.

1 0