https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #1 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 131753 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131753&action=edit Add a mapping for authority marc format field 040 to a new Subject-heading-thesaurus-conventions field Current status: NEW To test this patch do the following: 1. Load some sample authority records ./bulkmarcimport.pl -v -a -file /var/tmp/authorities.mrc -insert -c=MARC21 2. Load a bibliographic record ./bulkmarcimport.pl -v -b -file /var/tmp/biblios.mrc -insert -c=MARC21 3. Search the ISBN 0704328623 and verify that the subject heading is not linked 4. Apply the patch 5. Reset the ElasticSearch mappings (Administration > Search Engine Configuration (ElasticSearch)) 6. Reindex ElasticSearch authorities index. /home/vagrant/kohaclone/misc/search_tools/rebuild_elasticsearch.pl -a -d -v 7. Link bibliographic records to authority records /home/vagrant/kohaclone/misc/link_bibs_to_authorities.pl -v -l 8. Repeat the Search in step 3 and verify that all the Feminism headings are linked. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #3 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 131755 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131755&action=edit Sample authority records -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Frank Hansen <frank.hansen@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Frank Hansen <frank.hansen@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131754|0 |1 is obsolete| | --- Comment #6 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 131767 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131767&action=edit Handle subject headings from differens thesaurus -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #8 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 131769 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131769&action=edit Bug 30280: Add a new mapping authority marc field 040$f -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #10 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 131771 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131771&action=edit Bug 30280: Removing tab character in Heading.pm -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Frank Hansen <frank.hansen@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131766|0 |1 is obsolete| | Attachment #131767|0 |1 is obsolete| | Attachment #131768|0 |1 is obsolete| | Attachment #131769|0 |1 is obsolete| | Attachment #131770|0 |1 is obsolete| | Attachment #131771|0 |1 is obsolete| | --- Comment #11 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 131772 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131772&action=edit Bug 30280: Add a new mapping authority marc field 040\$f To test this patch do the following: 1. Load some sample authority records ./bulkmarcimport.pl -v -a -file /var/tmp/authorities.mrc -insert -c=MARC21 2. Load a bibliographic record ./bulkmarcimport.pl -v -b -file /var/tmp/biblios.mrc -insert -c=MARC21 3. Search the ISBN 0704328623 and verify that the subject heading is not linked 4. Apply the patch 5. Reset the ElasticSearch mappings (Administration > Search Engine Configuration (ElasticSearch)) 6. Reindex ElasticSearch authorities index. /home/vagrant/kohaclone/misc/search_tools/rebuild_elasticsearch.pl -a -d -v 7. Link bibliographic records to authority records /home/vagrant/kohaclone/misc/link_bibs_to_authorities.pl -v -l 8. Repeat the Search in step 3 and verify that all the Feminism headings are linked. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #13 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 131774 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=131774&action=edit Bug 30280: Removing tab character in Heading.pm -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |phil@chetcolibrary.org -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Sarah Cornell <sbcornell@cityofportsmouth.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |sbcornell@cityofportsmouth. | |com --- Comment #14 from Sarah Cornell <sbcornell@cityofportsmouth.com> --- We are using multiple thesauruses as well and would like the linker to be able to distinguish between them too. Here is a typical example: 600 1 0 ‡aGammage, Jeff.‡0n 2007005947‡9104714 650 _ 0 ‡aAdoptive parents‡vBiography.‡0sh 92003241 650 _ 0 ‡aIntercountry adoption‡zChina.‡0sh 85067220 650 _ 0 ‡aIntercountry adoption‡zUnited States.‡0sh2008104783‡9227784 650 _ 7 ‡aAdoptive parents.‡2homoit‡0homoit0000010‡9218151 We would like the linker to consider the 6XX$0 field and the 6XX$2 field when it selects the authority record for the 6XX$9 link. The 6XX0 field matches with either the 010 (LC headings) or the 024 (other thesauruses). Using the 6XX$0 to match would also make it possible for the linker to find the match for compound headings, which you can see it doesn't find for two of the 6XXs in my example. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |frank.hansen@ub.lu.se |ity.org | CC| |david@davidnind.com -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #17 from David Nind <david@davidnind.com> --- Created attachment 137440 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137440&action=edit Screenshot - Example record before patch applied (subject terms have links) Testing notes so far (using koha-testing-docker): 1. Enable Elasticsearch as the search engine and reindex (koha-elasticsearch --rebuild -d -b -a kohadev) 2. For step 1: misc/migration_tools/bulkmarcimport.pl -v -a -file authorities.mrc -insert -c=MARC21 3. For step 2: misc/migration_tools/bulkmarcimport.pl -v -b -file biblios.mrc -insert -c=MARC21 4. For step 4, subject terms have search links (see attached screenshot) - search query example for one of the links http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:%22Feminism%22 5. For step 5, I get the error message in comment #16 -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #18 from Frank Hansen <frank.hansen@ub.lu.se> --- (In reply to David Nind from comment #16)

Hi Frank.

I am getting an error message when resetting the mappings after the patch is applied (step 5 of the test plan) - this worked without error before the patch was applied:

DBIx::Class::Storage::DBI::_dbh_execute(): DBI Exception: DBD::mysql::st execute failed: Data truncated for column 'marc_type' at row 1 at /kohadevbox/koha/Koha/Objects.pm line 114 at /usr/share/perl5/DBIx/Class/Exception.pm line 77

in DBIx::Class::Exception::throw at /usr/share/perl5/DBIx/Class/Exception.pm line 77

74: my $self = { msg => $msg }; 75: bless $self => $class; 76: 77: die $self; 78: } 79: 80: =head2 rethrow

.....

David

Hi David, I don't get this error. I don't using docker myself but a kohadevbox. I using the latest debian 11 box to create the koha enviroment with ElasticSearch 6. I did the following: 1. Creating the koha database instance by using the webinstaller. 2. Loaded demo records into the database (i using kshell when I work with the records) perl ../misc4dev/insert_data.pl rebuild_elasticsearch.pl -d -v 3. Enabling Elasticsearch as search engine in intranet 4. Loading the sample records with bulkmarkimport.pl: ./bulkmarcimport.pl -v -a -file /var/tmp/authorities.mrc -insert -c=MARC21 ./bulkmarcimport.pl -v -b -file /var/tmp/biblios.mrc -insert -c=MARC21 5. I verified if the sample records were loaded properly. I experienced that the authority record was loaded and indexed ok. But the biblio record weren't indexed for some reason. They seems to have been loaded into the database but somehow Koha didn't indexed the record. This command worked for me before. I had to reindex the biblio records again before I applied the patch: /home/vagrant/kohaclone/misc/search_tools/rebuild_elasticsearch.pl -d -v 6. The sample biblio have topic terms but they are not linked to any authority records. 7. Applying the patch. 8. Resetting the mappings which work for me without any any errors as described above. 9. Rebuilding the authority index. /home/vagrant/kohaclone/misc/search_tools/rebuild_elasticsearch.pl -a -d -v 10. Link bibliographic records to authority records /home/vagrant/kohaclone/misc/link_bibs_to_authorities.pl -v -l 11. When I search the record, the topic_terms is linked. Is there some differences between docker and kohadevbox when it comes to mappings? Regards, Frank -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #20 from David Nind <david@davidnind.com> --- I tested (after making the change in comment #19) using koha-testing-docker. Test plan: 1. Enable Elasticsearch as the search engine (SearchEngine system preference = Elasticsearch) and reindex (koha-elasticsearch --rebuild -d -b -a kohadev). 2. Load sample authority records attached to the bug (in this case the file was saved in the koha directory - adjust file location as appropriate):[1] misc/migration_tools/bulkmarcimport.pl -v -a -file authorities.mrc -insert -c=MARC21 3. Load the sample bibliographic record attached to the bug: misc/migration_tools/bulkmarcimport.pl -v -b -file biblios.mrc -insert -c=MARC21 4. Search for ISBN 0704328623 and verify that the subject terms are not linked to any authority records (if you hover over the link terms, all links should look like ../cgi-bin/koha/catalogue/search.pl?q=su:"subjectterm" - none should have any ?q=an:XXX (where XXX = authority record ids) in the link). 5. Apply the patches.[2] 6. Reset the Elasticsearch mappings (Administration > Catalog > Search engine configuration (Elasticsearch)). 7. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev 8. Link bibliographic records to authority records: misc/link_bibs_to_authorities.pl -v -l 9. Repeat the search in step 4 (or refresh the record details page) and verify that all the Feminism headings are linked to an authority record (should have ?q=an:XXX (where XXX = different authority record ids for the various feminism authority records)).[3] [1] After importing the authorities, I got an error and had to reindex: misc/migration_tools/bulkmarcimport.pl -v -a -file authorities.mrc -insert -c=MARC21 Characteristic MARC flavour: MARC21 .One or more ElasticSearch errors occurred when indexing documents at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Indexer.pm line 148. .One or more ElasticSearch errors occurred when indexing documents at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Indexer.pm line 148. .One or more ElasticSearch errors occurred when indexing documents at /kohadevbox/koha/Koha/SearchEngine/Elasticsearch/Indexer.pm line 148. [2] After applying the patch I edited mappings.yaml to remove the nomarc entry that was added so the resetting the mappings worked vi admin/searchengine/elasticsearch/mappings.yaml - remove lines 750-754 added by the mapping patch [3] Not all links to feminism are linked to authority records. Links for subject terms for step 9 in order are: Subject(s:) Feminism | feminism | Feminism | Feminism | Idéhistoria | Litteratur http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1709 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1710 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1708 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Feminism" http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Idéhistoria" http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Litteratur" MARC info: 650 #7 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 0 Authority record control number or standard number https://id.kb.se/term/sao/Feminism 2 Source of heading or term sao 650 #7 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element feminism 2 Source of heading or term bnb 650 #0 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Idéhistoria 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Litteratur -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Frank Hansen <frank.hansen@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131772|0 |1 is obsolete| | --- Comment #22 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 137818 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137818&action=edit Bug 30280: Add a new mapping authority marc field 040\$f -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Frank Hansen <frank.hansen@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #131774|0 |1 is obsolete| | --- Comment #24 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 137820 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137820&action=edit Bug 30280: Removing tab character in Heading.pm -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137819|0 |1 is obsolete| | --- Comment #27 from David Nind <david@davidnind.com> --- Created attachment 137891 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137891&action=edit Bug 30280: Handle subject headings from differens thesaurus Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This enhancement adds release notes| |support for displaying and | |linking to subject headings | |from different thesaurus | |when using Elasticsearch. | |The thesaurus used for the | |term is added to the | |authority record using | |040$f. For the | |bibliographic record, set | |the second indicator to 7 | |for 650 and add the source | |to 650$2. For local terms, | |use 4 (Source not | |specified) as the | |indicator. | | | |Example for a | |bibliographic record: | | | |650 | |_ 0 $a Feminism | |650 _ 7 $a | |Feminism $2 sao | |650 _ 7 $a | |Feminism $2 barn | | | |The first | |example above is the LCSH | |term. The other two terms | |are from sao (controlled | |Swedish subject heading | |system) and barn (Swedish | |children subject heading | |system). These three are | |using the same TOPIC_TERM | |Feminism, but they belong | |to different thesaurus. --- Comment #29 from David Nind <david@davidnind.com> --- (In reply to Frank Hansen from comment #25)

I fixed the mapping issue

Thanks Frank! I have re-tested and signed off. Re replies on comment #21 - thanks for the explanations for my footnote [3]. I also didn't get the error message mentioned in my footnote [1] this time, so not sure what was going on whenI orginally tested this... The QA Team may ask you to amend the patches as per the commit message guidelines https://wiki.koha-community.org/wiki/Commit_messages I've had a go at the text for the release note, feel free to update it as I'm not sure I've described this correctly. Test plan: 1. Enable Elasticsearch as the search engine (SearchEngine system preference = Elasticsearch) and reindex (koha-elasticsearch --rebuild -d -b -a kohadev). 2. Load sample authority records attached to the bug (in this case the authorities.mrc file was saved in the koha directory - adjust file location as appropriate): misc/migration_tools/bulkmarcimport.pl -v -a -file authorities.mrc -insert -c=MARC21 3. Load the sample bibliographic record attached to the bug: misc/migration_tools/bulkmarcimport.pl -v -b -file biblios.mrc -insert -c=MARC21 4. Search for ISBN 0704328623 and verify that the subject terms are not linked to any authority records (if you hover over the link terms, all links should look like ../cgi-bin/koha/catalogue/search.pl?q=su:"subjectterm" - none should have any ?q=an:XXX (where XXX = authority record ids) in the link). 5. Apply the patches. 6. Reset the Elasticsearch mappings (Administration > Catalog > Search engine configuration (Elasticsearch)). 7. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev 8. Link bibliographic records to authority records: misc/link_bibs_to_authorities.pl -v -l 9. Repeat the search in step 4 (or refresh the record details page) and verify that the first three Feminism headings[1] are linked to an authority record (should have ?q=an:XXX (where XXX = different authority record IDs for the various feminism authority records linked to a specific thesaurus)). 10. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev [1] Links for subject terms for step 9 in order are (the authority ids may be different depending on how ou are testing): Subject(s:) Feminism | feminism | Feminism | Feminism | Idéhistoria | Litteratur http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1709 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1710 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1708 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Feminism" http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Idéhistoria" http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Litteratur" MARC info: 650 #7 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 0 Authority record control number or standard number https://id.kb.se/term/sao/Feminism 2 Source of heading or term sao 650 #7 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element feminism 2 Source of heading or term bnb 650 #0 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Idéhistoria 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Litteratur Note: The fourth Feminism term has second indicator 4 which means "Source not specified". So this term is only a local term and does not belong to any specific thesaurus and thus not linked. To specify a source of a term, the second indicator must be set as 7. The source of heading is then specified in subfield 2. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #31 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 137898 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137898&action=edit 30280: Add support for subject headings from different thesaurus when using Elasticsearch This patch allows subject headings from different thesaurus to co-exist in Koha. Test plan: 1. Enable Elasticsearch as the search engine (SearchEngine system preference = Elasticsearch) and reindex (koha-elasticsearch --rebuild -d -b -a kohadev). 2. Load sample authority records attached to the bug (in this case the authorities.mrc file was saved in the koha directory - adjust file location as appropriate): misc/migration_tools/bulkmarcimport.pl -v -a -file authorities.mrc -insert -c=MARC21 3. Load the sample bibliographic record attached to the bug: misc/migration_tools/bulkmarcimport.pl -v -b -file biblios.mrc -insert -c=MARC21 4. Search for ISBN 0704328623 and verify that the subject terms are not linked to any authority records (if you hover over the link terms, all links should look like ../cgi-bin/koha/catalogue/search.pl?q=su:"subjectterm" - none should have any ?q=an:XXX (where XXX = authority record ids) in the link). 5. Apply the patches. 6. Reset the Elasticsearch mappings (Administration > Catalog > Search engine configuration (Elasticsearch)). 7. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev 8. Link bibliographic records to authority records: misc/link_bibs_to_authorities.pl -v -l 9. Repeat the search in step 4 (or refresh the record details page) and verify that the first three Feminism headings[1] are linked to an authority record (should have ?q=an:XXX (where XXX = different authority record IDs for the various feminism authority records linked to a specific thesaurus)). 10. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev [1] Links for subject terms for step 9 in order are (the authority ids may be different depending on how ou are testing): Subject(s:) Feminism | feminism | Feminism | Feminism | Idéhistoria | Litteratur http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1709 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1710 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1708 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Feminism" http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Idéhistoria" http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Litteratur" MARC info: 650 #7 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 0 Authority record control number or standard number https://id.kb.se/term/sao/Feminism 2 Source of heading or term sao 650 #7 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element feminism 2 Source of heading or term bnb 650 #0 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Idéhistoria 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Litteratur Note: The fourth Feminism term has second indicator 4 which means "Source not specified". So this term is only a local term and does not belong to any specific thesaurus and thus not linked. To specify a source of a term, the second indicator must be set as 7. The source of heading is then specified in subfield 2. Sponsored-by: Lund University Library, Sweden Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Frank Hansen <frank.hansen@ub.lu.se> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137890|0 |1 is obsolete| | Attachment #137891|0 |1 is obsolete| | Attachment #137892|0 |1 is obsolete| | Attachment #137897|0 |1 is obsolete| | Attachment #137898|0 |1 is obsolete| | Attachment #137899|0 |1 is obsolete| | --- Comment #33 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 137900 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137900&action=edit Bug 30280: Elasticsearch - Add 040f to Subject-heading-thesaurus-conventions (new) authority mapping index field (MARC21) This patch adds 040 $f to a new field Subject-heading-thesaurus-conventions authority index mapping. To test: 1) Apply patch 2) Reindex using rebuild_elasticsearch.pl -r If you don't have access to a terminal (in a sandbox for example) 2a) Go to Administration > Search engine configuration, click "Reset mappings" and confirm 2b) Then reindex Sponsored-by: Lund University Library, Sweden Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #35 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 137902 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=137902&action=edit Bug 30280: (follow-up) Removing tab character in Heading.pm Sponsored-by: Lund University Library, Sweden Signed-off-by: David Nind <david@davidnind.com> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #37 from David Nind <david@davidnind.com> --- (In reply to Frank Hansen from comment #36) Excellent! Good luck for QA. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Koha does not currently |Support authority records |support authority records |with common subject |with common subject |headings from different |headings from different |thesaurus |thesaurus | -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #137901|0 |1 is obsolete| | --- Comment #39 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 139855 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=139855&action=edit 30280: Add support for subject headings from different thesaurus when using Elasticsearch This patch allows subject headings from different thesaurus to co-exist in Koha. Test plan: 1. Enable Elasticsearch as the search engine (SearchEngine system preference = Elasticsearch) and reindex (koha-elasticsearch --rebuild -d -b -a kohadev). 2. Load sample authority records attached to the bug (in this case the authorities.mrc file was saved in the koha directory - adjust file location as appropriate): misc/migration_tools/bulkmarcimport.pl -v -a -file authorities.mrc -insert -c=MARC21 3. Load the sample bibliographic record attached to the bug: misc/migration_tools/bulkmarcimport.pl -v -b -file biblios.mrc -insert -c=MARC21 4. Search for ISBN 0704328623 and verify that the subject terms are not linked to any authority records (if you hover over the link terms, all links should look like ../cgi-bin/koha/catalogue/search.pl?q=su:"subjectterm" - none should have any ?q=an:XXX (where XXX = authority record ids) in the link). 5. Apply the patches. 6. Reset the Elasticsearch mappings (Administration > Catalog > Search engine configuration (Elasticsearch)). 7. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev 8. Link bibliographic records to authority records: misc/link_bibs_to_authorities.pl -v -l 9. Repeat the search in step 4 (or refresh the record details page) and verify that the first three Feminism headings[1] are linked to an authority record (should have ?q=an:XXX (where XXX = different authority record IDs for the various feminism authority records linked to a specific thesaurus)). 10. Reindex: koha-elasticsearch --rebuild -d -b -a kohadev [1] Links for subject terms for step 9 in order are (the authority ids may be different depending on how ou are testing): Subject(s:) Feminism | feminism | Feminism | Feminism | Idéhistoria | Litteratur http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1709 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1710 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=an:1708 http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Feminism" http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Idéhistoria" http://127.0.0.1:8081/cgi-bin/koha/catalogue/search.pl?q=su:"Litteratur" MARC info: 650 #7 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 0 Authority record control number or standard number https://id.kb.se/term/sao/Feminism 2 Source of heading or term sao 650 #7 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element feminism 2 Source of heading or term bnb 650 #0 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Feminism 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Idéhistoria 650 #4 - SUBJECT ADDED ENTRY--TOPICAL TERM a Topical term or geographic name entry element Litteratur Note: The fourth Feminism term has second indicator 4 which means "Source not specified". So this term is only a local term and does not belong to any specific thesaurus and thus not linked. To specify a source of a term, the second indicator must be set as 7. The source of heading is then specified in subfield 2. Sponsored-by: Lund University Library, Sweden Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA --- Comment #41 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Thx for the great work on the test plan and sample data, this makes it much easier to test :) I am happy to add another sign-off, but for this to go through QA, we will also need unit tests to cover the changes in the .pm files. Please let us know if we can help with that! -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #43 from Frank Hansen <frank.hansen@ub.lu.se> --- (In reply to Katrin Fischer from comment #41)

Thx for the great work on the test plan and sample data, this makes it much easier to test :)

I am happy to add another sign-off, but for this to go through QA, we will also need unit tests to cover the changes in the .pm files. Please let us know if we can help with that!

Yes, there are already existing tests in these files: t/db_dependent/Linker_Default.t t/db_dependent/Koha/SearchEngine/Elasticsearch/QueryBuilder.t They need some minor changes so they will PASS tests. This test outputs warnings even if it says OK: t/db_dependent/Linker_FirstMatch.t Use of uninitialized value $thesaurus in concatenation (.) or string at /kohadevbox/koha/C4/Linker/Default.pm line 38. Use of uninitialized value $thesaurus in concatenation (.) or string at /kohadevbox/koha/C4/Linker/Default.pm line 79. Use of uninitialized value $thesaurus in concatenation (.) or string at /kohadevbox/koha/C4/Linker/Default.pm line 80. Use of uninitialized value $thesaurus in concatenation (.) or string at /kohadevbox/koha/C4/Linker/Default.pm line 81. Use of uninitialized value $thesaurus in concatenation (.) or string at /kohadevbox/koha/C4/Linker/Default.pm line 82. t/db_dependent/Linker_FirstMatch.t .. ok All tests successful. These warnings doesn't appear in t/db_dependent/Linker_Default.t I would like to add a few more test to t/db_dependent/Linker_Default.t to cover my changes in unit test. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #45 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 140041 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=140041&action=edit Bug 30280: (follow-up) Removing tab character in Heading.pm Sponsored-by: Lund University Library, Sweden Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #46 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 140042 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=140042&action=edit Bug 30280 - Unit tests -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #48 from Frank Hansen <frank.hansen@ub.lu.se> --- Created attachment 140045 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=140045&action=edit Bug 30280: (follow-up) Removing tab character in Heading.pm Sponsored-by: Lund University Library, Sweden Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #139854|0 |1 is obsolete| | --- Comment #50 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 140284 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=140284&action=edit Bug 30280: Elasticsearch - Add 040f to Subject-heading-thesaurus-conventions (new) authority mapping index field (MARC21) This patch adds 040 $f to a new field Subject-heading-thesaurus-conventions authority index mapping. To test: 1) Apply patch 2) Reindex using rebuild_elasticsearch.pl -r If you don't have access to a terminal (in a sandbox for example) 2a) Go to Administration > Search engine configuration, click "Reset mappings" and confirm 2b) Then reindex Sponsored-by: Lund University Library, Sweden Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #140045|0 |1 is obsolete| | --- Comment #52 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 140286 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=140286&action=edit Bug 30280: (follow-up) Removing tab character in Heading.pm Sponsored-by: Lund University Library, Sweden Signed-off-by: David Nind <david@davidnind.com> Signed-off-by: Katrin Fischer <katrin.fischer.83@web.de> -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #54 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 22.11. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |31925 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=31925 [Bug 31925] AutoCreateAuthorities needs to preserve subject heading thesaurus to not create duplicate authority records -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #56 from Frank Hansen <frank.hansen@ub.lu.se> --- (In reply to Katrin Fischer from comment #55)

hi Frank, could you maybe have a look at the new bugs in Blocks? bug 31924 and bug 31925.

Hi Katrin, Well 31924 is probably a Zebra related issue. I got the impression that linking of subject headings based on thesaurus already works with Zebra. It is at least that's what I thought when I read this in the authority-koha-indexdefs.xml file: <kohaidx:index_subject_thesaurus tag="008" offset="11" length="1" detail_tag="040" detail_subfields="f"> <kohaidx:target_index>Subject-heading-thesaurus:w</kohaidx:target_index> </kohaidx:index_subject_thesaurus> Here is already a reference to 040$f which is a crucial part when it comes to other thesaurus. But I see that the linking does not work at all in Zebra. Here at Lund University, we are only using ElasticSearch. In fact, we Zebra is not running at all. So it is difficult for me to test Zebra in real life. I'n not an expert on Zebra. Probably the new subject-heading-thesaurus-conventions index needs to be set up in Zebra as well. I'm not 100% certain how to do that. As for 31925, that's the next step. We don't use AutoLinkBiblios because it doesn't work. We import the subject headings from different thesaurus from Royal Library (Libris). So for avoid 30280 to being too complex, I have focused on getting the thesaurus based subject heading linking to work. Since AutoLinkBiblios only creates LCSH subject headings we cannot use this. But it would be awesome to get it to work later on. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 --- Comment #58 from Frank Hansen <frank.hansen@ub.lu.se> --- (In reply to Katrin Fischer from comment #57)

The problem is, we broke things with this one for people and a lot of libraries - we need to find a fix.

Katrin, I have been working on a fix to make it compatible with Zebra. There is some code to be uploaded as well. The patch should work in theory but I can get Zebra to work as expected. But it work nicely with Elasticsearch. Do you want me to submit the patch here or to Bug 31924? I will definitely need some help with setting up the search field correctly. -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Phil Ringnalda <phil@chetcolibrary.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |32916 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32916 [Bug 32916] [Bug 30280 follow-up] Problems in linking authorities to biblio fields (MARC 21) -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |caroline.cyr-la-rose@inlibr | |o.com Blocks| |33277, 33557 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33277 [Bug 33277] Correctly handle linking subfields with no defined thesaurus https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33557 [Bug 33557] Add a system preference to disable/enable thesaurus checking during authority linking -- You are receiving this mail because: You are watching all bug changes.

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30280 Nick Clemens (kidclamp) <nick@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |35125 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35125 [Bug 35125] AutoCreateAuthorities creates separate authorities when thesaurus differs, even with LinkerConsiderThesaurus set to Don't -- You are receiving this mail because: You are watching all bug changes.