[Bug 7445] New: Clicking on a tag gives "Language ... does not exist"
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Bug #: 7445 Summary: Clicking on a tag gives "Language ... does not exist" Classification: Unclassified Change sponsored?: --- Product: Koha Version: master Platform: All OS/Version: All Status: NEW Severity: normal Priority: P5 - low Component: OPAC AssignedTo: oleonard@myacpl.org ReportedBy: magnus@enger.priv.no QAContact: ian.walls@bywatersolutions.com To reproduce: - Install another language alongside English - Make sure this language is chosen in the OPAC - Click on a tag, either in a result list or on the "Tag cloud" page - The message "Language 'nb-no' does not exist" is displayed (or some other language code) When English is the chosen language, things work as expected. I have tested this on current master and 3.6.2, and the problem is the same in both places. It can be seen in master here: http://head.bibkat.no/cgi-bin/koha/opac-tags.pl (Remember to choose Norwegian as language) See also attached screenshot. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 --- Comment #1 from Magnus Enger <magnus@enger.priv.no> 2012-01-13 17:12:16 UTC --- Checking a little more... This gives the error: http://head.bibkat.no/cgi-bin/koha/opac-search.pl?tag=code&q=code This works: http://head.bibkat.no/cgi-bin/koha/opac-search.pl?q=code -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 --- Comment #2 from Magnus Enger <magnus@enger.priv.no> 2012-01-17 17:40:40 UTC --- The problem seems to stem from Lingua::Stem::Snowball and specifically the call to it on line 741 of C4/Search.pm - in the _build_stemmed_operand function: my $stemmer = Lingua::Stem::Snowball->new( lang => $lang, encoding => "UTF-8" ); For Norwegian, $lang holds "nb-NO". Replacing that temporarily with a literal value of "no" solves the problem. The documentation (e.g. http://search.cpan.org/dist/Lingua-Stem-Snowball/lib/Lingua/Stem/Snowball.pm) for Lingua::Stem::Snowball->new says this about the lang parameter: "lang: An ISO code taken from the table of supported languages, above." The table referenced only lists two-letter language codes, specifically these: da, nl, en, fi, fr, de, hu, it, no, pt, ro, ru, es, sv, tr. The syspref QueryStemming influences the observed behviour. When the chosen language in the OPAC is Norwegian (nb-NO), this works regardless of what QueryStemming is set to: http://head.bibkat.no/cgi-bin/koha/opac-search.pl?q=code Clicking on a tag in the result list gives this URL: http://head.bibkat.no/cgi-bin/koha/opac-search.pl?tag=code&q=code When QueryStemming = "Try" this fails with the "Language ... does not exist" message. When QueryStemming = "Don't try" a result list is shown, as expected. Still confused... -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 --- Comment #3 from Magnus Enger <magnus@enger.priv.no> 2012-01-17 20:34:56 UTC --- In 3.4.6 it works for Swedish, regardless of what QueryStemming is set to, but not for Norwegian. There's a difference between 3.4.6 and current master: In 3.4.6 $lang is taken from C4::Templates::getlanguagecookie: 389 sub getlanguagecookie { 390 my ($query) = @_; 391 my $lang; 392 if ($query->cookie('KohaOpacLanguage')){ 393 $lang = $query->cookie('KohaOpacLanguage') ; 394 }else{ 395 $lang = $ENV{HTTP_ACCEPT_LANGUAGE}; 396 397 } 398 $lang = substr($lang, 0, 2); 399 400 return $lang; 401 } In current master it comes from C4::Templates::getlanguage: 311 sub getlanguage { 312 my ($query, $interface) = @_; 313 314 # Select a language based on cookie, syspref available languages & browser 315 my $is_intranet = $interface eq 'intranet'; 316 my @languages = split(",", C4::Context->preference( 317 $is_intranet ? 'language' : 'opaclanguages')); 318 319 my $lang; 320 321 # cookie 322 if ( $query->cookie('KohaOpacLanguage') ) { 323 $lang = $query->cookie('KohaOpacLanguage'); 324 $lang =~ s/[^a-zA-Z_-]*//; # sanitize cookie 325 } 326 327 # HTTP_ACCEPT_LANGUAGE 328 unless ($lang) { 329 my $http_accept_language = $ENV{ HTTP_ACCEPT_LANGUAGE }; 330 $lang = accept_language( $http_accept_language, 331 getTranslatedLanguages($interface,'prog') ); 332 } 333 334 # Ignore a lang not selected in sysprefs 335 $lang = undef unless first { $_ eq $lang } @languages; 336 337 # Fall back to English if necessary 338 $lang = 'en' unless $lang; 339 340 return $lang; 341 } In 3.4.6 "sv-SE" is reduced to "sv" on line 398 and this is a valid language for Lingua::Stem::Snowball. "nb-NO" is reduced to "nb", which is not valid in the eyes of Lingua::Stem::Snowball. C4::Templates::getlanguage returns the unreduced strings, neither of which is valid for Lingua::Stem::Snowball. -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Albert Oller <ago@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ago@bywatersolutions.com --- Comment #4 from Albert Oller <ago@bywatersolutions.com> 2012-01-17 21:00:12 UTC --- Hi Magnus, we are working on similar problems. One site has es-ES language pack installed, 6.02. All of the English tags work, but when switching to Spanish, the tags won't work if there is only a single item returned. Error: Language 'es-es' does not exist At another site, with only English, none of the tags works: Error: Language 'en-us,en;q=0.5' does not exist Turning off Stemming, does allow the tags to work. Best regards, Albert -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 --- Comment #5 from Magnus Enger <magnus@enger.priv.no> 2012-01-17 21:14:22 UTC --- Looks to me like there are two problems here:
Error: Language 'es-es' does not exist
At another site, with only English, none of the tags works: Error: Language 'en-us,en;q=0.5' does not exist
1. These "long" language codes need to be reduced to two-letter codes that Lingua::Stem::Snowball will accept. 2. Why does the error from the stemmer only show up for opac-search.pl?tag=code&q=code and not for opac-search.pl?q=code ? -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Fridolyn SOMERS <fridolyn.somers@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolyn.somers@gmail.com --- Comment #6 from Fridolyn SOMERS <fridolyn.somers@gmail.com> 2012-01-27 09:28:32 UTC --- (In reply to comment #5)
1. These "long" language codes need to be reduced to two-letter codes that Lingua::Stem::Snowball will accept.
I confirm Lingua::Stem::Snowball needs a 2-characters lang. It can be simply corrected in Search.pm : my $stemmer = Lingua::Stem::Snowball->new( lang => lc(substr($lang, 0, 2)), encoding => "UTF-8" ); See : http://annocpan.org/~CREAMYG/Lingua-Stem-Snowball-0.952/lib/Lingua/Stem/Snow... (Sorry, I don't have time to create a patch). -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 --- Comment #7 from Magnus Enger <magnus@enger.priv.no> 2012-01-27 09:35:37 UTC --- (In reply to comment #6)
my $stemmer = Lingua::Stem::Snowball->new( lang => lc(substr($lang, 0, 2)),
If only it were that simple... ;-) For the two variants of Norwegian, this would result in nb-NO -> nb nn-NO -> nn but neither of these are valid for Lingua::Stem::Snowball, which expects Norwegian to be coded as "no". I have done a quick and dirty hack for my installations which looks like this: $lang = substr($lang, 0, 2); if ($lang eq 'nb' || $lang eq 'nn') { $lang = 'no'; } but that is probably not the way to go... -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Marc Véron <veron@veron.ch> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |veron@veron.ch --- Comment #8 from Marc Véron <veron@veron.ch> 2012-02-21 15:13:53 UTC --- The solution of Fridolyn Somers works on my server for German and French, and I expect it to work for Italian as well. I think the most pragmatic solution would be to implement the code of Magnus Enger, because Norwegian seems to be the only exception for the list of languages supported by Snowball, see: http://annocpan.org/~CREAMYG/Lingua-Stem-Snowball-0.952/lib/Lingua/Stem/Snow... Language | ISO code ----------------------- Danish | da Dutch | nl English | en Finnish | fi French | fr German | de Hungarian | hu Italian | it Norwegian | no Portuguese | pt Romanian | ro Russian | ru Spanish | es Swedish | sv Turkish | tr -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Marijana <mglavica@ffzg.hr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mglavica@ffzg.hr -- Configure bugmail: http://bugs.koha-community.org/bugzilla3/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 --- Comment #9 from Magnus Enger <magnus@enger.priv.no> --- Created attachment 8773 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=8773&action=edit Bug 7445 - Clicking on a tag gives "Language ... does not exist" To test: - Syspref QueryStemming = Try - Install Norwegian bokmål: cd misc/translator/ perl translate install nb-NO - Go to Home › Administration › System Preferences > I18N/L10N and enable "Norsk bokmål(nb-NO)" for opaclanguages as well as setting opaclanguagesdisplay = Allow - Make sure you have selected "Norsk bokmål" as the active language in the OPAC - Find a record that has a tag (which does not contain any digits) - Click on the tag and see that you get the error in the title of this bug - Apply the patch - Click on the tag again and the error should be gone -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Magnus Enger <magnus@enger.priv.no> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|oleonard@myacpl.org |magnus@enger.priv.no -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Dobrica Pavlinusic <dpavlin@rot13.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dpavlin@rot13.org -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #8773|0 |1 is obsolete| | --- Comment #10 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Created attachment 8888 --> http://bugs.koha-community.org/bugzilla3/attachment.cgi?id=8888&action=edit [SIGNED-OFF] Bug 7445 - Clicking on a tag gives "Language ... does not exist" To test: - Syspref QueryStemming = Try - Install Norwegian bokmål: cd misc/translator/ perl translate install nb-NO - Go to Home › Administration › System Preferences > I18N/L10N and enable "Norsk bokmål(nb-NO)" for opaclanguages as well as setting opaclanguagesdisplay = Allow - Make sure you have selected "Norsk bokmål" as the active language in the OPAC - Find a record that has a tag (which does not contain any digits) - Click on the tag and see that you get the error in the title of this bug - Apply the patch - Click on the tag again and the error should be gone Signed-off-by: Katrin Fischer <Katrin.Fischer.83@web.de> Easy to test with a great test plan. Works nicely. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off CC| |katrin.fischer@bsz-bw.de -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Ian Walls <koha.sekjal@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA --- Comment #11 from Ian Walls <koha.sekjal@gmail.com> --- This patch just does a substring call to give Snowball what it needs, and adds handling for the only exception we currently know about. If we start finding more exceptions, we may want to revisit our methodology, but for now, I think having 1 special case is okay. Passed QA. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Paul Poulain <paul.poulain@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to Master CC| |paul.poulain@biblibre.com Version|master |rel_3_6 --- Comment #12 from Paul Poulain <paul.poulain@biblibre.com> --- QA comment: this code should, in theory, not be accepted because it's a workaround using something hardcoded. But Marc Veron, in comment 8 seems to prove that's really the only exception, and it's more a problem with Lingua::Stem than a problem in Koha itself, so OK to push ... patch pushed -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Jared Camins-Esakov <jcamins@cpbibliography.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to Master |Pushed to Stable --- Comment #13 from Jared Camins-Esakov <jcamins@cpbibliography.com> --- This bug will be included in the Koha 3.6.5 release. -- You are receiving this mail because: You are watching all bug changes.
http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7445 Karam Qubsi <karamqubsi@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |karamqubsi@gmail.com -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org