[Bug 30150] New: Dutch option in language limits is garbled
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 Bug ID: 30150 Summary: Dutch option in language limits is garbled Change sponsored?: --- Product: Koha Version: 21.11 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: OPAC Assignee: oleonard@myacpl.org Reporter: ephetteplace@cca.edu QA Contact: testopia@bugs.koha-community.org Created attachment 130998 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=130998&action=edit screenshot of bug To recreate: - add Dutch "dut" to the AdvancedSearchLanguages option - go to the advanced search page /cgi-bin/koha/opac-search.pl - click open the Language select menu - note that the text for Dutch is "nedərlɑns (Dutch)" It should render as "nedərlɑns (Dutch)" but instead the numerical character references for the fourth and seventh characters are rendered literally, as if they were escaped. I haven't looked into this further so I don't know _why_ this is happening but Koha's OPAC uses charset=utf-8 so I think we should be able to simply use the actual characters without any references. Perhaps it's a problem with the encoding of strings in the database? But when I run a report to pull all the rows from the `language_descriptions` table, the word "nedərlɑns" renders correctly. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 Eric Phetteplace <ephetteplace@cca.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ephetteplace@cca.edu -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 --- Comment #1 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Hi Eric, I see "Nederlands (Dutch)". What was your query to the language_descriptions table? For me it looks like this: MariaDB [koha_kohadev]> select * from language_descriptions where subtag = "nl"; +--------+----------+------+-----------------+-----+ | subtag | type | lang | description | id | +--------+----------+------+-----------------+-----+ | nl | language | de | Niederländisch | 148 | | nl | language | en | Dutch | 146 | | nl | language | fr | Néerlandais | 147 | | nl | language | nl | Nederlands | 145 | | NL | region | en | Netherlands | 319 | | NL | region | nl | Nederland | 320 | +--------+----------+------+-----------------+-----+ -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 --- Comment #2 from Eric Phetteplace <ephetteplace@cca.edu> --- I just used a report `select * FROM language_descriptions` and found the Dutch entries further down. If I run your query I get this: subtag type lang description id nl language de Niederländisch 138 nl language en Dutch 26 nl language fr Néerlandais 27 nl language nl nedərlɑns 25 NL region en Netherlands 299 NL region nl Nederland 300 Should we change the "nedərlɑns" description in the database to "Nederlands" then? It displays right in the reports results, the only problem is on the advanced search page. I see the same results you've shown on koha-testing-docker so perhaps this was changed at some point? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #3 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Eric Phetteplace from comment #2)
I just used a report `select * FROM language_descriptions` and found the Dutch entries further down. If I run your query I get this:
subtag type lang description id nl language de Niederländisch 138 nl language en Dutch 26 nl language fr Néerlandais 27 nl language nl nedərlɑns 25 NL region en Netherlands 299 NL region nl Nederland 300
Should we change the "nedərlɑns" description in the database to "Nederlands" then? It displays right in the reports results, the only problem is on the advanced search page. I see the same results you've shown on koha-testing-docker so perhaps this was changed at some point?
I am not sure if we can trust the report here... what you see here are nedərlɑns HTML-entities that would look correct in the browser as it renders them (unless they are escaped, as it probably happens on the advanced search. Could you try and download the CSV export to compare? About the right naming I am not sure, but adding Marcel for maybe some help. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 --- Comment #4 from Eric Phetteplace <ephetteplace@cca.edu> --- You're correct, it's stored in the database with HTML character references, but renders as you would want on the reports page. This is the CSV export's content from our live Koha instance (21.05.09.000): subtag,type,lang,description,id nl,language,de,Niederländisch,138 nl,language,en,Dutch,26 nl,language,fr,Néerlandais,27 nl,language,nl,nedərlɑns,25 NL,region,en,Netherlands,299 NL,region,nl,Nederland,300 So the question is why the character references aren't interpreted in the <select> menu on the advanced search page. All I could think to check is the encoding, but I see that both pages have "<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />" so I'm stumped. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 --- Comment #5 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- I believe the reason is that we are using the | html filter in the select - which is as it should be and escapes any code that might be there. The problem is in the database, as you said earlier we should just store the name in utf-8 and without the entities. Do you have access to your database to fix this with SQL? We might still want to look into why it was not updated when it was for others - it seems ok on newer installations, but that could take a while. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 --- Comment #6 from Eric Phetteplace <ephetteplace@cca.edu> --- (In reply to Katrin Fischer from comment #5)
I believe the reason is that we are using the | html filter in the select - which is as it should be and escapes any code that might be there. The problem is in the database, as you said earlier we should just store the name in utf-8 and without the entities. Do you have access to your database to fix this with SQL?
We might still want to look into why it was not updated when it was for others - it seems ok on newer installations, but that could take a while.
Our instance is hosted by ByWater but I'll reach out to them and link to this bug report. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 --- Comment #7 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Eric Phetteplace from comment #2)
I just used a report `select * FROM language_descriptions` and found the Dutch entries further down. If I run your query I get this:
subtag type lang description id nl language de Niederländisch 138 nl language en Dutch 26 nl language fr Néerlandais 27 nl language nl nedərlɑns 25 NL region en Netherlands 299 NL region nl Nederland 300
Should we change the "nedərlɑns" description in the database to "Nederlands" then?
Yes, Nederlands is the correct spelling. Change it. The first description seems to be a fonetic one. But apart from replacing it, I would say that there is some encoding issue too. I already have this entry: | nl | language | nl | Nederlands | 25 | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 --- Comment #8 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Should we create a just-in-case database update here in case there are more old installations with the wrong description? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30150 --- Comment #9 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- (In reply to Katrin Fischer from comment #8)
Should we create a just-in-case database update here in case there are more old installations with the wrong description?
Good plan -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org