<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body>A new request with request id 8493 has been created by koha-devel-request@lists.koha-community.org. Short info on the request is : <br><br>Title : Koha-devel Digest, Vol 200, Issue 5<br>Category : <br>Description : <div>Send Koha-devel mailing list submissions to<br> koha-devel@lists.koha-community.org<br><br>To subscribe or unsubscribe via the World Wide Web, visit<br> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>or, via email, send a message with subject or body 'help' to<br> koha-devel-request@lists.koha-community.org<br><br>You can reach the person managing the list at<br> koha-devel-owner@lists.koha-community.org<br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of Koha-devel digest..."<br><br><br>Today's Topics:<br><br> 1. Re: How can I improve this report? (dcook@prosentient.com.au)<br> 2. Re: Multilingual label printing in Koha (praise be to<br> Bywater!) (dcook@prosentient.com.au)<br><br><br>----------------------------------------------------------------------<br><br>Message: 1<br>Date: Mon, 4 Jul 2022 10:07:27 +1000<br>From: <dcook@prosentient.com.au><br>To: "'Doug Kingston'" <dpk@randomnotes.org>, "'koha-devel'"<br> <koha-devel@lists.koha-community.org><br>Subject: Re: [Koha-devel] How can I improve this report?<br>Message-ID: <104e01d88f3a$12f4fc30$38def490$@prosentient.com.au><br>Content-Type: text/plain; charset="utf-8"<br><br>Hey Doug,<br><br> <br><br>In terms of analyzing queries, I find MySQL/MariaDB to be far more opaque than PostgreSQL, but in theory the “EXPLAIN” and “ANALYZE” keywords can help you out with this one[1]. <br><br> <br><br>Overall, I’d say it depends on what you’re trying to do and your data set. I have databases with hundreds of thousands of authority records, and for those I’d probably write a custom script to query Zebra since it’s already indexed the data. <br><br> <br><br>--<br><br>[1]<br><br>Using koha-testing-docker, here is the query you provided:<br><br> <br><br>MariaDB [koha_kohadev]> ANALYZE SELECT DISTINCT(author) AS heading FROM biblio WHERE author NOT IN (SELECT ExtractValue(marcxml,'//datafield[@tag="100"]/subfield[@code="a"]') AS heading FROM auth_header WHERE authtypecode='PERSO_NAME') O<br><br>RDER BY heading;<br><br>+------+--------------------+-------------+------+---------------+------+---------+------+------+--------+----------+------------+----------------------------------------------+<br><br>| id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra |<br><br>+------+--------------------+-------------+------+---------------+------+---------+------+------+--------+----------+------------+----------------------------------------------+<br><br>| 1 | PRIMARY | biblio | ALL | NULL | NULL | NULL | NULL | 436 | 436.00 | 100.00 | 16.28 | Using where; Using temporary; Using filesort |<br><br>| 2 | DEPENDENT SUBQUERY | auth_header | ALL | NULL | NULL | NULL | NULL | 1560 | 653.58 | 100.00 | 0.13 | Using where |<br><br>+------+--------------------+-------------+------+---------------+------+---------+------+------+--------+----------+------------+----------------------------------------------+<br><br>2 rows in set (1.380 sec)<br><br> <br><br>Here is another one I cooked up which apparently is marginally faster:<br><br> <br><br>MariaDB [koha_kohadev]> ANALYZE SELECT DISTINCT(biblio.author) AS heading, derived.ah FROM biblio LEFT JOIN (SELECT ExtractValue(marcxml,'//datafield[@tag="100"]/subfield[@code="a"]') as ah FROM auth_header WHERE authtypecode='PERSO_NAM<br><br>E') AS derived ON biblio.author = derived.ah WHERE derived.ah is null and biblio.author is not null ORDER BY biblio.author;<br><br>+------+-------------+-------------+------+---------------+------+---------+------+------+---------+----------+------------+-------------------------------------------------+<br><br>| id | select_type | table | type | possible_keys | key | key_len | ref | rows | r_rows | filtered | r_filtered | Extra |<br><br>+------+-------------+-------------+------+---------------+------+---------+------+------+---------+----------+------------+-------------------------------------------------+<br><br>| 1 | SIMPLE | biblio | ALL | NULL | NULL | NULL | NULL | 436 | 436.00 | 100.00 | 75.00 | Using where; Using temporary; Using filesort |<br><br>| 1 | SIMPLE | auth_header | ALL | NULL | NULL | NULL | NULL | 1560 | 1706.00 | 100.00 | 0.06 | Using where; Using join buffer (flat, BNL join) |<br><br>+------+-------------+-------------+------+---------------+------+---------+------+------+---------+----------+------------+-------------------------------------------------+<br><br>2 rows in set (1.145 sec)<br><br> <br><br> <br><br> <br><br>David Cook<br><br>Senior Software Engineer<br><br>Prosentient Systems<br><br>Suite 7.03<br><br>6a Glen St<br><br>Milsons Point NSW 2061<br><br>Australia<br><br> <br><br>Office: 02 9212 0899<br><br>Online: 02 8005 0595<br><br> <br><br>From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of Doug Kingston<br>Sent: Saturday, 2 July 2022 5:39 AM<br>To: koha-devel <koha-devel@lists.koha-community.org><br>Subject: [Koha-devel] How can I improve this report?<br><br> <br><br>Report: Authors not in Authorities<br><br> <br><br>SELECT DISTINCT(author) AS heading FROM biblio WHERE author NOT IN (SELECT ExtractValue(marcxml,'//datafield[@tag="100"]/subfield[@code="a"]') AS heading FROM auth_header WHERE authtypecode='PERSO_NAME')ORDER BY heading<br><br> <br><br>I am not a SQL expert, but I believe this is doing an unkeyed troll through every authority record for every biblio (O(Nbibs * Mauths), which is pretty bad). I suspect we can do better by trolling Auths once to create a set and then compare Bib authors against it. How can we do this?<br><br> <br><br>-Doug-<br><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20220704/20381820/attachment-0001.htm><br><br>------------------------------<br><br>Message: 2<br>Date: Mon, 4 Jul 2022 13:02:03 +1000<br>From: <dcook@prosentient.com.au><br>To: "'Kyle Hall'" <kyle@bywatersolutions.com><br>Cc: "'Indranil Das Gupta'" <indradg@l2c2.co.in>, "'Koha-devel'"<br> <koha-devel@lists.koha-community.org><br>Subject: Re: [Koha-devel] Multilingual label printing in Koha (praise<br> be to Bywater!)<br>Message-ID: <000c01d88f52$775fb1c0$661f1540$@prosentient.com.au><br>Content-Type: text/plain; charset="utf-8"<br><br>Thanks, Kyle!<br><br> <br><br>Yeah, it looks like Noto is Open Font and WenQuanYi is GPL. <br><br> <br><br>My understanding with the OFL is that you can embed the fonts in software you sell, and you can share fonts but you can’t sell them standalone as fonts. <br><br> <br><br>So yeah I probably could share the custom font file. It could be interesting to set up a Koha Community font repository. <br><br> <br><br>I was thinking of posting something to the general Koha listserv on this topic as well… <br><br> <br><br>David Cook<br><br>Senior Software Engineer<br><br>Prosentient Systems<br><br>Suite 7.03<br><br>6a Glen St<br><br>Milsons Point NSW 2061<br><br>Australia<br><br> <br><br>Office: 02 9212 0899<br><br>Online: 02 8005 0595<br><br> <br><br>From: Kyle Hall <kyle@bywatersolutions.com> <br>Sent: Friday, 1 July 2022 8:23 PM<br>To: dcook@prosentient.com.au<br>Cc: Indranil Das Gupta <indradg@l2c2.co.in>; Koha-devel <koha-devel@lists.koha-community.org><br>Subject: Re: [Koha-devel] Multilingual label printing in Koha (praise be to Bywater!)<br><br> <br><br>Dang! Nice work! I wonder if the licenses allow you to publish your custom font file for others to use. It looks like one is Open Font and the other is GPL?<br><br> <br><br>On Fri, Jul 1, 2022 at 1:17 AM <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote:<br><br>While this has consumed more of my day than I would’ve anticipated, it’s been a very useful exercise!<br><br> <br><br>I downloaded NotoSansTC.otf from Google (although you can just install it from Debian/Ubuntu too although they supply .ttc files), and opened it in FontForge. I used “CID > Flatten”, “Encoding > Reencode > UnicodeBmp”, and then I noticed that the “space” and “hyphen” characters were missing, so I copied them in from the WenQuanYi font. I used “File > Generate Fonts” and chose the “TrueType” export format, which exports as a “.ttf” file. <br><br> <br><br>I loaded that into Koha via the “ttf” section of koha-conf.xml, and now I have Noto fonts rendering English and Chinese via the Koha Label Creator.<br><br> <br><br>It’s taken a few hours to learn the basics of FontForge and font files, but now I feel fairly comfortable. I am sure that many other Koha libraries could leverage the same tool and processes to get TTF font files that are able to meet their multilingual needs. <br><br> <br><br>Browsing through the font file, I see a fair number of European symbols from French, Spanish, and Norwegian in the font file.<br><br> <br><br>Anyway… get in touch if you want more information about working with font files I guess! These past 48 hours have been very illuminating! <br><br> <br><br>David Cook<br><br>Senior Software Engineer<br><br>Prosentient Systems<br><br>Suite 7.03<br><br>6a Glen St<br><br>Milsons Point NSW 2061<br><br>Australia<br><br> <br><br>Office: 02 9212 0899<br><br>Online: 02 8005 0595<br><br> <br><br>From: dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > <br>Sent: Friday, 1 July 2022 1:32 PM<br>To: 'Kyle Hall' <kyle@bywatersolutions.com <mailto:kyle@bywatersolutions.com> >; 'Indranil Das Gupta' <indradg@l2c2.co.in <mailto:indradg@l2c2.co.in> ><br>Cc: 'Koha-devel' <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> >; 'Brendan Gallagher' <brendan@bywatersolutions.com <mailto:brendan@bywatersolutions.com> ><br>Subject: RE: [Koha-devel] Multilingual label printing in Koha (praise be to Bywater!)<br><br> <br><br>Kyle:<br><br> <br><br>TIL a new word! I don’t think that I’d ever heard “yinz” before. Thanks for keeping life interesting, Kyle!<br><br> <br><br>Once you start going down that path of trying to turn HTML and CSS into a PDF programmatically, I think that you need to start using a proper rendering engine. I’ve used “wkhtmltopdf” (Qt Webkit engine), Puppeteer, Playwright, and headless Chrome.<br><br> <br><br>There are a few CPAN modules for working with “wkhtmltopdf” although it looks like they all wrap the binary rather than being Perl bindings for the C library but that’s probably OK. <br><br>https://metacpan.org/pod/WKHTMLTOPDF<br><br>https://metacpan.org/pod/PDF::WebKit<br><br> <br><br>Looks like it’s also available in Debian and Ubuntu: <br><br>https://packages.debian.org/search?keywords=wkhtmltopdf<br><br>https://packages.ubuntu.com/bionic/wkhtmltopdf. <br><br> <br><br>--<br><br> <br><br>Indranil: <br><br> <br><br>I think that you may be right about PDF::Reuse (and friends) not being up to the task overall…<br><br> <br><br>However, I discovered that a number of Chinese fonts like Noto CJK fonts (e.g. Noto Sans TC) and WenQuanYi (https://packages.ubuntu.com/bionic/fonts-wqy-zenhei) actually contain glyphs for both English and Chinese. Using “fontforge”, I was able to create a TTF file from the WenQuanYi TTC file, which can be used by Koha’s Label Creator to print both English and Chinese on the same label using the same font. I just referenced my new TTF via koha-conf.xml and voila!<br><br> <br><br>(I did notice that if I copy the text from the PDF and paste it elsewhere, it will just show gibberish, but apparently that’s a pre-existing problem with the PDF generator…) <br><br> <br><br>Unfortunately, I wasn’t able to get a good quality TTF from the Noto Sans TC. It worked well on the Chinese and English but it couldn’t display space characters, so I couldn’t use that in the end. <br><br> <br><br>I find this whole font business very interesting. Personally, I’d love to know how browsers and word processors decide which font to apply. I keep thinking that they must have maps based off of Unicode code points and then choose an appropriate font based on that. But I don’t know for sure. <br><br> <br><br>In theory, PDF::Reuse could be changed to use “fontconfig” to see what font is available on the OS for a particular Unicode codepoint, so we could write a simple algorithm combined with a hash map to provide better font support for Koha’s Label Creator… but yeah it would take some time and energy. While I find it interesting, I’m not sure that I find it interesting enough to devote my free time to it. <br><br> <br><br>I guess we’ll see how long this tickles my brain.<br><br> <br><br>David Cook<br><br>Senior Software Engineer<br><br>Prosentient Systems<br><br>Suite 7.03<br><br>6a Glen St<br><br>Milsons Point NSW 2061<br><br>Australia<br><br> <br><br>Office: 02 9212 0899<br><br>Online: 02 8005 0595<br><br> <br><br>From: Kyle Hall <kyle@bywatersolutions.com <mailto:kyle@bywatersolutions.com> > <br>Sent: Thursday, 30 June 2022 9:27 PM<br>To: Indranil Das Gupta <indradg@l2c2.co.in <mailto:indradg@l2c2.co.in> ><br>Cc: dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> ; Koha-devel <koha-devel@lists.koha-community.org <mailto:koha-devel@lists.koha-community.org> >; Brendan Gallagher <brendan@bywatersolutions.com <mailto:brendan@bywatersolutions.com> ><br>Subject: Re: [Koha-devel] Multilingual label printing in Koha (praise be to Bywater!)<br><br> <br><br>That's awesome! Thanks Yinz!<br><br> <br><br>I think the unicorn we need is something like https://metacpan.org/pod/PDF::FromHTML but with support for CSS. AFAIK such a thing doesn't exist :/<br><br> <br><br>Kyle<br><br> <br><br>On Thu, Jun 30, 2022 at 5:48 AM Indranil Das Gupta <indradg@l2c2.co.in <mailto:indradg@l2c2.co.in> > wrote:<br><br>Yep! the Bywater Solutions plugin by Kyle does the job nicely "outside the box" :-) <br><br> <br><br>IIRC the label printing code was last worked on by Chris N a long long time back. As far as I can tell the non Latin printing issue is due to the PDF library we are using. Prolly needs a rewrite for that part which requires both time and funds :-( <br><br> <br><br>-Indranil <br><br> <br><br> <br><br> <br><br>On Thu, 30 Jun, 2022, 12:34 pm , <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote:<br><br>Hi all,<br><br>I had a problem where I couldn't print Chinese and English on barcode labels<br>using Koha's built-in Label Creator, since it uses individual font files and<br>I couldn't find a free and open individual font that supports both European<br>and Chinese scripts.<br><br>Fortunately, I recalled that Kyle @ ByWater Solutions created this plugin:<br>https://github.com/bywatersolutions/koha-plugin-label-maker. It uses HTML<br>and CSS to generate labels in the browser, and the browser is already<br>capable of using multiple different fonts to support multiple different<br>scripts. The plugin did a great job of rendering barcodes with both Chinese<br>and English written on them! Thanks, Bywater Solutions, for saving the day!<br><br>--<br>Nerdy thoughts:<br><br>If I look at the "Computed" styles in the F12 tools in Chrome on Windows 10,<br>it says the English is rendered using the "Times New Roman" font, and it<br>says that the Chinese is rendered using the "Microsoft YaHei" font. If I do<br>the same thing in Microsoft Word, the English is in "Calibri" and the<br>Chinese is in "DengXian". If you read<br>https://fonts.google.com/noto/use#how-are-noto-fonts-organized it says "If<br>you have text in multiple languages that use different scripts, either the<br>app or the person who typesets the text needs to switch between the<br>different fonts accordingly."<br><br>I'm not an expert on fonts, so I'm not sure what would be involved with<br>getting Koha's PDF libraries to use different fonts for different scripts. I<br>don't know how Chrome and Word detect that they need to use a different<br>font. I don't know if it's a map of Unicode codepoints to existing font<br>mappings, or if it iterates through a list of fallbacks, or what. <br><br>I probably won't go too far down this rabbit hole, since the ByWater Koha<br>Plugin solved my immediate use case, but it's an interesting topic, and this<br>is a major limitation with Koha's Label Creator, so we might want to think<br>about a change in the future...<br><br>David Cook<br>Senior Software Engineer<br>Prosentient Systems<br>Suite 7.03<br>6a Glen St<br>Milsons Point NSW 2061<br>Australia<br><br>Office: 02 9212 0899<br>Online: 02 8005 0595<br><br><br>_______________________________________________<br>Koha-devel mailing list<br>Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> <br>https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>website : https://www.koha-community.org/<br>git : https://git.koha-community.org/<br>bugs : https://bugs.koha-community.org/<br><br><br><br>-- <br><br>Kyle M. Hall<br><br>Loose Cannon, ByWater Solutions<br><br> <mailto:kyle@bywatersolutions.com> kyle@bywatersolutions.com<br> <http://bywatersolutions.com/> https://bywatersolutions.com<br><br> <http://bywatersolutions.com/what-is-koha/> What is Koha?<br><br><br><br>-- <br><br>Kyle M. Hall<br><br>Loose Cannon, ByWater Solutions<br><br> <mailto:kyle@bywatersolutions.com> kyle@bywatersolutions.com<br> <http://bywatersolutions.com/> https://bywatersolutions.com<br><br> <http://bywatersolutions.com/what-is-koha/> What is Koha?<br><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <http://lists.koha-community.org/pipermail/koha-devel/attachments/20220704/2ff06d5e/attachment.htm><br><br>------------------------------<br><br>Subject: Digest Footer<br><br>_______________________________________________<br>Koha-devel mailing list<br>Koha-devel@lists.koha-community.org<br>https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>website : https://www.koha-community.org/<br>git : https://git.koha-community.org/<br>bugs : https://bugs.koha-community.org/<br><br><br>------------------------------<br><br>End of Koha-devel Digest, Vol 200, Issue 5<br>******************************************<br></div><br><br>NOTE: You are receiving this mail because, the Requester/Technician wanted you to get notified on this request creation.<br></body></html>