December 2022 - Koha-devel - lists.koha-community.org

Jenkins notifications
by Tomas Cohen Arazi 15 Dec '22

15 Dec '22

Hi all, I've (finally) restored jenkins notifications, and I plan to send failure notifications to: - Release maintainers - QA team Developers whose commits are suspected to have broken the tests will still be notified. I created two forwarding addresses to simplify notifications as well. -- Tomás Cohen Arazi Theke Solutions (http://theke.io) ✆ +54 9351 3513384 GPG: B2F3C15F

2 1

Call for news - Newsletter December 2022
by Koha Newsletter 13 Dec '22

13 Dec '22

Hi I'm collecting news for the December 2022 Koha Community Newsletter. Please send anything noteworthy to: kohanews (at) gmail (dot) com News criteria: * News items can be of any length. * Images are fine. * Anything and everything Koha. * Submit by the 26th of the month. Text format criteria: * Just structured plain text, or * HTML text to include tables or similar For events: * Consider adding your event to the Koha Community calendar at https://koha-community.org/calendar/ Thank you! Michael Kuhn Editor, Koha Community Newsletter

1 0

What's the definitive way to use Perl tidy in koha-testing-docker?
by David Cook 11 Dec '22

11 Dec '22

Hi all, I've heard a few people saying lately that they wish people would use Perl tidy more. So I used the ":Tidy" command in koha-testing-docker on my visual selection and I got the following: + $this_item->{shelves} = + Koha::Virtualshelves->get_shelves_containing_record( + { + biblionumber => $biblionumber, + borrowernumber => $loggedinuser, + } + ); But that goes against the Indentation coding guidelines from https://wiki.koha-community.org/wiki/Coding_Guidelines#Indentation So. what are we supposed to be doing these days? David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

3 3

Formally give up on trying to provide PostgreSQL DB support?
by David Cook 11 Dec '22

11 Dec '22

Hi all, Should we formally give up on trying to provide PostgreSQL support? I find it highly unlikely that we will ever provide it. We may as well close issues like the following: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7365 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7802 https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11422 Over the last year or so, the Keycloak community has undergone a lot of conversations on database support. Historically, it's provided support for MariaDB, MSSQL, MySQL, Oracle, PostgreSQL, and more. However, going forward, Keycloak plans to only provide first class support for PostgreSQL and CockroachDB databases. CockroachDB uses the same wire protocol as PostgreSQL, so really they're supporting 1 database type. You can read more about it all at the following links: https://www.keycloak.org/2022/02/dbs.html https://www.keycloak.org/2022/07/storage-map.html Personally, I really like PostgreSQL. I use it for a number of other projects. But I think we're pretty wedded to MySQL/MariaDB with Koha, and maybe we just need to own that? David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

5 4

MySQL/MariaDB index prefix length
by David Cook 09 Dec '22

09 Dec '22

Hi all, We often create indexes with a prefix length of 191 because we use 4 byte encoded fields. 191 * 4 = 764, which comes in under the prefix index length threshold of 767. But that looks like a limit that was imposed due to the COMPACT row format. Most newer MySQL/MariaDB databases will be using the DYNAMIC row format, which can support an index key prefix limit of 3072 bytes (or 768 4-byte encoded characters). On Bugzilla, there have been times where I've speculated that maybe it's the sysadmin's job to update the database row format, but perhaps we should take a more proactive effort to move row format from COMPACT to DYNAMIC, so that we can more systematically take advantage of longer index prefixes. I often think about how to index URLs, since the maximum length of URLs are very large. Web browsers hover around the 2000 character mark while web servers can be much higher than that. Of course, even with 3072 bytes, that still wouldn't be enough for a full URL. But it would get a person a lot farther than 191 characters. David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

1 0

Error - browsing Patrons by Last Name
by Wainui Witika-Park 07 Dec '22

07 Dec '22

Hi everyone, I was wondering if anyone can help me please, I am getting an error when Browsing Patrons by Last Name. When clicking on some letters, the search gets stuck on "Processing". The letters that it seems to get stuck on are: A, H, N, T, V and W It also gives a pop up error when clicking on S that sometimes says "Something went wrong when loading the table. 500: Internal Server Error" and sometimes says "Something went wrong. The note has not been saved". The plack-error.log says: [2022-12-06 14:47:41.21998] [19070] [warn] OpenAPI >>> GET api/v1/patrons [{"message":"Expected string - got null.","path":"\/body\/4\/extended_attributes\/0\/value"}] Any help would very much appreciated :) Wainui

2 1

Good enough?
by Jonathan Druart 06 Dec '22

06 Dec '22

Hi devs, I was wondering... How good is your "good enough"? It's a question I often ask myself, when writing patches or QAing yours: is it good enough to be into Koha? It does not have to be perfect or it may never reach master, but it must meet certain standards. When I was RM I tried to put that limit quite high. Not necessarily by asking the author for improving the follow-up patches, but also by adding the missing bits myself. There are various types of "good enough", depending on what we look at: good enough to be reviewed, good enough to be tested, to be put in production, get feedback, try an idea, etc. Most of the time, what is driving the limit is answering the questions "Is it maintainable?" / "Is it future proof?". Making sure the code you are writing won't be broken inadvertently is very important and must be part of the reflection. Katrin asked the QA team members what were our plans for 23.05. In my opinion we should enforce this "be future proof" concept. Writing code is easy, especially in Koha (yes it is!). Writing maintainable and robust code, following our guidelines is a bit harder. That's why we have a QA process that tries to catch inconsistencies or edge cases you may have missed. But I think we can be even more rigorous, and aim for better standards. We can elevate our ambitions and do better. When we see the number of new enhancements we are releasing every 6 months, it shows well that writing code is definitely not a problem. However sometimes developers are tempted to abandon their work once they are pushed to master. Pushed does not mean "done". Providing bug fixes, following-up with patches when needed, fixing CI jenkins, etc. is part of job (/fun) As a Koha developer for a long time now, I know how frustrating it can be to be asked for follow-ups/rewrite/tests to have our patches stamped with the precious PQA mark. But from the other point of view (RM, RMaints, QA team), I also know it's very frustrating when you are in charge of the release and you do not get the appropriate follow-up work once it's pushed to master. There are some easy steps to write/review better patches. All have been discussed already several times, but that can be enforced even more: 1. Perltidy (!) This is really a very trivial step. Please perltidy your code. There are hundreds of commits that have been pushed in the last months that are not tidy (alignment, indentation, lines too long, etc.) This can easily be configured in your IDE! [1] 2. Provide clean code. As said it's not necessarily easy, but the QA team and RM are supposed to know if the code is clean regarding Koha guidelines. If the code is not clean, don't PQA, don't push. Either clean yourself, or ask the original author of the patch to do it (explaining to them how it can be improved ofc). 3. Squash! I have been away for a couple of months and had to read the git history to know what I missed. And it was really hard to follow what was going on. First of all, we are not consistent: the commit message must tell what the patch is doing, not what the bug was (if you are writing a bug fix). Then, there are way too many follow-ups: tidiness, indentation fix, typo, spelling, etc. All those tiny follow-ups could be squashed into the original patch. We don't need unnecessary tons of entries in our git log for that. For instance, I usually add a "JD Amended patch: perltidy" for instance when I tidy the original patch, to keep track of the modification. Squash can be done by the original author, the QAer, the RM. So yes, you are losing one commit in the stats but the git log is easy to read! We could have an "Amended-by" marker if we really want to add credit on the dashboard (and/or release notes). 4. Run tests. Don't wait for Jenkins to fail. This is valid for the author and QA. Anticipate the failures by running more tests. If you are modifying C4::Circulation, then run prove on t/db_dependent/Circulation*, not only Circulation.t. It will help you catch edge cases. When something is pushed, track down jenkins failures that could be caused by your patches. 6. Be strict if you are QAing. Each QA member has their own "good enough", and the RM as well (either relying on the QAer or providing a full review). But QA must fail if the code is old Koha style code, or not "good enough". 7. Provide support for failing tests, fix things you broke. The QA team will be more comfortable with your patches if you show them you are providing support for your stuff. It's not because it's pushed that you don't have any more efforts to make. Provide follow-up patches you promised, provide bug fixes, etc. We don't have a good way to keep track of such demands, which does not make tracking easier for devs, QA and RM. Any suggestions? 8. QA team MUST NEVER* pass QA a change that is not covered by tests, never. You should not provide change to modules without tests! * almost never... 9. Stick to existing patterns. We should not have different ways to do the same thing. We should not have different places where a code is doing the same thing. Ask for help or advice on the list or IRC before you start coding. We will be happy to guide you. Even if you are a regular Koha developer it's not always easy to be aware of the latest master changes. We will tell you what's the current good practice, or point you to examples you could reuse for what you want to implement. 10. CI should drive the pushes. No more push if CI is not green. The more we wait the harder it is to track down the origin of the problem. Last cycle some jobs have been red for months, and we released 22.11.00 with D10, D11, D12 marked unstable... What will I do next cycle? All of that, and more. I will track down jenkins failures and responsibilize developers telling them when they break tests (and won't fix them anymore as I have been doing for years). I will raise on the bug reports what could have been improved. Yes, read that I will be even more annoying (to put it politely) than before. I've noticed that the pre-commit git hook on the wiki has been broken for more than 3 years. And also caught some core developers that do not have it in place. I am relying on it to keep Vue files tidy so it's important to have it set up properly. I am planning to force its usage for ktd users [2]. Adding more checks to it will help us to catch inconsistencies from the beginning. To summarize, writing code is cheap, maintaining code is way more expensive! It is easier to get the attention of developers before the patches are pushed to master than after, so we could be more ambitious and ask more. For discussion :) Cheers, Jonathan [1] If you are using vim, open ~/vimrc, add vmap <F8> :!perltidy -q<CR> Reload vim, select code in visual mode [2] https://gitlab.com/koha-community/koha-misc4dev/-/issues/59

5 13

Maximum execution time for background jobs?
by David Cook 06 Dec '22

06 Dec '22

Hi all, I was closing old Bugzilla bugs when I bumped into "Bug 10460 - way to stop a run away report". It got me thinking about maximum execution time for background jobs. What do people think about background jobs workers killing their child processes if they exceed a maximum execution time? I don't have a particular use case for it at the moment, so not a priority by any means, but thoughts on the concept? David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

1 0

Fundamental flaw in Koha REST API
by David Cook 06 Dec '22

06 Dec '22

Hi all, Just wondering if my email titled “x-koha-query header doesn't support UTF-8 data” actually made it to the koha-devel list as I didn’t get any replies to what I perceive as a significant issue. Basically, the x-koha-query header can only support ISO 8859-1/Latin-1 encoded characters. If you try to send UTF-8 encoded data to the API via Javascript, it will silently fail. See Bug 32401 for a practical example of this. I encountered this issue when using Arabic, but it is also obvious when using Hebrew, Chinese, Devanagari, or any non-Latin based script. It is less obvious – but still a problem – for some languages like Dutch, Finnish, French, and German. For instance, a x-koha-query header containing the French word “cœur” will fail, because œ is not a ISO 8859-1/Latin-1 character. (See the Wikipedia entry for ISO 8859-1 for more examples of problems with languages mostly covered by Latin-1.) Koha devs need to discuss this and possibly remove “x-koha-query” from Koha. At the moment, it’s not widely used by Koha itself, so I don’t think it will be hard to remove from Koha, but any third-party integrations would need to refactor to use a different option. Anyway, it would be great if someone could acknowledge receipt of this email. I have figured out how to get DataTables to stop using x-koha-query, so I’m going to do that for the particular acquisitions problem mentioned in Bug 32401, but we do need to talk about this issue. David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595

3 4

Setting up Koha Testing Docker
by Lamancusa, Emily 05 Dec '22

05 Dec '22

Hello all, I'm trying to set up a Koha Testing Docker on my local machine (running Pop!_OS 22.04). I haven't used Docker before, and I have a few questions about how to get it set up. 1. It looks like Linux users have the option to install the whole Docker Desktop VM, or install just Docker Engine, Docker CLI, and the Compose plugin. Do I need Docker Desktop to run KTD, or will the latter installation option work on its own? (Aside from not wanting to install a bunch of extra stuff that I don't need, I would also need to request that my library system purchase a license for Docker Desktop, so I want to check if that is actually necessary.) 2. Is the idea to do all of the code editing, git commands, etc, from inside the container shell? Or would all of that be done outside the container, and the container be used only to run code/tests? 3. From what I've read in the wiki/getting started resources, I need Docker, the KTD itself, git-bz, and the git hooks from https://wiki.koha-community.org/wiki/Tips_and_tricks#git_tips_and_tricks (other than the pre-commit hook that is currently broken?) And then there's a tutorial on creating and submitting patches here (https://gitlab.com/koha-community/koha-howto/blob/master/README.md) that I can run once I have the KTD working. Am I missing anything? Hopefully this will make more sense once I have everything installed and can actually start playing with it, but I'll admit I'm feeling a little overwhelmed right now. Thank you so much to anyone who can give this newbie a little extra guidance on how to get started! Emily Lamancusa (she/her) IT Specialist III Montgomery County Public Libraries 21 Maryland Ave, Suite 310 Rockville, MD 20850 (240) 777-0676 [https://www.montgomerycountymd.gov/mcg/Resources/Images/Cybersecurity-foote…] For more helpful Cybersecurity Resources, visit: https://www.montgomerycountymd.gov/cybersecurity

3 2