[Bug 24712] New: Use YAML files for some of the update data
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Bug ID: 24712 Summary: Use YAML files for some of the update data Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: I18N/L10N Assignee: koha-bugs@lists.koha-community.org Reporter: martin.renvoize@ptfs-europe.com QA Contact: testopia@bugs.koha-community.org CC: frederic@tamil.fr Depends on: 13897, 24262 Bug 13897 with bug 24262 refine our translation process for install files converting the old SQL files used in the installer for mandatory and optional data into YAML files and adding processing to allow centralised translation of those files alongside the rest of the translation project. It raises the question, however, of how we should deal with the same data during upgrades. Currently we add the American English values with standard SQL in an atomicupdate file (copied into updatedatabase at push time). I'm envisaging we may be able to do away with the atomicupdates for this type of data insertion during upgrade and replace it with a step inside updatedatabase that pulls the data, translations and all from the yaml files. Opening this bug to discus and attempt such an implementation. Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=13897 [Bug 13897] Use YAML files for installer data https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24262 [Bug 24262] Translate installer data in YAML format -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bgkriegel@gmail.com, | |jonathan.druart@bugs.koha-c | |ommunity.org, | |julian.maurice@biblibre.com | |, katrin.fischer@bsz-bw.de, | |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |rel_20_05_target, | |RM_priority -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|rel_20_05_candidate |rel_20_11_target -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|rel_20_11_target, | |RM_priority | --- Comment #1 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- That would require quite a lot of work (at least non trivial) IMO. I have checked the different entries for 20.11 and I don't think we would benefit much from having such features. I am removing the "candidate" tag but feel free to add it back if you are willing to work on it! -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #2 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- This would be especially useful for new notices for example. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution|--- |DUPLICATE CC| |caroline.cyr-la-rose@inlibr | |o.com --- Comment #3 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- *** This bug has been marked as a duplicate of bug 34994 *** -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|DUPLICATE |--- Status|RESOLVED |REOPENED --- Comment #4 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- Oops sorry, I wanted to do the opposite… -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|REOPENED |NEW -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #5 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- *** Bug 34994 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #6 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- *** Bug 36737 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Koha collecto <koha@collecto.ca> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |koha@collecto.ca -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Marion Durand <marion.durand@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |marion.durand@biblibre.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |41979 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=41979 [Bug 41979] Use translation in database updates for authorised_values -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |In Discussion Assignee|koha-bugs@lists.koha-commun |jonathan.druart@gmail.com |ity.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #7 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 194999 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=194999&action=edit Bug 24712: Maintenance script to load installer data When new strings are translated in the yml files of the installer (auth values, notice templates, etc.) we don't update the database. This patch adds a maintenance script to add new entries, or update the existing rows. The script misc/maintenance/load_installer_data.pl takes 4 parameters: --file: to provide a yml file to load --table and --lang to define a file to load (eg. authorised_values and fr-FR) --force to replace the existing rows Without --force only new entries will be added Examples: perl misc/maintenance/load_installer_data.pl --table letter --lang es-ES perl misc/maintenance/load_installer_data.pl --table sample_notices --lang es-ES perl misc/maintenance/load_installer_data.pl --table letter --lang es-ES --force You can also use authorised_values, account_credit_types and account_debit_types -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #8 from Jonathan Druart <jonathan.druart@gmail.com> --- Patch for discussion. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #9 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- I tested this a little bit. Here are my comments so far : - If you don't use --force, it doesn't do anything - Check TranslateNotices syspref, set language to 'default' if syspref is off - Make help a bit more helpful, with all the options described and examples like the one in Comment 7 - For the --table option, maybe split it in --file (for the file name) and --table (for the table name) ? This is just a suggestion to alleviate the confusion :) - Add --letter_code flag (repeatable) to specify which letter code to update NOTE : My TranslateNotices is OFF. Most of my clients don't use it and only communicate with their patrons in French, so there is no need to have the notices in many languages, only French. Some of my clients have TranslateNotices ON, but I haven't tested this scenario yet. Test plan for KTD 1. Start KTD ktd up 2. Install a language (fr-CA in the example) ktd --shell gulp po:update --lang fr-CA ./misc/translator/translate install fr-CA 3. Drop the database and restart (this step is necessary to have KTD use the installer files in the other language) koha-mysql kohadev drop database koha_kohadev; create database koha_kohadev; Ctrl+D (or exit?) restart_all 4. Go to the staff interface (http://localhost:8081) and do the Web installer steps **making sure to choose the other language in the first step** (this step is necessary to have KTD use the installer files in the other language) (the username is koha_kohadev and pw is password) * Note the username and password you set for the admin user... it will probably be different from the usual 5. Copy the test24712.pl file into installer/data/mysql/atomicupdate/ cp ~/Documents/test24712.pl installer/data/mysql/atomicupdate/ 6. Copy the updated sample_notices.yaml file in in installer/data/mysql/en/mandatory/ cp ~/Documents/sample_notices.yml installer/data/mysql/en/mandatory/ 7. Run updatedatabase.pl ./installer/data/mysql/updatedatabase.pl --> This should insert a letter in English 8. Log in to the staff interface (with credentials set in step 4) 9. Set the StaffInterfaceLanguages and OPACLanguages sysprefs to the other language, put it in first position in both prefs 10. Go to Tools > Notices and slips --> There should be a new letter "TEST_24712" in English among the letters in French (Current problem!) 11. Edit one of the other letters (like PREDUE) with custom text 12. Update the translation strings ktd --shell gulp po:update --lang fr-CA 13. View the strings in fr-CA-installer.po and translate them - English name 24712 - English title 24712 - This is an English message sent to patrons 14. Install the new translations ./misc/translator/translate install fr-CA --> The installer/data/mysql/fr-CA/mandatory/sample_notices.yml file should have the translated strings 15. In the staff interface, you should be able to see the default text in French (this is great, but we don't want to have to do this manually for each notice) 16. Apply patch 17. Run the script ./misc/maintenance/load_installer_data.pl --table letter --lang fr-CA --> This should only translate the new letter (TEST_24712) and not touch the modified letter (PREDUE) -- This does not work currently -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #10 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- Created attachment 196441 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=196441&action=edit Atomic update to insert new letter in English Atomic update file to insert new letter in English -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #11 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- Created attachment 196442 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=196442&action=edit YAML file with new letter YAML file with new letter -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #12 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- Still to test : anuthorised values -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Jonathan Druart <jonathan.druart@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #194999|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #13 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 196996 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=196996&action=edit Bug 24712: Maintenance script to load installer data When new strings are translated in the yml files of the installer (auth values, notice templates, etc.) we don't update the database. This patch adds a maintenance script to add new entries, or update the existing rows. See the help output of the script. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #14 from Jonathan Druart <jonathan.druart@gmail.com> --- Caroline, I had another look and I think I have found a way to answer your need. It gets dirty, but it works. Can you please retest? Note that: * There is now a --verbose flag to understand more what the script is doing * I only focused on the notice templates for now. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #15 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- Hi Jonathan! Thanks for looking at this again. Did you mean to commit the modified sample_notices.yaml file in your patch? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #16 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- I finished the test plan for letters and it works (very excited about this!!!) [1] Processing members/TEST_24712/email * Already exists * Checking if needs to be replaced... * Yes, replacing! * Replaced successfully! although the verbose always says that it's different from the default when it's not. I don't think its critical, but it is misleading. In my test plan I only changed PREDUE, so this is correct : [35] Processing circulation/PREDUE/email * Already exists * Checking if needs to be replaced... * The content is different than the default one, skipping I did not change ODUE, so this is not quite true even though I still want it to skip : [33] Processing circulation/ODUE/email * Already exists * Checking if needs to be replaced... * The content is different than the default one, skipping -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #17 from Jonathan Druart <jonathan.druart@gmail.com> --- Created attachment 198988 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=198988&action=edit Bug 24712: Disambiguate 'identical' and 'different than the default one' In the verbose output we want to make the difference between contents that don't need to be adjusted. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 --- Comment #18 from Jonathan Druart <jonathan.druart@gmail.com> --- (In reply to Caroline Cyr La Rose from comment #16)
In my test plan I only changed PREDUE, so this is correct :
[35] Processing circulation/PREDUE/email * Already exists * Checking if needs to be replaced... * The content is different than the default one, skipping
I did not change ODUE, so this is not quite true even though I still want it to skip :
[33] Processing circulation/ODUE/email * Already exists * Checking if needs to be replaced... * The content is different than the default one, skipping
This last patch takes care of that -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Martin Renvoize (ashimema) <martin.renvoize@openfifth.co.uk> changed: What |Removed |Added ---------------------------------------------------------------------------- Initiative type|--- |Feature Target Milestone|--- |26.11 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24712 Michaela Sieber <michaela.sieber@kit.edu> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |clemens.tubach@kit.edu, | |katrin.fischer@bsz-bw.de, | |michaela.sieber@kit.edu -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org