[Bug 32609] New: Remove compiled files from src
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Bug ID: 32609 Summary: Remove compiled files from src Change sponsored?: --- Product: Koha Version: unspecified Hardware: All OS: All Status: ASSIGNED Severity: enhancement Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: jonathan.druart+koha@gmail.com Reporter: jonathan.druart+koha@gmail.com QA Contact: testopia@bugs.koha-community.org We need to improve our yarn build commands to generate css and js compiled files. Then remove them from our codebase, and generate them when needed: - when generating the debian packages - when ktd is starting up - when patches are applied on the sandbox -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=32473 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #1 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 145209 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=145209&action=edit Bug 32609: Remove JS vars from gulpfile They are not used. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #2 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 145210 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=145210&action=edit Bug 32609: Improve yarn build commands We had: yarn css # Build staff css yarn css --view opac # Build OPAC css yarn build # Nothing, was broken yarn watch # Nothing, was broken yarn build_js # Build JS/Vue yarn watch_js # Watch JS/Vue This patch is suggesting to have something more consistent: yarn css:build # Build CSS for OPAC and staff (dev) yarn css:build:prod # Build CSS for OPAC and staff (prod) yarn css:watch # Watch CSS for OPAC and staff (dev) yarn js:build # Build JS (dev) yarn js:build:prod # Build JS (prod) yarn js:watch # Watch JS (dev) yarn build # Same as css:build and js:build (dev) yarn build:prod # Same as css:build:prod and js:build:prod (prod) Note that the cssnano was breaking the build function. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #3 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 145211 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=145211&action=edit Bug 32609: Add compiled css files to gitignore -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #4 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 145212 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=145212&action=edit Bug 32609: Add warning to the about page -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #5 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- And the last patch didn't make it, too large. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |In Discussion CC| |martin.renvoize@ptfs-europe | |.com, mtj@kohaaloha.com, | |oleonard@myacpl.org, | |paul.derscheid@lmscloud.de, | |tomascohen@gmail.com --- Comment #6 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- I think this is going toward the good direction, what do you think? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- URL| |https://gitlab.com/joubu/Ko | |ha/-/commits/bug_32609 --- Comment #7 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- The full patch set is on a remote branch: https://gitlab.com/joubu/Koha/-/commits/bug_32609 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #8 from Tomás Cohen Arazi <tomascohen@gmail.com> --- +1 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #9 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- (In reply to Jonathan Druart from comment #0)
We need to improve our yarn build commands to generate css and js compiled files.
Then remove them from our codebase, and generate them when needed:
- when generating the debian packages - when ktd is starting up - when patches are applied on the sandbox
Hm, what about tarballs? We roll our own packages - what would be the suggested workflow? Generate the files on the git branch before building the packages? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #10 from Mason James <mtj@kohaaloha.com> --- (In reply to Katrin Fischer from comment #9)
(In reply to Jonathan Druart from comment #0)
We need to improve our yarn build commands to generate css and js compiled files.
Then remove them from our codebase, and generate them when needed:
- when generating the debian packages - when ktd is starting up - when patches are applied on the sandbox
Hm, what about tarballs?
We roll our own packages - what would be the suggested workflow? Generate the files on the git branch before building the packages?
i think we might be able to add these steps to Makefile.PL, so they get run for both tarball and deb pkg builds -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart+koha@gmail. | |com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #145209|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #145210|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #145211|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #145212|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #11 from David Cook <dcook@prosentient.com.au> --- (In reply to Mason James from comment #10)
(In reply to Katrin Fischer from comment #9)
Hm, what about tarballs?
We roll our own packages - what would be the suggested workflow? Generate the files on the git branch before building the packages?
i think we might be able to add these steps to Makefile.PL, so they get run for both tarball and deb pkg builds
Yeah I reckon we add something to PL_FILES: https://metacpan.org/pod/ExtUtils::MakeMaker#PL_FILES That way it would apply to tarballs, deb pkg builds, and technically even git dev installs. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #12 from David Cook <dcook@prosentient.com.au> --- (In reply to Jonathan Druart from comment #0)
We need to improve our yarn build commands to generate css and js compiled files.
Then remove them from our codebase, and generate them when needed:
- when generating the debian packages - when ktd is starting up - when patches are applied on the sandbox
I suppose "ktd" startup time makes sense. On one hand, I might like to generate the files 1 time, and re-use them over days/weeks/months, but... having a fresh build on each startup is probably wise. When I switch version branches, I'll destroy my container and rebuild it, so this would work well with that workflow. And if you change branches that have the same version but changes to css and js... you just need to re-compile anyway. So yeah that sounds good to me. -- If we create a Perl script to run these commands, we can include it in Makefile.PL, plus manually call it for "ktd" and sandboxes, so... that should be pretty straightforward overall. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #13 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- This is looking great.. we clearly need to continue down this route in my opinion. Let me know how I can help.. obviously it will require changes in surrounding projects like ktd, sandboxes, packaging pipeline.. we should be able to split the load at least a little. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Fridolin Somers <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #14 from Fridolin Somers <fridolin.somers@biblibre.com> --- Makes completely sense, I'm in :D -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #15 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to David Cook from comment #12)
If we create a Perl script to run these commands, we can include it in Makefile.PL, plus manually call it for "ktd" and sandboxes, so... that should be pretty straightforward overall.
Actually it's not obvious at all for me how this can be done easily. There are dev paths in gulpfile... We could rely no an env var (or an option?) that would pass the path. Like: css_path=/usr/share/koha/intranet/htdocs/intranet-tmpl/prog/css/ yarn css:build:prod I've done this already. For the js/vue dist file, it's a bit more tricky (node path, relative js path, modules, etc.) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #16 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- 2 new commits on the remote branch Bug 32609: Rely no env var to generate js files Bug 32609: Rely on env var to generate css files The CSS one is working, the JS is not. It's failing with: ERROR in ../../usr/share/koha/intranet/htdocs/intranet-tmpl/prog/js/vue/main-erm.ts 3:12-26 Module not found: Error: Can't resolve 'vue' in '/usr/share/koha/intranet/htdocs/intranet-tmpl/prog/js/vue' and many more. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 David Nind <david@davidnind.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |david@davidnind.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #17 from David Cook <dcook@prosentient.com.au> --- (In reply to Jonathan Druart from comment #15)
(In reply to David Cook from comment #12)
If we create a Perl script to run these commands, we can include it in Makefile.PL, plus manually call it for "ktd" and sandboxes, so... that should be pretty straightforward overall.
Actually it's not obvious at all for me how this can be done easily.
There are dev paths in gulpfile...
We could rely no an env var (or an option?) that would pass the path.
Like: css_path=/usr/share/koha/intranet/htdocs/intranet-tmpl/prog/css/ yarn css:build:prod
I've done this already. For the js/vue dist file, it's a bit more tricky (node path, relative js path, modules, etc.)
In my mind, you would only be running the build from the source directory and not a deployed directory, so the paths should always be those dev paths. We should just need to add a script to Koha for running the JS build, and then for ktd and sandboxes, we invoke it (targeting the git checkouts) via their respective toolchains. We add a call to that script in Makefile.PL as well to build the JS for single/standard/dev installs (and package installs which use the standard install implicitly). -- Of course, I might be missing something here. If you can tell me which commands we need to build the JS and CSS, I'm happy to give this a try... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #18 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to David Cook from comment #17)
If you can tell me which commands we need to build the JS and CSS, I'm happy to give this a try...
It's in commit message of the second commit: This patch is suggesting to have something more consistent: yarn css:build # Build CSS for OPAC and staff (dev) yarn css:build:prod # Build CSS for OPAC and staff (prod) yarn css:watch # Watch CSS for OPAC and staff (dev) yarn js:build # Build JS (dev) yarn js:build:prod # Build JS (prod) yarn js:watch # Watch JS (dev) yarn build # Same as css:build and js:build (dev) yarn build:prod # Same as css:build:prod and js:build:prod (prod) Here we will need `yarn build:prod` that will generate the minified css files and the dist file for the Vue app. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #19 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 145802 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=145802&action=edit Bug 32609: (follow-up) Add build-resources.PL This patch adds a trivial perl wrapper around the yarn build and calls it from Makefile.PL. I 'think' this is all we need to get packaging, dev and standard installs happy with the lack of our compiled files in source. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #20 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- My patch is on top of Jonathans branch (though with it I think we don't need his last two patches). I've not actually tested it in the end to end sense of building a package. Also, it assumes dependencies for yarn and friends are there.. but I think that's a fair assumption for a build box right? Let me know if we're thinking along the same lines David and please feel free to improve further. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #21 from David Cook <dcook@prosentient.com.au> --- (In reply to Martin Renvoize from comment #20)
Let me know if we're thinking along the same lines David and please feel free to improve further.
I'm thinking along the same lines although I don't think it'll work as expected at present. I think that I can get this to work though. Patch incoming... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #22 from David Cook <dcook@prosentient.com.au> --- If we're going to get a script file from PL_FILES to run "before" pm_to_blib, then we need to use a target that can be found in "PM". Since Jonathan removed the files, "perl Makefile.PL" creates a Makefile that doesn't include the CSS and JS targets in "PM", so they don't get copied into "blib" and don't wind up in the tarball (for tarball/package installs). If we're going to use Makefile.PL to include generated files, we just need to add them to the "PM" argument in Makefile.PL and to call build-resources.PL with an argument that can be found in "PM". (We don't want multiple arguments, or build-resources.PL will be run multiple times.) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #23 from David Cook <dcook@prosentient.com.au> --- Currently having this issue: root@kohadevbox:koha(bug_32609)$ make "/usr/bin/perl" build-resources.PL koha-tmpl/intranet-tmpl/prog/js/vue/dist/main.js yarn run v1.22.19 $ yarn css:build:prod && yarn js:build:prod $ gulp build && gulp css --view opac [00:35:54] Local modules not found in /kohadevbox/koha [00:35:54] Try running: yarn install error Command failed with exit code 1. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. error Command failed with exit code 1. info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #24 from David Cook <dcook@prosentient.com.au> --- According to https://wiki.koha-community.org/wiki/Working_with_SCSS_in_the_OPAC_and_staff..., I first need to run "yarn install" I wonder if that should be in build-resources.PL or not. I'll add it in and see how I go... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #25 from David Cook <dcook@prosentient.com.au> --- With my (yet to be attached) patch, I run the following: 1. perl Makefile.PL #NOTE: On the first run, 2. make 3. diff -q -r koha-tmpl/intranet-tmpl/ blib/INTRANET_TMPL_DIR/ #NOTE: No output 4. diff -q -r koha-tmpl/opac-tmpl blib/OPAC_TMPL_DIR/ #NOTE: No output That means that the CSS and JS files have been built and copied across to "blib". From there, they can be installed using a source install or built into a Debian package. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #26 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #25)
That means that the CSS and JS files have been built and copied across to "blib". From there, they can be installed using a source install or built into a Debian package.
I verified this by running "make install" in a "single" deployment. /opt/koha/intranet/htdocs/intranet-tmpl/prog/js/vue/dist/main.js /opt/koha/opac/htdocs/opac-tmpl/bootstrap/css/opac.css You could manually verify the rest. There's just a bunch of generated files to list. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #27 from David Cook <dcook@prosentient.com.au> --- However, I'm noticing a problem... at the moment I'm hooking off "./koha-tmpl/intranet-tmpl/prog/js/vue/dist/main.js". If that file exists in the source directory, then the yarn build won't run when running "make". That shouldn't be a problem for a tarball/package install... since the generated files won't exist... but it wouldn't work for a git dev install. However, as we noted before, we're not really catering to that. So a git dev install, ktd, and sandboxes would need to run "build-resources.PL" manually to update the JS and CSS. -- But I'm going to keep playing as I have a couple more ideas... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #28 from David Cook <dcook@prosentient.com.au> --- Created attachment 145838 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=145838&action=edit Bug 32609: (follow-up) enhance Makefile.PL This patch adds a new target to the Makefile which gets called as the first step of the "all" target. This means that CSS and JS files get compiled every time "make" is called. (Note: Since the "install" target depends on "all", it will be re-run at install time.) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #29 from David Cook <dcook@prosentient.com.au> --- Ok, so I've fixed it, so that the CSS and JS get compiled every time "make"[1] is called, so they'll always be updated for source/package installs (which use "make" and "make install") and git dev installs (if they use "make" and "make upgrade"). For "ktd" and sandboxes (and git dev installs that don't use "make"), just call "build-resources.PL" to compile the CSS and JS. -- My patch updates the Makefile so that the "all" target relies on a new "compile_css_js" target which runs build-resources.PL. That's fairly elegant. The only non-elegant part is needing to list all the compiled files in Makefile.PL so that they get correctly copied into the "blib" directory at build time. That code could be further streamlined to reduce boilerplate and the chance of human error, although ultimately we will need to provide a list of generated files at "perl Makefile.PL" time, or else the generated files won't get deployed. In theory, this list should not change often. -- [1] Note that someone many many years made the "install" target depend on "all", so "make install" actually re-runs "make" effectively. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #30 from David Cook <dcook@prosentient.com.au> --- Anyway, I'm pretty happy with how that turned out overall. It certainly does the trick. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #31 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Thanks Martin and David, I've removed my patches and added yours to the remote branch. I have one concern about the `yarn install` step. At some point we want to freeze the node_modules directory, or we may get expected regressions at a very bad time. Imagine we are testing a beta version D-15 before the release, everything is working great, we decide to release on D-day, package maintainer releases using `yarn install && yarn build:prod`. If something changed in the node modules, we haven't tested it and we release a broken version. It's tricky and I don't have a solution right now, but we need to: * Let developers yarn install whenever they want (/kohadevbox/koha/node_modules will be used/tested) * Have a generated (when?) version (/kohadevbox/node_modules) that would be used to generate the compiled files on ktd startup (sandboxes and devs). We could run build:prod to avoid last minute bugs to popup. * Pick the last generated version for the release -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #145802|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #145838|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #32 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Indeed, that's why I was hesitant to put the 'yarn install' within the PL. I'm sure there must be something we can pass there to ensure it installed pinned versions from our committed lock file.. but I'm damned if I can spot such an option. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #33 from David Cook <dcook@prosentient.com.au> --- (In reply to Martin Renvoize from comment #32)
Indeed, that's why I was hesitant to put the 'yarn install' within the PL. I'm sure there must be something we can pass there to ensure it installed pinned versions from our committed lock file.. but I'm damned if I can spot such an option.
I think "yarn install --frozen-lockfile" might do the trick? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #34 from David Cook <dcook@prosentient.com.au> --- (In reply to Jonathan Druart from comment #31)
I have one concern about the `yarn install` step. At some point we want to freeze the node_modules directory, or we may get expected regressions at a very bad time. Imagine we are testing a beta version D-15 before the release, everything is working great, we decide to release on D-day, package maintainer releases using `yarn install && yarn build:prod`. If something changed in the node modules, we haven't tested it and we release a broken version.
If we use "yarn install --frozen-lockfile" when running via Makefile.PL, that should avoid that scenario. We'd be using only the versions from yarn.lock that hypothetically were tested. That said, testing will be interesting. I suppose you'll get generated files at ktd startup time, and those should be OK until you change branches. At that point, you might have the wrong generated files and not know it. When it comes to local dev/test, we might need to just get into the habit of being mindful of doing yarn install/build when we switch branches (if required). Sandboxes shouldn't be a problem since we could automate the file re-generation...
It's tricky and I don't have a solution right now, but we need to: * Let developers yarn install whenever they want (/kohadevbox/koha/node_modules will be used/tested) * Have a generated (when?) version (/kohadevbox/node_modules) that would be used to generate the compiled files on ktd startup (sandboxes and devs). We could run build:prod to avoid last minute bugs to popup. * Pick the last generated version for the release
I suppose test/QA theoretically should catch if there's a problem with generated files. So the "last generated version for the release" will just be what's in the package.json and yarn.lock files. Maybe we're overthinking it a bit? Although I think changing "yarn install" to "yarn install --frozen-lockfile" sounds necessary for sure. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #35 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to David Cook from comment #34)
(In reply to Jonathan Druart from comment #31)
I have one concern about the `yarn install` step. At some point we want to freeze the node_modules directory, or we may get expected regressions at a very bad time. Imagine we are testing a beta version D-15 before the release, everything is working great, we decide to release on D-day, package maintainer releases using `yarn install && yarn build:prod`. If something changed in the node modules, we haven't tested it and we release a broken version.
If we use "yarn install --frozen-lockfile" when running via Makefile.PL, that should avoid that scenario. We'd be using only the versions from yarn.lock that hypothetically were tested.
Indeed, we discussed that already but forgot. Good :)
That said, testing will be interesting. I suppose you'll get generated files at ktd startup time, and those should be OK until you change branches. At that point, you might have the wrong generated files and not know it.
When it comes to local dev/test, we might need to just get into the habit of being mindful of doing yarn install/build when we switch branches (if required).
Or use watch :)
Sandboxes shouldn't be a problem since we could automate the file re-generation...
It's tricky and I don't have a solution right now, but we need to: * Let developers yarn install whenever they want (/kohadevbox/koha/node_modules will be used/tested) * Have a generated (when?) version (/kohadevbox/node_modules) that would be used to generate the compiled files on ktd startup (sandboxes and devs). We could run build:prod to avoid last minute bugs to popup. * Pick the last generated version for the release
I suppose test/QA theoretically should catch if there's a problem with generated files. So the "last generated version for the release" will just be what's in the package.json and yarn.lock files.
Maybe we're overthinking it a bit?
I did overthink it yesterday, yes! Glad we have an easy solution. The question is then when do we run `yarn install` and update the lock file? But we can discuss that later. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #36 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- New commit added to the remote branch: "Bug 32609: Use the current yarn.lock to generate node_modules" -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #37 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- This is ready to be tested! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #38 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Awesome teamwork here chaps.. I'll resume testing today, should be able to get a signoff done. Do we need a list of other tasks that need taking care as part of pushing this.. i.e. we'll need to update the sandboxes and ktd to run the new yarn commands (or perhaps point them at the new build_resources.PL script alternatively). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #39 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to Martin Renvoize from comment #38)
Do we need a list of other tasks that need taking care as part of pushing this.. i.e. we'll need to update the sandboxes and ktd to run the new yarn commands (or perhaps point them at the new build_resources.PL script alternatively).
This is for https://gitlab.com/koha-community/koha-misc4dev/-/issues/60 I have a patch almost done, but I need a DB rev number to finish it (yes, that's not nice, we will depend on a DB rev for something not DB related...) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #40 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- OK.. been testing running through the make process and this is looking great so far. Not a blocker, but do we need both compiled css/js and source scss/vue files in blib? I can't see a need for them and can't see a problem having them there.. but wondered if we wanted to clean that up whilst we're in here? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #41 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- I tried to do an old-fashioned dev install to test this.. but alas, I failed miserably. I had to sort out all sorts of dependencies to get the 'perl Makefile.PL' step happy.. then upon running 'make' it complained about yarn not being found.. so I installed yarn but then it complained with yarn: error: no such option: --frozen-lockfile So I tried replacing --frozen-lockfile with --immutable and that complained too. In short.. I can run `build-resources.PL` perfectly from inside ktd.. though of course in that case we don't need the `yarn install` step at all as we have our our node_modules all there and waiting for us.. but trying to run it outside of ktd we end up stuck in dependancies hell. I'm not sure where that leaves us.. we're not really actually expecting people to be running dev installs now.. and we don't really document it anywhere anymore as far as I can see on googling.. So maybe this is OK as I believe this should all work for packaging as it stands. We need Mason to weight in really. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #42 from David Cook <dcook@prosentient.com.au> --- (In reply to Martin Renvoize from comment #40)
Not a blocker, but do we need both compiled css/js and source scss/vue files in blib? I can't see a need for them and can't see a problem having them there.. but wondered if we wanted to clean that up whilst we're in here?
Yeah, that thought crossed my mind as well. I don't know enough about scss/vue to know whether we should be putting them in a different source/build folder all together, or if we should update the Makefile.PL to just exclude them from being included. I think certainly a different ticket in any case though. But one worth thinking about just from a maintenance perspective I think. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #43 from David Cook <dcook@prosentient.com.au> --- (In reply to Martin Renvoize from comment #41)
I had to sort out all sorts of dependencies to get the 'perl Makefile.PL' step happy.. then upon running 'make' it complained about yarn not being found.. so I installed yarn but then it complained with yarn: error: no such option: --frozen-lockfile
So I tried replacing --frozen-lockfile with --immutable and that complained too.
Which kind of dependencies did you need to sort out? Was it stuff like "build-essential gcc gettext" or something else? Which OS and OS version were you using? Was this in or outside of Docker? Which version of yarn were you using? Was it Yarn v1, v2, or v3? What was the --immutable complaint? It looks like Debian/Ubuntu provide Yarn 1.x, and npm looks like it uses 1.x (although it provides 2.x too).
In short.. I can run `build-resources.PL` perfectly from inside ktd.. though of course in that case we don't need the `yarn install` step at all as we have our our node_modules all there and waiting for us.. but trying to run it outside of ktd we end up stuck in dependancies hell.
I needed the `yarn install` in my ktd as I didn't have node_modules already there. After a bit of reading, it sounds like running some type of `yarn install` invocation is a common step (although I suppose we could debate about when to run it). I used to run Koha in OpenSuse and getting the dependencies right was always a pain, which is why we switched over to Debian/Ubuntu (and ktd for dev/test).
I'm not sure where that leaves us.. we're not really actually expecting people to be running dev installs now.. and we don't really document it anywhere anymore as far as I can see on googling.. So maybe this is OK as I believe this should all work for packaging as it stands.
We need Mason to weight in really.
Yeah, it would be good to hear from Mason. Perhaps I should have a go with my local package building as well and see if I bump into any gotchas... Both single and dev installs are a bit fraught because of dependencies (which is a big reason I stopped doing them heh). Overall, I don't think the community directly supports anything except Debian package installs on Debian/Ubuntu, although we could always document better. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #44 from David Cook <dcook@prosentient.com.au> --- At the end of the day, I suppose the most important thing is that the Debian packages are built correctly for consumers of Koha, and then the second most important thing is that developers/testers are able to generate Koha installs without too much friction. __Packages__ If Mason can look into how this will work with the existing community package build workflow, that would be useful. I need to do some other work right now... but I'll make a note to look at building a package using these patches. __Dev/Test__ Maybe we should have a crack at testing this process. I was interested by Jonathan's comment about using "watch" for dev/test. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #45 from David Cook <dcook@prosentient.com.au> --- So I'm looking at building a package with these patches, and I notice that the "yarn" in ktd is provided by this source list: deb https://dl.yarnpkg.com/debian/ stable main -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #46 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #45)
So I'm looking at building a package with these patches, and I notice that the "yarn" in ktd is provided by this source list:
deb https://dl.yarnpkg.com/debian/ stable main
After I set up the repo: sudo apt-get install yarn Reading package lists... Done Building dependency tree... Done Reading state information... Done The following additional packages will be installed: libc-ares2 libnode72 libuv1 nodejs nodejs-doc Suggested packages: npm The following NEW packages will be installed: libc-ares2 libnode72 libuv1 nodejs nodejs-doc yarn 0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded. Need to get 12.2 MB of archives. After this operation, 54.1 MB of additional disk space will be used. Do you want to continue? [Y/n] y -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #47 from David Cook <dcook@prosentient.com.au> --- Package builder output on "make": make[1]: Entering directory '/home/koha/build/koha-22.12+test.bb175389' "/usr/bin/perl" build-resources.PL yarn install v1.22.19 [1/4] Resolving packages... [2/4] Fetching packages... [3/4] Linking dependencies... warning " > bootstrap@4.6.2" has unmet peer dependency "jquery@1.9.1 - 3". warning " > bootstrap@4.6.2" has unmet peer dependency "popper.js@^1.16.1". [4/4] Building fresh packages... Done in 242.50s. yarn run v1.22.19 $ yarn css:build:prod && yarn js:build:prod $ gulp build && gulp css --view opac [05:19:53] Using gulpfile ~/build/koha-22.12+test.bb175389/gulpfile.js [05:19:54] Starting 'build'... [05:19:54] Finished 'build' after 10 ms [05:19:58] Using gulpfile ~/build/koha-22.12+test.bb175389/gulpfile.js [05:19:58] Starting 'css'... [05:19:58] Finished 'css' after 12 ms $ webpack --mode production asset main.js 1.44 MiB [emitted] [minimized] [big] (name: main) 1 related asset orphan modules 838 KiB [orphan] 277 modules runtime modules 1.17 KiB 6 modules cacheable modules 2.69 MiB modules by path ./node_modules/ 2.18 MiB 30 modules modules by path ./koha-tmpl/intranet-tmpl/prog/js/vue/ 525 KiB modules by path ./koha-tmpl/intranet-tmpl/prog/js/vue/components/ERM/*.vue 35.6 KiB 20 modules modules by path ./koha-tmpl/intranet-tmpl/prog/js/vue/stores/*.js 3.52 KiB ./koha-tmpl/intranet-tmpl/prog/js/vue/stores/main.js 1.12 KiB [built] [code generated] + 2 modules modules by path ./koha-tmpl/intranet-tmpl/prog/js/vue/*.js 484 KiB ./koha-tmpl/intranet-tmpl/prog/js/vue/routes.js + 232 modules 474 KiB [built] [code generated] + 2 modules ./koha-tmpl/intranet-tmpl/prog/js/vue/main-erm.ts 1.88 KiB [built] [code generated] WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB). This can impact web performance. Assets: main.js (1.44 MiB) WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance. Entrypoints: main (1.44 MiB) main.js WARNING in webpack performance recommendations: You can limit the size of your bundles by using import() or require.ensure to lazy load some parts of your application. For more info visit https://webpack.js.org/guides/code-splitting/ webpack 5.74.0 compiled with 3 warnings in 27212 ms Done in 72.50s. -- Package builder output on "make install": make[1]: Entering directory '/home/koha/build/koha-22.12+test.bb175389' dh_auto_install dh_auto_install: warning: Compatibility levels before 10 are deprecated (level 9 in use) make -j1 install DESTDIR=/home/koha/build/koha-22.12\+test.bb175389/debian/tmp AM_UPDATE_INFO_DIR=no PREFIX=/usr make[2]: Entering directory '/home/koha/build/koha-22.12+test.bb175389' "/usr/bin/perl" build-resources.PL yarn install v1.22.19 [1/4] Resolving packages... success Already up-to-date. Done in 0.83s. yarn run v1.22.19 $ yarn css:build:prod && yarn js:build:prod $ gulp build && gulp css --view opac [05:22:18] Using gulpfile ~/build/koha-22.12+test.bb175389/gulpfile.js [05:22:18] Starting 'build'... [05:22:18] Finished 'build' after 10 ms [05:22:21] Using gulpfile ~/build/koha-22.12+test.bb175389/gulpfile.js [05:22:21] Starting 'css'... [05:22:21] Finished 'css' after 11 ms $ webpack --mode production asset main.js 1.44 MiB [compared for emit] [minimized] [big] (name: main) 1 related asset orphan modules 838 KiB [orphan] 277 modules runtime modules 1.17 KiB 6 modules cacheable modules 2.69 MiB modules by path ./node_modules/ 2.18 MiB 30 modules modules by path ./koha-tmpl/intranet-tmpl/prog/js/vue/ 525 KiB modules by path ./koha-tmpl/intranet-tmpl/prog/js/vue/components/ERM/*.vue 35.6 KiB 20 modules modules by path ./koha-tmpl/intranet-tmpl/prog/js/vue/stores/*.js 3.52 KiB ./koha-tmpl/intranet-tmpl/prog/js/vue/stores/main.js 1.12 KiB [built] [code generated] + 2 modules modules by path ./koha-tmpl/intranet-tmpl/prog/js/vue/*.js 484 KiB ./koha-tmpl/intranet-tmpl/prog/js/vue/routes.js + 232 modules 474 KiB [built] [code generated] + 2 modules ./koha-tmpl/intranet-tmpl/prog/js/vue/main-erm.ts 1.88 KiB [built] [code generated] WARNING in asset size limit: The following asset(s) exceed the recommended size limit (244 KiB). This can impact web performance. Assets: main.js (1.44 MiB) WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance. Entrypoints: main (1.44 MiB) main.js WARNING in webpack performance recommendations: You can limit the size of your bundles by using import() or require.ensure to lazy load some parts of your application. For more info visit https://webpack.js.org/guides/code-splitting/ webpack 5.74.0 compiled with 3 warnings in 20990 ms Done in 33.78s. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #48 from David Cook <dcook@prosentient.com.au> --- Everything was looking good in my build until I hit the issue where debian/control is still incorrect in the master branch... xsltproc --output /home/koha/build/koha-22.12+test.bb175389/debian/tmp/debian/tmp_docbook/ \ /usr/share/xml/docbook/stylesheet/docbook-xsl-ns/manpages/docbook.xsl \ debian/docs/*.xml warning: failed to load external entity "/usr/share/xml/docbook/stylesheet/docbook-xsl-ns/manpages/docbook.xsl" cannot parse /usr/share/xml/docbook/stylesheet/docbook-xsl-ns/manpages/docbook.xsl make[1]: *** [debian/rules:39: override_dh_auto_install] Error 4 make[1]: Leaving directory '/home/koha/build/koha-22.12+test.bb175389' make: *** [debian/rules:9: binary] Error 2 dpkg-buildpackage: error: fakeroot debian/rules binary subprocess returned exit status 2 debuild: fatal error at line 1182: dpkg-buildpackage -us -uc -ui failed -- If the debian/control were correct, then I think the build would've succeeded. For example: Installing /home/koha/build/koha-22.12+test.bb175389/debian/tmp/usr/share/koha/intranet/htdocs/intranet-tmpl/prog/js/vue/dist/main.js.LICENSE.txt Installing /home/koha/build/koha-22.12+test.bb175389/debian/tmp/usr/share/koha/intranet/htdocs/intranet-tmpl/prog/js/vue/dist/main.js Installing /home/koha/build/koha-22.12+test.bb175389/debian/tmp/usr/share/koha/opac/htdocs/opac-tmpl/bootstrap/css/opac.css -- So... as long as the yarn is version 1.22.19... it's all looking good to me. I was building on a Debian Bullseye Docker image using https://gitlab.com/minusdavid/koha-deb-builder-docker -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #49 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- So I was installing on a fresh Ubuntu.. and trying initially to use ubuntu apt repos.. Yarn came in at a newer version that Koha expected.. that was the main stumbling block really. I've since managed to get it all working.. thanks for the tip about that yarn repo. We don't document a dev install anywhere now so it's become more difficult and I've become very rusty ;). My feeling at this point is that we should go with this as is now :). You can add my SO lines to the remote branch Jonathan.. I'm happy I got it all working for a standard install and a dev type install.. and it sounds like David was pretty happy with building a package, all be it he had expected issues due to upstream having a bad debian.control file at the minute (I fixed that in 22.11.x branch already.. not sure if he's got it building and applying to master regularly yet?) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #50 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to Martin Renvoize from comment #49)
You can add my SO lines to the remote branch Jonathan
Done, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #51 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- When everything is set, could you add a little write-up to the wiki? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #52 from David Cook <dcook@prosentient.com.au> --- (In reply to Martin Renvoize from comment #49)
You can add my SO lines to the remote branch Jonathan.. I'm happy I got it all working for a standard install and a dev type install.. and it sounds like David was pretty happy with building a package, all be it he had expected issues due to upstream having a bad debian.control file at the minute (I fixed that in 22.11.x branch already.. not sure if he's got it building and applying to master regularly yet?)
Yeah I'm pretty happy :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=32806 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |32806 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32806 [Bug 32806] Some Vue files need to be moved for better reusability -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #53 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to David Cook from comment #52)
(In reply to Martin Renvoize from comment #49)
You can add my SO lines to the remote branch Jonathan.. I'm happy I got it all working for a standard install and a dev type install.. and it sounds like David was pretty happy with building a package, all be it he had expected issues due to upstream having a bad debian.control file at the minute (I fixed that in 22.11.x branch already.. not sure if he's got it building and applying to master regularly yet?)
Yeah I'm pretty happy :)
PQA then? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #54 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- There is a merge request in misc4dev to support both build_js and build, see https://gitlab.com/koha-community/koha-misc4dev It must be merged before this is pushed to master. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #55 from David Cook <dcook@prosentient.com.au> --- I wonder if Mason should install "yarn" on the package build machine(s) before this is pushed too, so that there aren't any delays building packages going forward? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #56 from Mason James <mtj@kohaaloha.com> --- (In reply to David Cook from comment #55)
I wonder if Mason should install "yarn" on the package build machine(s) before this is pushed too, so that there aren't any delays building packages going forward?
i've added yarn (1.22.19-1) to the pkg build system -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact|testopia@bugs.koha-communit |martin.renvoize@ptfs-europe |y.org |.com Status|Signed Off |Passed QA --- Comment #57 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- I've reviewed the joined misc4dev merge request too now.. I reckon we're PQA here so marking as such. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |23.05.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #58 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Pushed to master for 23.05. Nice work everyone, thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to master |Needs documenting --- Comment #59 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- We decided not to backport this architectural enhancement. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also|https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=32806 | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Owen Leonard <oleonard@myacpl.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |32975 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32975 [Bug 32975] Error in package.json's definition of css:build vs css:build:prod -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #60 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- Any reason why the build-resouces.PL script includes a 1; at the end ? Why the uppercase PL btw? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #61 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to Marcel de Rooy from comment #60)
Any reason why the build-resouces.PL script includes a 1; at the end ?
Maybe not needed, but should not hurt.
Why the uppercase PL btw?
https://metacpan.org/pod/ExtUtils::MakeMaker#How-To-Write-A-Makefile.PL -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=32994 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Jonathan Druart <jonathan.druart+koha@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |32994 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32994 [Bug 32994] Remove compiled files from src (2) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|23.05.00 |23.05.00, 22.11.04 released in| | --- Comment #62 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- I decided to backport this one now.. it will aid in future maintenance, as it's already proved to cause a few minor issues with erm bugfix backports we've wanted to keep. It's also proved itself nicely in master now so I feel it's a safe bet. Backported for release in 22.11.04 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Lucas Gass <lucas@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |lucas@bywatersolutions.com --- Comment #63 from Lucas Gass <lucas@bywatersolutions.com> --- Won't backport to 22.05.x, unless requested. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the| |This important release notes| |architectural change | |removes the built css and | |javascript files from | |source control and | |introduces a build process | |and trigger into our | |packaging routines. | | | |This | |will both save space in the | |repository and lead to less | |mistakes from developers by | |dropping the need to build, | |add and commit these files | |at release time. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #64 from David Cook <dcook@prosentient.com.au> --- (In reply to Martin Renvoize from comment #62)
I decided to backport this one now.. it will aid in future maintenance, as it's already proved to cause a few minor issues with erm bugfix backports we've wanted to keep.
It's also proved itself nicely in master now so I feel it's a safe bet.
Backported for release in 22.11.04
Will know more soon! I'm working on a customized 22.11.03 and planning to merge in 22.11.04 when it comes out. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |aleisha@catalyst.net.nz --- Comment #65 from Aleisha Amohia <aleisha@catalyst.net.nz> --- Hi all, This has caused some problems for our libraries upgrading to 22.11.04. For example, - the calendar icon that shows in the flatpickr text input fields does not show - making return claims on the checkouts page shows the 'loading' icon forever, have to refresh the page to show the claim Is there something we need to additionally do or install? Thanks Aleisha -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #66 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #64)
Will know more soon! I'm working on a customized 22.11.03 and planning to merge in 22.11.04 when it comes out.
(In reply to Aleisha Amohia from comment #65)
This has caused some problems for our libraries upgrading to 22.11.04.
(In reply to Aleisha Amohia from comment #65)
Is there something we need to additionally do or install?
I recently decided to pin at 22.11.03 for the time being, so I'm not sure what the problem might be. I could probably look at this later this week if no one else does. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #67 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- (In reply to Martin Renvoize from comment #62)
Backported for release in 22.11.04
Did you ask Mason to adjust the pkg workflow to embed the css and js files? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #68 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- (In reply to Jonathan Druart from comment #67)
(In reply to Martin Renvoize from comment #62)
Backported for release in 22.11.04
Did you ask Mason to adjust the pkg workflow to embed the css and js files?
Community packages are all working as expected here.. I presume your building your own Aliesha.. in which case the core thing is to ensure you run the build-resources.PL script as part of your package building process.. that will compile the CSS and JS resources for you. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #69 from Aleisha Amohia <aleisha@catalyst.net.nz> --- (In reply to Martin Renvoize from comment #68)
(In reply to Jonathan Druart from comment #67)
(In reply to Martin Renvoize from comment #62)
Backported for release in 22.11.04
Did you ask Mason to adjust the pkg workflow to embed the css and js files?
Community packages are all working as expected here.. I presume your building your own Aliesha.. in which case the core thing is to ensure you run the build-resources.PL script as part of your package building process.. that will compile the CSS and JS resources for you.
Hey, nope, this problem is occurring for our libraries installing stock Koha from the community tag, so not building the packages ourselves $ apt-cache policy koha-common koha-common: Installed: 22.11.04-1 Candidate: 22.11.04-1 Version table: *** 22.11.04-1 500 500 http://debian.koha-community.org/koha 22.11/main amd64 Packages 100 /var/lib/dpkg/status How can I run the build-resources.PL script please? Couldn't see any documentation on the Community wiki -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #70 from David Cook <dcook@prosentient.com.au> --- (In reply to Martin Renvoize from comment #68)
Community packages are all working as expected here.. I presume your building your own Aliesha.. in which case the core thing is to ensure you run the build-resources.PL script as part of your package building process.. that will compile the CSS and JS resources for you.
The Makefile should be running build-resources.PL as per https://gitlab.com/joubu/Koha/-/commit/b64db106e0db41da06fbf72f1674b2ed43b3f... If the overall process was failing, I think Aleisha would have different symptoms. The issue she's having might be unrelated to this change but won't know without more information (provided either by Aleisha or someone else investigating). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #71 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #70)
If the overall process was failing, I think Aleisha would have different symptoms. The issue she's having might be unrelated to this change but won't know without more information (provided either by Aleisha or someone else investigating).
Actually, on that note, I think Aleisha should probably open a separate bug report for the issues she's facing and do a "See Also" to this one in case it's related. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #72 from Aleisha Amohia <aleisha@catalyst.net.nz> --- (In reply to David Cook from comment #71)
(In reply to David Cook from comment #70)
If the overall process was failing, I think Aleisha would have different symptoms. The issue she's having might be unrelated to this change but won't know without more information (provided either by Aleisha or someone else investigating).
Actually, on that note, I think Aleisha should probably open a separate bug report for the issues she's facing and do a "See Also" to this one in case it's related.
Will do. I did a git bisect to narrow down the commit where we stopped seeing things we expected to see but there could be something else at play in the packaging process or other. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #73 from Tomás Cohen Arazi <tomascohen@gmail.com> --- As I posted on the koha-devel list: " Kyle and I have been working on this: https://gitlab.com/koha-community/koha-dpkg-docker#usage which is a simple way to build packages, and considers that required step. Hope it helps. " -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #74 from Aleisha Amohia <aleisha@catalyst.net.nz> --- Thank you Tomás but as I said, we aren't building our own packages. It is community packages that we get this problem -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #75 from David Cook <dcook@prosentient.com.au> --- (In reply to Tomás Cohen Arazi from comment #73)
As I posted on the koha-devel list:
" Kyle and I have been working on this:
https://gitlab.com/koha-community/koha-dpkg-docker#usage
which is a simple way to build packages, and considers that required step.
Hope it helps. "
Could we get more details on why you did it that way? It should be unnecessary... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #76 from David Cook <dcook@prosentient.com.au> --- (In reply to Aleisha Amohia from comment #74)
Thank you Tomás but as I said, we aren't building our own packages. It is community packages that we get this problem
It looks like there are some demo boxes on 22.11 so I'm happy to help Aleisha out with this one (on a separate bug report). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=33621 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #77 from Aleisha Amohia <aleisha@catalyst.net.nz> --- (In reply to David Cook from comment #76)
(In reply to Aleisha Amohia from comment #74)
Thank you Tomás but as I said, we aren't building our own packages. It is community packages that we get this problem
It looks like there are some demo boxes on 22.11 so I'm happy to help Aleisha out with this one (on a separate bug report).
Thank you. I've reported Bug 33621 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #78 from David Cook <dcook@prosentient.com.au> --- (In reply to Aleisha Amohia from comment #77)
Thank you. I've reported Bug 33621
Thanks, Aleisha. It looks like one issue is probably due to bug 32978 and the other looks like a coding bug. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Aleisha Amohia <aleisha@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=33615 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |caroline.cyr-la-rose@inlibr | |o.com Status|Needs documenting |RESOLVED Resolution|--- |FIXED --- Comment #79 from Caroline Cyr La Rose <caroline.cyr-la-rose@inlibro.com> --- I don't think there is anything to document here. Closing. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Text to go in the|This important |This important release notes|architectural change |architectural change |removes the built css and |removes the built CSS and |javascript files from |JavaScript files from |source control and |source control and |introduces a build process |introduces a build process |and trigger into our |and trigger into our |packaging routines. |packaging routines. | |This |This |will both save space in the |will both save space in the |repository and lead to less |repository and lead to less |mistakes from developers by |mistakes from developers by |dropping the need to build, |dropping the need to build, |add and commit these files |add and commit these files |at release time. |at release time. | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 --- Comment #80 from David Cook <dcook@prosentient.com.au> --- I was just looking at the Debian packaging logs and wondering why build-resources.PL gets run twice, but it's because it's run for "make" and "make install". And that's because in Makefile.PL there is the following line: install :: all install_koha set_koha_ownership set_koha_permissions warn_koha_env_vars I suppose if it really bothered us we could do something about it, but it doesn't really seem like a big issue. Wastes a bit of time, but better to do it too much than not enough... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=39623 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=32609 Andrii Nugged <nugged@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=42850 -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org