Hello koha community, We made some changes to our Koha instance and are now looking for a mechanism to apply them to a new installation. - some of the changes are simply new files, thats easy. - some changes were possible to put into plugins or use existing hooks. but how do you deal with changes to the koha sourcecode? this is our strategy: 1) we have a git repo with our koha source code. 2) if we change code in Foobar.pm we create 3 files. Foobar.pm.orig Foobar.pm.changed Foobar.pm.patch 3) on a new koha instance next we have a script `install.sh` that compares Foobar.pm.orig with the installed file. if they match, the patch is applied. Does that sound like a good idea? How do YOU manage core changes that do not go upstream? cheers, david
Hi, Sounds like old-fashion version management :) We use GIT versioning system https://git-scm.com. The community real repository is https://git.koha-community.org/Koha-community/Koha.git. It is also on GitHub : https://github.com/Koha-Community/Koha I suggest you create a GitHub account and fork it to propose your changes (a new branch per feature). Best regards, Le 02/11/2021 à 00:00, David Schmidt a écrit :
Hello koha community,
We made some changes to our Koha instance and are now looking for a mechanism to apply them to a new installation.
- some of the changes are simply new files, thats easy. - some changes were possible to put into plugins or use existing hooks.
but how do you deal with changes to the koha sourcecode?
this is our strategy:
1) we have a git repo with our koha source code. 2) if we change code in Foobar.pm we create 3 files. Foobar.pm.orig Foobar.pm.changed Foobar.pm.patch 3) on a new koha instance next we have a script `install.sh` that compares Foobar.pm.orig with the installed file. if they match, the patch is applied.
Does that sound like a good idea? How do YOU manage core changes that do not go upstream?
cheers, david
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/
-- Fridolin SOMERS <fridolin.somers@biblibre.com> Software and system maintainer 🦄 BibLibre, France
Hi David, I won’t make a comment on good vs bad strategies, as those are just subjective judgements, and ultimately it really all depends on your context. However, it sounds like you’re making a lot of work for yourself with those files. Using git commands, you could extract equivalents of those .orig, .changed, and .patched files. Also, by patching prod files like that, you don’t necessarily know exactly what your application state is at any particular time. I suppose you could probably do a git-based manual patching deployment system using git hashes to represent versions, but I’ve found in practice that those cause more headaches than they solve. Most Koha support providers (like the one I work for) will fork the branch(es) they want to customize, and then generate Debian packages which they distribute around the world as necessary. As soon as you’re running more than 1 Koha server, you really want a way to easily and quickly deploy changes across multiple machines. Koha is large and complex, so I think you’ll find that leveraging the work the community has done to build robust Debian packages is your best bet. Carrying customizations forward through time takes work, so it’s best to upstream everything you can, but I think we all know that’s not always feasible. Again git commands can help you show the changes you’ve made between different versions, and then you can make periodic efforts to review your customizations and port over anything you need to when a new version comes out. I hope that you find that helpful. David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595 From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of David Schmidt Sent: Tuesday, 2 November 2021 9:01 PM To: Koha-devel@lists.koha-community.org Subject: [Koha-devel] custom core patches management Hello koha community, We made some changes to our Koha instance and are now looking for a mechanism to apply them to a new installation. - some of the changes are simply new files, thats easy. - some changes were possible to put into plugins or use existing hooks. but how do you deal with changes to the koha sourcecode? this is our strategy: 1) we have a git repo with our koha source code. 2) if we change code in Foobar.pm we create 3 files. Foobar.pm.orig Foobar.pm.changed Foobar.pm.patch 3) on a new koha instance next we have a script `install.sh` that compares Foobar.pm.orig with the installed file. if they match, the patch is applied. Does that sound like a good idea? How do YOU manage core changes that do not go upstream? cheers, david
This is exactly how I manage my Koha installs, with the exception that instead of using the packages I run my production servers from a dev install. When I have changes ready I just `git fetch; git reset --hard origin/my_release_branch` to apply those changes, but then I have to manually go through koha-upgrade-schema and koha-plack --restart for each instance (I have 3 active instances). So I have traded building and distributing debian packages for manually doing the post upgrade work that the packages would do for me. On Tue, Nov 2, 2021 at 7:00 PM <dcook@prosentient.com.au> wrote:
Hi David,
I won’t make a comment on good vs bad strategies, as those are just subjective judgements, and ultimately it really all depends on your context.
However, it sounds like you’re making a lot of work for yourself with those files. Using git commands, you could extract equivalents of those .orig, .changed, and .patched files.
Also, by patching prod files like that, you don’t necessarily know exactly what your application state is at any particular time. I suppose you could probably do a git-based manual patching deployment system using git hashes to represent versions, but I’ve found in practice that those cause more headaches than they solve.
Most Koha support providers (like the one I work for) will fork the branch(es) they want to customize, and then generate Debian packages which they distribute around the world as necessary. As soon as you’re running more than 1 Koha server, you really want a way to easily and quickly deploy changes across multiple machines. Koha is large and complex, so I think you’ll find that leveraging the work the community has done to build robust Debian packages is your best bet.
Carrying customizations forward through time takes work, so it’s best to upstream everything you can, but I think we all know that’s not always feasible. Again git commands can help you show the changes you’ve made between different versions, and then you can make periodic efforts to review your customizations and port over anything you need to when a new version comes out.
I hope that you find that helpful.
David Cook
Senior Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
*From:* Koha-devel <koha-devel-bounces@lists.koha-community.org> *On Behalf Of *David Schmidt *Sent:* Tuesday, 2 November 2021 9:01 PM *To:* Koha-devel@lists.koha-community.org *Subject:* [Koha-devel] custom core patches management
Hello koha community,
We made some changes to our Koha instance and are now looking for a mechanism to apply them to a new installation.
- some of the changes are simply new files, thats easy.
- some changes were possible to put into plugins or use existing hooks.
but how do you deal with changes to the koha sourcecode?
this is our strategy:
1) we have a git repo with our koha source code.
2) if we change code in Foobar.pm we create 3 files.
Foobar.pm.orig
Foobar.pm.changed
Foobar.pm.patch
3) on a new koha instance next we have a script `install.sh` that compares Foobar.pm.orig with the installed file. if they match, the patch is applied.
Does that sound like a good idea?
How do YOU manage core changes that do not go upstream?
cheers,
david _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/
-- Michael Hafen Washington County School District Technology Department Systems Analyst
I would actually be a bit more interested in git based deployments if we fixed up the directory layout for the Koha codebase. One downside to the git approach is that you’ll also need to manually update generated configuration files. Not a big problem for small scale deployments, but not something you really want to be doing when you’re updating 100 Koha instances across 15 servers across 3 continents at the same time (which is still small scale by some measures haha). I think the git-based installation makes it easier to wind up with a broken Koha by accident. Software deployment is a bit of a passion of mine, so really curious to hear from more people. I’m quite interested in using Carton to handle Koha’s Perl dependencies instead of using OS-based Perl dependencies. I’m planning to move that way for a different project, although I’m actually still using OS packages for deploying my app. Actually, that’s another reason to consider going with package-based deployments: dependency management. (That said, I have noticed an issue where Koha gets uninstalled when updating the OS. Something I still need to investigate. Think it is Zebra related.) At some point, I’d like to move to Docker-based deployments, but there’s still more work to do on that front. It’s very doable, but without a critical mass of Koha community members wanting to do it, I think it would be a waste of my time to invest too much in it. There’s a saying that you go faster alone but farther together, and I think that’s something worth keeping in mind. David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595 From: Michael Hafen (TECH) <michael.hafen@washk12.org> Sent: Thursday, 4 November 2021 3:37 AM To: dcook@prosentient.com.au Cc: David Schmidt <mail@davidschmidt.at>; koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] custom core patches management This is exactly how I manage my Koha installs, with the exception that instead of using the packages I run my production servers from a dev install. When I have changes ready I just `git fetch; git reset --hard origin/my_release_branch` to apply those changes, but then I have to manually go through koha-upgrade-schema and koha-plack --restart for each instance (I have 3 active instances). So I have traded building and distributing debian packages for manually doing the post upgrade work that the packages would do for me. On Tue, Nov 2, 2021 at 7:00 PM <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> > wrote: Hi David, I won’t make a comment on good vs bad strategies, as those are just subjective judgements, and ultimately it really all depends on your context. However, it sounds like you’re making a lot of work for yourself with those files. Using git commands, you could extract equivalents of those .orig, .changed, and .patched files. Also, by patching prod files like that, you don’t necessarily know exactly what your application state is at any particular time. I suppose you could probably do a git-based manual patching deployment system using git hashes to represent versions, but I’ve found in practice that those cause more headaches than they solve. Most Koha support providers (like the one I work for) will fork the branch(es) they want to customize, and then generate Debian packages which they distribute around the world as necessary. As soon as you’re running more than 1 Koha server, you really want a way to easily and quickly deploy changes across multiple machines. Koha is large and complex, so I think you’ll find that leveraging the work the community has done to build robust Debian packages is your best bet. Carrying customizations forward through time takes work, so it’s best to upstream everything you can, but I think we all know that’s not always feasible. Again git commands can help you show the changes you’ve made between different versions, and then you can make periodic efforts to review your customizations and port over anything you need to when a new version comes out. I hope that you find that helpful. David Cook Senior Software Engineer Prosentient Systems Suite 7.03 6a Glen St Milsons Point NSW 2061 Australia Office: 02 9212 0899 Online: 02 8005 0595 From: Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org> > On Behalf Of David Schmidt Sent: Tuesday, 2 November 2021 9:01 PM To: Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> Subject: [Koha-devel] custom core patches management Hello koha community, We made some changes to our Koha instance and are now looking for a mechanism to apply them to a new installation. - some of the changes are simply new files, thats easy. - some changes were possible to put into plugins or use existing hooks. but how do you deal with changes to the koha sourcecode? this is our strategy: 1) we have a git repo with our koha source code. 2) if we change code in Foobar.pm we create 3 files. Foobar.pm.orig Foobar.pm.changed Foobar.pm.patch 3) on a new koha instance next we have a script `install.sh` that compares Foobar.pm.orig with the installed file. if they match, the patch is applied. Does that sound like a good idea? How do YOU manage core changes that do not go upstream? cheers, david _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/ -- Michael Hafen Washington County School District Technology Department Systems Analyst
I'm happy to hear someone else raising the issue about koha-perldeps going wild on OS updates. Every perl package suddenly become suggested for autoremove. I experience that on most updates on every server. Regarding you request for info, David, we also are git-based deployment. Each of our customers have personnalized branches so every update require personnalized attention, but we have code in the main branch to help. Since they require (and get) their requirements "quickly", this approach is very fast. Major updates (19.05 to 20.05 for example) have to be planned carefully, though. I don't see us moving to a different method unless we could standardize our code base. Philippe Blouin, Directeur de la technologie Tél. : (833) 465-4276, poste 230 philippe.blouin@inLibro.com <mailto:philippe.blouin@inLibro.com> inLibro | pour esprit libre | www.inLibro.com <http://www.inLibro.com> On 2021-11-03 6:39 p.m., dcook@prosentient.com.au wrote:
I would actually be a bit more interested in git based deployments if we fixed up the directory layout for the Koha codebase.
One downside to the git approach is that you’ll also need to manually update generated configuration files. Not a big problem for small scale deployments, but not something you really want to be doing when you’re updating 100 Koha instances across 15 servers across 3 continents at the same time (which is still small scale by some measures haha). I think the git-based installation makes it easier to wind up with a broken Koha by accident.
Software deployment is a bit of a passion of mine, so really curious to hear from more people. I’m quite interested in using Carton to handle Koha’s Perl dependencies instead of using OS-based Perl dependencies. I’m planning to move that way for a different project, although I’m actually still using OS packages for deploying my app. Actually, that’s another reason to consider going with package-based deployments: dependency management. (That said, I have noticed an issue where Koha gets uninstalled when updating the OS. Something I still need to investigate. Think it is Zebra related.)
At some point, I’d like to move to Docker-based deployments, but there’s still more work to do on that front. It’s very doable, but without a critical mass of Koha community members wanting to do it, I think it would be a waste of my time to invest too much in it. There’s a saying that you go faster alone but farther together, and I think that’s something worth keeping in mind.
David Cook
Senior Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
*From:*Michael Hafen (TECH) <michael.hafen@washk12.org> *Sent:* Thursday, 4 November 2021 3:37 AM *To:* dcook@prosentient.com.au *Cc:* David Schmidt <mail@davidschmidt.at>; koha-devel@lists.koha-community.org *Subject:* Re: [Koha-devel] custom core patches management
This is exactly how I manage my Koha installs, with the exception that instead of using the packages I run my production servers from a dev install. When I have changes ready I just `git fetch; git reset --hard origin/my_release_branch` to apply those changes, but then I have to manually go through koha-upgrade-schema and koha-plack --restart for each instance (I have 3 active instances). So I have traded building and distributing debian packages for manually doing the post upgrade work that the packages would do for me.
On Tue, Nov 2, 2021 at 7:00 PM <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au>> wrote:
Hi David,
I won’t make a comment on good vs bad strategies, as those are just subjective judgements, and ultimately it really all depends on your context.
However, it sounds like you’re making a lot of work for yourself with those files. Using git commands, you could extract equivalents of those .orig, .changed, and .patched files.
Also, by patching prod files like that, you don’t necessarily know exactly what your application state is at any particular time. I suppose you could probably do a git-based manual patching deployment system using git hashes to represent versions, but I’ve found in practice that those cause more headaches than they solve.
Most Koha support providers (like the one I work for) will fork the branch(es) they want to customize, and then generate Debian packages which they distribute around the world as necessary. As soon as you’re running more than 1 Koha server, you really want a way to easily and quickly deploy changes across multiple machines. Koha is large and complex, so I think you’ll find that leveraging the work the community has done to build robust Debian packages is your best bet.
Carrying customizations forward through time takes work, so it’s best to upstream everything you can, but I think we all know that’s not always feasible. Again git commands can help you show the changes you’ve made between different versions, and then you can make periodic efforts to review your customizations and port over anything you need to when a new version comes out.
I hope that you find that helpful.
David Cook
Senior Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia
Office: 02 9212 0899
Online: 02 8005 0595
*From:*Koha-devel <koha-devel-bounces@lists.koha-community.org <mailto:koha-devel-bounces@lists.koha-community.org>> *On Behalf Of *David Schmidt *Sent:* Tuesday, 2 November 2021 9:01 PM *To:* Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> *Subject:* [Koha-devel] custom core patches management
Hello koha community,
We made some changes to our Koha instance and are now looking for a mechanism to apply them to a new installation.
- some of the changes are simply new files, thats easy.
- some changes were possible to put into plugins or use existing hooks.
but how do you deal with changes to the koha sourcecode?
this is our strategy:
1) we have a git repo with our koha source code.
2) if we change code in Foobar.pm we create 3 files.
Foobar.pm.orig
Foobar.pm.changed
Foobar.pm.patch
3) on a new koha instance next we have a script `install.sh` that compares Foobar.pm.orig with the installed file. if they match, the patch is applied.
Does that sound like a good idea?
How do YOU manage core changes that do not go upstream?
cheers,
david
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org <mailto:Koha-devel@lists.koha-community.org> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel <https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel> website : https://www.koha-community.org/ <https://www.koha-community.org/> git : https://git.koha-community.org/ <https://git.koha-community.org/> bugs : https://bugs.koha-community.org/ <https://bugs.koha-community.org/>
--
Michael Hafen
Washington County School District Technology Department
Systems Analyst
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : https://www.koha-community.org/ git : https://git.koha-community.org/ bugs : https://bugs.koha-community.org/
we went along with our strategy and wrote a perl script that outputs a bash script. After reviewing the output you can run the script and it will create the necessary symlinks and applies patches. thats our short term approach, in the long run we might stop installing from koha debian packages and use the official koha git checkout directly and maintain branches for each koha instance. if anyone is interested in the perl script I wrote let me know. cheers david On Tue, 2 Nov 2021, at 11:00 AM, David Schmidt wrote:
Hello koha community,
We made some changes to our Koha instance and are now looking for a mechanism to apply them to a new installation.
- some of the changes are simply new files, thats easy. - some changes were possible to put into plugins or use existing hooks.
but how do you deal with changes to the koha sourcecode?
this is our strategy:
1) we have a git repo with our koha source code. 2) if we change code in Foobar.pm we create 3 files. Foobar.pm.orig Foobar.pm.changed Foobar.pm.patch 3) on a new koha instance next we have a script `install.sh` that compares Foobar.pm.orig with the installed file. if they match, the patch is applied.
Does that sound like a good idea? How do YOU manage core changes that do not go upstream?
cheers, david
participants (6)
-
David Schmidt -
dcook＠prosentient.com.au -
Fridolin SOMERS -
Michael Hafen (TECH) -
Philippe Blouin -
Victor Grousset/tuxayo