koha-upgrade-schema extremely slow even when no upgrades to make
Hi all, Does anyone else notice that koha-upgrade-schema runs extremely slowly, particularly when given a large list of instances, even when there are no upgrades to make? In theory, it should just return very quickly, because there are no schema upgrades to make. I haven't looked into the code yet, but figured I'd put the question to the community. David Cook Software Engineer Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Online: 02 8005 0595
What is "extremely slow"? It takes 1.5s to run for me, time to load the modules (and especially dbic schema) Le lun. 21 sept. 2020 à 04:28, <dcook@prosentient.com.au> a écrit :
Hi all,
Does anyone else notice that koha-upgrade-schema runs extremely slowly, particularly when given a large list of instances, even when there are no upgrades to make?
In theory, it should just return very quickly, because there are no schema upgrades to make.
I haven’t looked into the code yet, but figured I’d put the question to the community.
David Cook
Software Engineer
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
I'd have to do a time test. However, when you say 1.5s, do you mean per instance? If so, that is probably about right. I have about 54 instances I'm upgrading, and it feels like it's taking around 2+ minutes. I suppose there are also nearly 20,000 lines of conditionals in updatedatabase.pl as well. I reckon updatedatabase.pl should run C4::Context->preference("Version") and compare it against $Koha::VERSION at the start of the script. If the former is less than the latter, then launch into the 20,000 lines of conditionals (or better yet a more optimized install path but that would be out of scope here). Although as you say... the majority of the time is probably to load the DBIC schema and the modules. It just seems a shame it takes that long just for it to conclude that there's no work to do. Maybe koha-upgrade-schema could be optimized to use some utility scripts that rapidly report the code version and the database version to know if it even needs to run updatedatabase.pl... At the moment, I just find it unsettling how long it takes to deploy Koha to production when using the Debian packages for a server with many tenants. David Cook Software Engineer Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Online: 02 8005 0595 -----Original Message----- From: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Sent: Monday, 28 September 2020 9:18 PM To: David Cook <dcook@prosentient.com.au> Cc: Koha Devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] koha-upgrade-schema extremely slow even when no upgrades to make What is "extremely slow"? It takes 1.5s to run for me, time to load the modules (and especially dbic schema) Le lun. 21 sept. 2020 à 04:28, <dcook@prosentient.com.au> a écrit :
Hi all,
Does anyone else notice that koha-upgrade-schema runs extremely slowly, particularly when given a large list of instances, even when there are no upgrades to make?
In theory, it should just return very quickly, because there are no schema upgrades to make.
I haven’t looked into the code yet, but figured I’d put the question to the community.
David Cook
Software Engineer
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
The code that calls updatedatabase.pl should deal with the comparaison, see C4::Auth::_version_check It is quite trivial to have the bash script calling the perl code, but since you will call C4::Context->preference you will face the same problem :) A bit more work would be needed to avoid that and fetch the pref value from the DB. It's worth a new bug report ;) Le jeu. 1 oct. 2020 à 08:20, <dcook@prosentient.com.au> a écrit :
I'd have to do a time test.
However, when you say 1.5s, do you mean per instance? If so, that is probably about right. I have about 54 instances I'm upgrading, and it feels like it's taking around 2+ minutes.
I suppose there are also nearly 20,000 lines of conditionals in updatedatabase.pl as well.
I reckon updatedatabase.pl should run C4::Context->preference("Version") and compare it against $Koha::VERSION at the start of the script. If the former is less than the latter, then launch into the 20,000 lines of conditionals (or better yet a more optimized install path but that would be out of scope here).
Although as you say... the majority of the time is probably to load the DBIC schema and the modules.
It just seems a shame it takes that long just for it to conclude that there's no work to do.
Maybe koha-upgrade-schema could be optimized to use some utility scripts that rapidly report the code version and the database version to know if it even needs to run updatedatabase.pl...
At the moment, I just find it unsettling how long it takes to deploy Koha to production when using the Debian packages for a server with many tenants.
David Cook Software Engineer Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia
Office: 02 9212 0899 Online: 02 8005 0595
-----Original Message----- From: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Sent: Monday, 28 September 2020 9:18 PM To: David Cook <dcook@prosentient.com.au> Cc: Koha Devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] koha-upgrade-schema extremely slow even when no upgrades to make
What is "extremely slow"? It takes 1.5s to run for me, time to load the modules (and especially dbic schema)
Le lun. 21 sept. 2020 à 04:28, <dcook@prosentient.com.au> a écrit :
Hi all,
Does anyone else notice that koha-upgrade-schema runs extremely slowly, particularly when given a large list of instances, even when there are no upgrades to make?
In theory, it should just return very quickly, because there are no schema upgrades to make.
I haven’t looked into the code yet, but figured I’d put the question to the community.
David Cook
Software Engineer
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Agreed about the code calling updatedatabase.pl dealing with the comparison. Like you say, I was thinking rather than using C4::Context->preference that we'd just use a raw SQL query without the ORM. Cool. I'll raise one! David Cook Software Engineer Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Online: 02 8005 0595 -----Original Message----- From: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Sent: Thursday, 1 October 2020 7:59 PM To: David Cook <dcook@prosentient.com.au> Cc: Koha Devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] koha-upgrade-schema extremely slow even when no upgrades to make The code that calls updatedatabase.pl should deal with the comparaison, see C4::Auth::_version_check It is quite trivial to have the bash script calling the perl code, but since you will call C4::Context->preference you will face the same problem :) A bit more work would be needed to avoid that and fetch the pref value from the DB. It's worth a new bug report ;) Le jeu. 1 oct. 2020 à 08:20, <dcook@prosentient.com.au> a écrit :
I'd have to do a time test.
However, when you say 1.5s, do you mean per instance? If so, that is probably about right. I have about 54 instances I'm upgrading, and it feels like it's taking around 2+ minutes.
I suppose there are also nearly 20,000 lines of conditionals in updatedatabase.pl as well.
I reckon updatedatabase.pl should run C4::Context->preference("Version") and compare it against $Koha::VERSION at the start of the script. If the former is less than the latter, then launch into the 20,000 lines of conditionals (or better yet a more optimized install path but that would be out of scope here).
Although as you say... the majority of the time is probably to load the DBIC schema and the modules.
It just seems a shame it takes that long just for it to conclude that there's no work to do.
Maybe koha-upgrade-schema could be optimized to use some utility scripts that rapidly report the code version and the database version to know if it even needs to run updatedatabase.pl...
At the moment, I just find it unsettling how long it takes to deploy Koha to production when using the Debian packages for a server with many tenants.
David Cook Software Engineer Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia
Office: 02 9212 0899 Online: 02 8005 0595
-----Original Message----- From: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Sent: Monday, 28 September 2020 9:18 PM To: David Cook <dcook@prosentient.com.au> Cc: Koha Devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] koha-upgrade-schema extremely slow even when no upgrades to make
What is "extremely slow"? It takes 1.5s to run for me, time to load the modules (and especially dbic schema)
Le lun. 21 sept. 2020 à 04:28, <dcook@prosentient.com.au> a écrit :
Hi all,
Does anyone else notice that koha-upgrade-schema runs extremely slowly, particularly when given a large list of instances, even when there are no upgrades to make?
In theory, it should just return very quickly, because there are no schema upgrades to make.
I haven’t looked into the code yet, but figured I’d put the question to the community.
David Cook
Software Engineer
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
participants (2)
-
dcook＠prosentient.com.au -
Jonathan Druart