Update database changes proposal [IMPORTANT]
Hello, During the hackfest in Mumbaï, 5 of us had a brainstorming about database changes to try to improve it. Here are the goals: * must result in patches being easier to test by testers (to sign-off) * must result in less conflict when trying to apply a patch with a DB update when another patch has been applied (the .XXXX problem) * must fix the linear problem (where patches are applied in a completly linear way) * be easy to implement We throwed many ideas, here is the one that seems both easy and usefull : * patches would be submitted in a specific file, in atomicupdate directory * a new file (YAML) would be added, that would have 2 columns, one for the version number, one for the atomicupdate file name * updatedatabase wouldn't change anymore: we would just add something like : LoadYAMLFile() ForEachLine { if (C4::Context->preference("Version") < FirstColumnYAML) { exec(secondColumnYAML); SetVersion (FirstColumnYAML); } } The YAML file would be updated only by the Release Manager, when he pushes a patch. This idea fully achieve most of our initial goals: * (very) easy to implement * it will be easier to test a patch : is there an atomic update in the patch ? OK, to test it, i'll first execute the file * there won't be any conflict anymore of a patch applied and your .XXXX file not applying anymore, needing a rebase * it fixes partially the linear problem = if you have a local branch, you can see what has been applied just looking in the yaml file. Let's say Limoges library has sponsored stuff that has resulted in atomicupdate/limogeupdate.pl Those changes are live in Limoges, but still not in official/community version. The Limoges YAML file will be: 3.07.0001 Community_change1.pl 3.07.0002 community_change2.pl localchange limogesupdate.pl 3.07.0003 community_change3.pl the "localchange" in first column will mean for the updater : "OK, apply the change but don't change the version number" When the localchange has been applied on master community branch, the official/community YAML file will look like: 3.07.0001 Community_change1.pl 3.07.0002 community_change2.pl 3.07.0003 community_change3.pl 3.07.0004 limogesupdate.pl Switching back limoges to a community version would require some manual checking, but that will be *much* easier than what we have today : * diff limoges.yaml official.yaml * apply community changes if/where needed * UPDATE systempreferences SET kohaversion=3.07.0004 manually It's done ! And if the localchange that has been made is never merged into master, whatever the reason, it will be easier to keep track of it too. And if you want to apply a submitted patch to your local setup before it's pushed onto master, you can, just by running the file in atomicupdate directory. And add a "localchange" line to your YAML file to remember the change you've made. Does anyone see a problem with this new updatedatabase procedure ? Until now, I have pushed no changes in database, so if we agree, i'll implement this change quickly (before pushing any change having a database consequence) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
For my part, I'm very excited about this change. I think we're going to have a much easier time of submitting database changes this way. There will be a learning curve for folks used to doing it the old way, but like the change to using a common systempreferences.sql instead of one for each language. It'll make life easier, so I would assume adoption would occur readily. In the event it doesn't, the QA team could agree to reform old-style submissions into the new style, at least for the rest of the 3.8 release cycle. Any issues anyone can see with this approach, please bring them up. We're only a few at the hackfest, and need more opinions before we proceed. But I think this is going to work well, and solve several problems, while still preserving the quick version checking we've already got. Cheers, -Ian On Sat, Nov 5, 2011 at 2:55 AM, Paul Poulain <paul.poulain@biblibre.com>wrote:
Hello,
During the hackfest in Mumbaï, 5 of us had a brainstorming about database changes to try to improve it. Here are the goals: * must result in patches being easier to test by testers (to sign-off) * must result in less conflict when trying to apply a patch with a DB update when another patch has been applied (the .XXXX problem) * must fix the linear problem (where patches are applied in a completly linear way) * be easy to implement
We throwed many ideas, here is the one that seems both easy and usefull : * patches would be submitted in a specific file, in atomicupdate directory * a new file (YAML) would be added, that would have 2 columns, one for the version number, one for the atomicupdate file name * updatedatabase wouldn't change anymore: we would just add something like : LoadYAMLFile() ForEachLine { if (C4::Context->preference("Version") < FirstColumnYAML) { exec(secondColumnYAML); SetVersion (FirstColumnYAML); } } The YAML file would be updated only by the Release Manager, when he pushes a patch.
This idea fully achieve most of our initial goals: * (very) easy to implement * it will be easier to test a patch : is there an atomic update in the patch ? OK, to test it, i'll first execute the file * there won't be any conflict anymore of a patch applied and your .XXXX file not applying anymore, needing a rebase * it fixes partially the linear problem = if you have a local branch, you can see what has been applied just looking in the yaml file. Let's say Limoges library has sponsored stuff that has resulted in atomicupdate/limogeupdate.pl Those changes are live in Limoges, but still not in official/community version. The Limoges YAML file will be: 3.07.0001 Community_change1.pl 3.07.0002 community_change2.pl localchange limogesupdate.pl 3.07.0003 community_change3.pl the "localchange" in first column will mean for the updater : "OK, apply the change but don't change the version number"
When the localchange has been applied on master community branch, the official/community YAML file will look like: 3.07.0001 Community_change1.pl 3.07.0002 community_change2.pl 3.07.0003 community_change3.pl 3.07.0004 limogesupdate.pl
Switching back limoges to a community version would require some manual checking, but that will be *much* easier than what we have today : * diff limoges.yaml official.yaml * apply community changes if/where needed * UPDATE systempreferences SET kohaversion=3.07.0004 manually It's done !
And if the localchange that has been made is never merged into master, whatever the reason, it will be easier to keep track of it too.
And if you want to apply a submitted patch to your local setup before it's pushed onto master, you can, just by running the file in atomicupdate directory. And add a "localchange" line to your YAML file to remember the change you've made.
Does anyone see a problem with this new updatedatabase procedure ? Until now, I have pushed no changes in database, so if we agree, i'll implement this change quickly (before pushing any change having a database consequence)
-- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
* there won't be any conflict anymore of a patch applied and your .XXXX file not applying anymore, needing a rebase Ok there will be no more conflicts in the updatedatabase.pl file, but what about the YAML file? It still the same problem. If you have versions 001 and 002, and you add a localversion after them, and then a new version 003 is pushed into master. If you try to apply the patch,
It seems to be a great evolution, but there is something which I think I don't understand: there will be a conflit, isn't it? -- Julian Maurice<julian.maurice@biblibre.com> BibLibre
Le 05/11/2011 08:32, Julian Maurice a écrit :
* there won't be any conflict anymore of a patch applied and your .XXXX file not applying anymore, needing a rebase Ok there will be no more conflicts in the updatedatabase.pl file, but what about the YAML file? It still the same problem. If you have versions 001 and 002, and you add a localversion after them, and then a new version 003 is pushed into master. If you try to apply the patch,
It seems to be a great evolution, but there is something which I think I don't understand: there will be a conflit, isn't it?
The idea is that only the RM will take care of the YAML file. If you want to submit a DB change, just create a file in atomicupdate directory. If you want to test a patch, just check if there is something about atomicupdate in the patch, and if yes, run "atomicupdate/blabla.pl" before testing => no possible conflict here (except if 2 ppl use the same name for the file, that's pretty impossible) If you have a patch that must be applied to your customer before it is pushed in Koha : - apply atomicupdate/blabla.pl - add "localchange blabla.pl" at the end of your YAML file = it will keep track of when you've added the change, once the patch is in Koha, check the diff just of the YAML file, to know what has been applied and what should be applied. Much much easier than what we have today ! In case there is a conflict because of a local change, the conflict will be only on the YAML file, and will be very easy to deal with, as each entry is just one line. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
Another way to say it: No patches will be accepted with edits to the YAML file. also If you edit your local copy of the YAML file, do not assign a number; use localchange instead. Otherwise you will encounter problems. -Ian On Sat, Nov 5, 2011 at 4:35 AM, Paul Poulain <paul.poulain@biblibre.com>wrote:
Le 05/11/2011 08:32, Julian Maurice a écrit :
* there won't be any conflict anymore of a patch applied and your .XXXX file not applying anymore, needing a rebase Ok there will be no more conflicts in the updatedatabase.pl file, but what about the YAML file? It still the same problem. If you have versions 001 and 002, and you add a localversion after them, and then a new version 003 is pushed into master. If you try to apply the patch,
It seems to be a great evolution, but there is something which I think I don't understand: there will be a conflit, isn't it?
The idea is that only the RM will take care of the YAML file.
If you want to submit a DB change, just create a file in atomicupdate directory. If you want to test a patch, just check if there is something about atomicupdate in the patch, and if yes, run "atomicupdate/blabla.pl" before testing => no possible conflict here (except if 2 ppl use the same name for the file, that's pretty impossible) If you have a patch that must be applied to your customer before it is pushed in Koha : - apply atomicupdate/blabla.pl - add "localchange blabla.pl" at the end of your YAML file = it will keep track of when you've added the change, once the patch is in Koha, check the diff just of the YAML file, to know what has been applied and what should be applied. Much much easier than what we have today !
In case there is a conflict because of a local change, the conflict will be only on the YAML file, and will be very easy to deal with, as each entry is just one line.
-- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
Le 05/11/2011 15:02, Ian Walls a écrit :
Another way to say it:
No patches will be accepted with edits to the YAML file.
also
If you edit your local copy of the YAML file, do not assign a number; use localchange instead. Otherwise you will encounter problems.
-Ian
On Sat, Nov 5, 2011 at 4:35 AM, Paul Poulain <paul.poulain@biblibre.com <mailto:paul.poulain@biblibre.com>> wrote:
Le 05/11/2011 08:32, Julian Maurice a écrit : > It seems to be a great evolution, but there is something which I think I > don't understand: >> * there won't be any conflict anymore of a patch applied and your .XXXX >> file not applying anymore, needing a rebase > Ok there will be no more conflicts in the updatedatabase.pl <http://updatedatabase.pl> file, but > what about the YAML file? It still the same problem. If you have > versions 001 and 002, and you add a localversion after them, and then a > new version 003 is pushed into master. If you try to apply the patch, > there will be a conflit, isn't it? > The idea is that only the RM will take care of the YAML file.
If you want to submit a DB change, just create a file in atomicupdate directory. If you want to test a patch, just check if there is something about atomicupdate in the patch, and if yes, run "atomicupdate/blabla.pl <http://blabla.pl>" before testing => no possible conflict here (except if 2 ppl use the same name for the file, that's pretty impossible) If you have a patch that must be applied to your customer before it is pushed in Koha : - apply atomicupdate/blabla.pl <http://blabla.pl> - add "localchange blabla.pl <http://blabla.pl>" at the end of your YAML file = it will keep track of when you've added the change, once the patch is in Koha, check the diff just of the YAML file, to know what has been applied and what should be applied. Much much easier than what we have today !
In case there is a conflict because of a local change, the conflict will be only on the YAML file, and will be very easy to deal with, as each entry is just one line.
Ok I got it, thank you. ;-) -- Julian Maurice<julian.maurice@biblibre.com> BibLibre
During the hackfest in Mumbaï, 5 of us had a brainstorming about database changes to try to improve it. Would it be feasible for the database not to track a linear version number of the update status, but a list of tickets, one for every DB update? Given these tickets are unique (centrally managed or random, i.e., a hash of the DB statements) you would have no troubles whatsoever applying patches in any order. You would even (with a corresponding un-apply DB statements file) be able to roll back individual patches. Of course, this means the update has to walk through the ticket numbers instead of checking a single version number. In case that's too expensive for the general case of a non-patched system, the update script could be taught a list of tickets of the updates that each released version contains. But I guess looking up a ticket number in a list maintained by the DB is exactly what a DB is good at.
Since right now we check that the DB is up to date with every page load, we'd need something fast. A numeric comparison is simple enough; checking through a whole directory of updates, or scanning a hashref or some other mechanism, would likely take much more processing power. Also, we've got to factor in update's depending on one another. We've got to have some kind of chain of dependence; doing it as a linear value is the easiest. The other option would be for each update to specify it's dependencies explicitly; that would give us a directed acyclic graph, like Git uses, but would be MUCH more complex to implement and maintain. -Ian On Sat, Nov 5, 2011 at 6:01 AM, Edgar Fuß <ef@math.uni-bonn.de> wrote:
During the hackfest in Mumbaï, 5 of us had a brainstorming about database changes to try to improve it. Would it be feasible for the database not to track a linear version number of the update status, but a list of tickets, one for every DB update? Given these tickets are unique (centrally managed or random, i.e., a hash of the DB statements) you would have no troubles whatsoever applying patches in any order. You would even (with a corresponding un-apply DB statements file) be able to roll back individual patches. Of course, this means the update has to walk through the ticket numbers instead of checking a single version number. In case that's too expensive for the general case of a non-patched system, the update script could be taught a list of tickets of the updates that each released version contains. But I guess looking up a ticket number in a list maintained by the DB is exactly what a DB is good at.
Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
Paul Poulain <paul.poulain@biblibre.com>
Let's say Limoges library has sponsored stuff that has resulted in atomicupdate/limogeupdate.pl [...] Does anyone see a problem with this new updatedatabase procedure ?
The only problem that I can see is that we should encourage updates to be .sql not .pl files. Other than that, it's a great leap forwards and removes one of the main blockers in contributing database-related changes back to koha-community! Thanks, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. Webmaster, Debian Developer, Past Koha RM, statistician, former lecturer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for various work through http://www.software.coop/
On 2011-11-8, at 3:42 AM, MJ Ray wrote:
Paul Poulain <paul.poulain@biblibre.com>
Let's say Limoges library has sponsored stuff that has resulted in atomicupdate/limogeupdate.pl [...] Does anyone see a problem with this new updatedatabase procedure ?
The only problem that I can see is that we should encourage updates to be .sql not .pl files.
oooh, great idea! i think this would simplify the proposed process even more do other people agree?
I'd like to see the atomic update files continue to be perl, not SQL. SQL is often sufficient to do an update, but not always, particular if we're swapping out one implementation of a feature for a more generic one. Also, I'd like to see 3 different subroutines/functions for each atomic update: Check: Logic to see if the change is actually required: returns 0 or 1 Do: just like what have now, makes the change Undo: undoes the change to the database, allowing rollback to previous code states (this would be a new function) -Ian 2011/11/9 Mason James <mtj@kohaaloha.com>
On 2011-11-8, at 3:42 AM, MJ Ray wrote:
Paul Poulain <paul.poulain@biblibre.com>
Let's say Limoges library has sponsored stuff that has resulted in atomicupdate/limogeupdate.pl [...] Does anyone see a problem with this new updatedatabase procedure ?
The only problem that I can see is that we should encourage updates to be .sql not .pl files.
oooh, great idea! i think this would simplify the proposed process even more
do other people agree?
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
2011/11/10 Ian Walls <ian.walls@bywatersolutions.com>:
Undo: undoes the change to the database, allowing rollback to previous code states (this would be a new function)
This could get very tricky if the update also made changes to existing data. It seems there would be the need for a sort of "undelete" table/file/foo. Otherwise, the whole idea seems sound to me. Kind Regards, Chris
Ian Walls <ian.walls@bywatersolutions.com>
I'd like to see the atomic update files continue to be perl, not SQL. SQL is often sufficient to do an update, but not always, particular if we're swapping out one implementation of a feature for a more generic one.
Most things can be done with some combination of ALTER, UPDATE and occasionally temporary tables. Has any update ever absolutely needed perl? At worst, could we allow and encourage .sql files alongside .pl please? Thanks, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. http://koha-community.org supporter, web and LMS developer, statistician. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for Koha work http://www.software.coop/products/koha
Hi all ! I have already developed a new version of updatedatabase. It's not totally validated yet but in order to avoid parallels initiatives, let me explain you what I have already done. Work is on BibLibre git repository giit://git.biblibre.com/koha_biblibre.giton branch solr/ft/updatedb The goal was to have an interface to know status of different versions executed on a database. Requirements: on this branch, a precedent patch create a C4/Config/File/YAML.pm file. I use this module and you need an installdir section in your koha-conf.xml (<installdir>__INSTALL_BASE__</installdir>). How it works : 1 file per version (.pl ou .sql). if .sql, you could use comments, separator and query one after each others. if .pl please see skeleton.pl file. in admin/updatedatabase.pl, a table contains one line per version. For each line you can execute, see status (e.g. execution was ok or not (with errors)) or "force ok" (if you know theses queries don't need to be executed or have already been handled manually). You can't rollback a version. This tool only aim to have an interface showing status for each versions. Before execution, the routine C4::Update::Database::check_coherency check (for the most common queries) if the execution is coherent (if table or syspref already exists, etc.). You are welcome if you have any question or comment. Jonathan ps: I didn''t write Unit Tests :-/ 2011/11/10 Ian Walls <ian.walls@bywatersolutions.com>
I'd like to see the atomic update files continue to be perl, not SQL. SQL is often sufficient to do an update, but not always, particular if we're swapping out one implementation of a feature for a more generic one.
Also, I'd like to see 3 different subroutines/functions for each atomic update:
Check: Logic to see if the change is actually required: returns 0 or 1 Do: just like what have now, makes the change Undo: undoes the change to the database, allowing rollback to previous code states (this would be a new function)
-Ian
2011/11/9 Mason James <mtj@kohaaloha.com>
On 2011-11-8, at 3:42 AM, MJ Ray wrote:
Paul Poulain <paul.poulain@biblibre.com>
Let's say Limoges library has sponsored stuff that has resulted in atomicupdate/limogeupdate.pl [...] Does anyone see a problem with this new updatedatabase procedure ?
The only problem that I can see is that we should encourage updates to be .sql not .pl files.
oooh, great idea! i think this would simplify the proposed process even more
do other people agree?
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Le 12/11/2011 13:00, Jonathan Druart a écrit :
Hi all ! I have already developed a new version of updatedatabase. It's not totally validated yet but in order to avoid parallels initiatives, let me explain you what I have already done. I just took one hour with Jonathan, that shows me what he did. It's already live for many of our customers, and I was not aware of this... more communication needed inside BibLibre :\
So, here is how it works: * each patch is stored in a file in a specific directory. It can be .pl or .sql, both are adressed * there is a new page on admin page, that displays, for each patch: - if it has been applied or no - the result, either failed or OK. If failed, the error is also available. You can decide to "force OK" a given patch to ignore the problem On this page, you can also apply all pending patches and/or apply them one by one. This is completly "delinearized" Those 2 points are what we have built during hackfest, but there are some differences: * there is no more automatic control of update needed. That's intended: when you update, you'll now have to go to admin/update page. This sounds quite logic to me: it's delinearized, so checking what has been applied/must be applied on everypage will be much too CPU consuming * patch numbering = in this patch, patches still have a number, thus they can be applied in order. but, as it's not linear, they can also be applied in non-linear order How does it work internally ? A table is added, that contains all applied patches, with the logging of the result. It also contains a md5sum of the applied patches, to detect a duplicate number-but-not-the-same Jonathan has sent me the patch, I'll test & submit it soon. It relies on jquery datatable, so the patch will require bug 6836 to be applied first. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Le 09/11/2011 18:54, Mason James a écrit :
do other people agree? not at all ! Sometimes (not very often, but not rarely either), more complex calculations must be made, like for
For example: depending on your marcflavour you do something or something else.
$DBversion = '3.05.00.017'; if (C4::Context->preference("Version") < TransformToNum($DBversion)) { if (C4::Context->preference("marcflavour") eq 'MARC21' || C4::Context->preference("marcflavour") eq 'NORMARC'){ $dbh->do("INSERT INTO `marc_subfield_structure` (`tagfield`, `tagsubfield`, `liblibrarian`, `libopac`, `repeatable`, `mandatory`, `kohafield`, `tab`, `authorised_value` , `authtypecode`, `value_builder`, `isurl`, `hidden`, `frameworkcode`, `seealso`, `link`, `defaultvalue`) VALUES ('773', '0', 'Host Biblionumber', 'Host Biblionumber', 0, 0, NULL, 7, NULL, NULL, '', NULL, -6, '', '', '', NULL)"); $dbh->do("INSERT INTO `marc_subfield_structure` (`tagfield`, `tagsubfield`, `liblibrarian`, `libopac`, `repeatable`, `mandatory`, `kohafield`, `tab`, `authorised_value` , `authtypecode`, `value_builder`, `isurl`, `hidden`, `frameworkcode`, `seealso`, `link`, `defaultvalue`) VALUES ('773', '9', 'Host Itemnumber', 'Host Itemnumber', 0, 0, NULL, 7, NULL, NULL, '', NULL, -6, '', '', '', NULL)"); print "Upgrade to $DBversion done (Add 773 subfield 9 and 0 to default framework)\n"; SetVersion ($DBversion); } elsif (C4::Context->preference("marcflavour") eq 'UNIMARC'){ $dbh->do("INSERT INTO `marc_subfield_structure` (`tagfield`, `tagsubfield`, `liblibrarian`, `libopac`, `repeatable`, `mandatory`, `kohafield`, `tab`, `authorised_value` , `authtypecode`, `value_builder`, `isurl`, `hidden`, `frameworkcode`, `seealso`, `link`, `defaultvalue`) VALUES ('461', '9', 'Host Itemnumber', 'Host Itemnumber', 0, 0, NULL, 7, NULL, NULL, '', NULL, -6, '', '', '', NULL)"); print "Upgrade to $DBversion done (Add 461 subfield 9 to default framework)\n"; SetVersion ($DBversion); }
So we must have a .pl file, not SQL - -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOxTmUAAoJEK81SonuhyGoIscH/i1yDbz+viO/Qcy+y5h/Ttlw tyc2xqWlCY0ph4XE7TyE5sz6PBWB57tg0KZGPAz8ubCd3AThUkzTgU33rrRLOkg6 zIguyp9GOXH8VDWGtttnXGCMJlLqKp3wRargBZHOIRGBJVUjDkhiat1X/feqlFyr 3V3LT+H9sg+mWGSOhhbd1xRbd6XtiLBccopMllcz/bzqq133HuMbfS7Au/qw9bYF DpIrFCoJgiVM3fyUwqSpDe9rkq+sTIbLw3KN6B17lSWjUqOHCYJKtcXIskUM0fLk 1RaNqXtob9ZvvY6VibjHnBUTgsZApGzt6NRE2tvy4d/HJXUmXHD70KVebw8WHek= =F8G6 -----END PGP SIGNATURE-----
On 2011-11-18, at 5:43 AM, Paul Poulain wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Le 09/11/2011 18:54, Mason James a écrit :
do other people agree? not at all ! Sometimes (not very often, but not rarely either), more complex calculations must be made, like for
For example: depending on your marcflavour you do something or something else.
$DBversion = '3.05.00.017'; if (C4::Context->preference("Version") < TransformToNum($DBversion)) { if (C4::Context->preference("marcflavour") eq 'MARC21' || C4::Context->preference("marcflavour") eq 'NORMARC'){
...
So we must have a .pl file, not SQL - -- Paul POULAIN
yes, yes... a good point Paul! i think if we want to do Ian's suggested check/apply/roll-back idea for DB changes, we will indeed need to use .pl files not .sql files for new patches so, i change my mind on this point :) Mason
Mason James <mtj@kohaaloha.com>
On 2011-11-18, at 5:43 AM, Paul Poulain wrote:
Le 09/11/2011 18:54, Mason James a écrit :
do other people agree? not at all ! Sometimes (not very often, but not rarely either), more complex calculations must be made, like for
For example: depending on your marcflavour you do something or something else.
$DBversion = '3.05.00.017'; if (C4::Context->preference("Version") < TransformToNum($DBversion)) { if (C4::Context->preference("marcflavour") eq 'MARC21' || C4::Context->preference("marcflavour") eq 'NORMARC'){
...
So we must have a .pl file, not SQL
The bit quoted above doesn't require a .pl file. SQL has conditionals.
yes, yes... a good point Paul!
i think if we want to do Ian's suggested check/apply/roll-back idea for DB changes, we will indeed need to use .pl files not .sql files for new patches
I'm not sure about that either. Surely it could be done by setting SQL session variables in the updatedatabase that loads the sql file and conditionals in the SQL? But anyway, what about supporting both? If I'm wrong and .sql has to be trivially converted to .pl one day, it's not that awful. If I'm right and we can eventually replace all .pl with .sql, that makes updating a lot cleaner and safer. Regards, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. Webmaster, Debian Developer, Past Koha RM, statistician, former lecturer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for various work through http://www.software.coop/
On 2011-11-19, at 8:18 AM, MJ Ray wrote:
Mason James <mtj@kohaaloha.com>
i think if we want to do Ian's suggested check/apply/roll-back idea for DB changes, we will indeed need to use .pl files not .sql files for new patches
I'm not sure about that either. Surely it could be done by setting SQL session variables in the updatedatabase that loads the sql file and conditionals in the SQL?
But anyway, what about supporting both? If I'm wrong and .sql has to be trivially converted to .pl one day, it's not that awful. If I'm right and we can eventually replace all .pl with .sql, that makes updating a lot cleaner and safer.
Regards, -- MJ Ray
ok, i can easily imagine future situations where .sql patch files alone just *won't* work for upgrades (lets ignore Paul's previous example) the situation i am imagining is where a patch requires a database change to made via say... an internal Koha subroutine(). (like an update made to all bib or item records, via calls to the ModBiblio() or ModItem() subroutines) now.. how could an .SQL only solution achieve this? - its impossible, right? - using .pl patches, with a *combination* of perl and sql, would allow us the best of both options - using .sql patches alone, means a patch won't be able to use any internal Koha subroutines? i realize i was originally keen on an .sql solution... but after more thought, i believe an .sql solution will be too limited for some upgrade tasks using perl, we wont *ever* bump into this potential limitation. do i have a valid point here? cheers, Mason
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Le 20/11/2011 01:09, Mason James a écrit :
now.. how could an .SQL only solution achieve this? - its impossible, right?
- using .pl patches, with a *combination* of perl and sql, would allow us the best of both options - using .sql patches alone, means a patch won't be able to use any internal Koha subroutines?
Ian (& all): see my mail about what Jonathan has already written and that I should submit quickly (today ?) : he deals with both .sql and .pl, so we get the best of both worlds !!! (the updater detect the extension and work accordingly) HTH - -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOygM/AAoJEK81SonuhyGo/G8H/3qJevFE40LXY9ei20toW720 TIMl8G2VSSFfIidSWDJW5B8z1EcNX2QUTMsWr0w5OIvpNgxTrWzcxBOox1DCfM4t W5ZhN39L0H4LuEmzIO/8Anmlsw4q2jDyUzFyJJcJf3dcag99tJDM1Pl1PP/nXG+b EbEkox73K7EEx1PNHsHea6VdcpZYThs+0OKRGsXdRFTOlYEMj4Im6lb/hUi3Q3P9 1rgrM3Y2EtAZ81z+1opkooIaVyjwCNw+cjNJUtZhblBMX6w7MtFkir2ne7oegCh4 CjiqQ++PaP1AT5Z7jz2cwvD6yKEVdoHG3JXQE3LaH4sFSDZaFLbKp9I9kNgutEk= =fUJh -----END PGP SIGNATURE-----
Quote from Bugzilla #7167 Here is how it works: * each database update is stored in a numbered file, in installer/data/mysql/versions * database updates can be .sql or .pl files. 2 skeletons are provided to explain how it works * there is no more automatic checking of the database update on each page. The librarian/sysadmin must go to the admin/updatedatabase.pl page on each update. * the updatedatabase.pl script keep track of all updates that have been applied, how it went. And it's non-linear: 3.7.2.3 can be applied after 3.7.2.4 * The about.pl will display the highest db update applied, but maybe there are some missing, it's only the highest applied [End of quote] My questions are now: How do I submit a new db update? I cannot give it a number myself; in that case somebody else could have a patch pending using that number already. So I assume that the RM renames the file. What convention do we use? Is there still a check at login time if all updates have been run which redirects to the update screen? I would say that we do not need that check everywhere, but I would keep it at login time. Would you allow an admin to install only partially the updates? Isn't that asking for trouble? It is not linear, but some patches with db updates will be sensitive to the order applied. Is there any logic to prevent problems in that area? Marcel ________________________________________ Van: koha-devel-bounces@lists.koha-community.org [koha-devel-bounces@lists.koha-community.org] namens Paul Poulain [paul.poulain@biblibre.com] Verzonden: maandag 21 november 2011 8:52 Aan: koha-devel@lists.koha-community.org Onderwerp: Re: [Koha-devel] Update database changes proposal [IMPORTANT] -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Le 20/11/2011 01:09, Mason James a écrit :
now.. how could an .SQL only solution achieve this? - its impossible, right?
- using .pl patches, with a *combination* of perl and sql, would allow us the best of both options - using .sql patches alone, means a patch won't be able to use any internal Koha subroutines?
Ian (& all): see my mail about what Jonathan has already written and that I should submit quickly (today ?) : he deals with both .sql and .pl, so we get the best of both worlds !!! (the updater detect the extension and work accordingly) HTH - -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOygM/AAoJEK81SonuhyGo/G8H/3qJevFE40LXY9ei20toW720 TIMl8G2VSSFfIidSWDJW5B8z1EcNX2QUTMsWr0w5OIvpNgxTrWzcxBOox1DCfM4t W5ZhN39L0H4LuEmzIO/8Anmlsw4q2jDyUzFyJJcJf3dcag99tJDM1Pl1PP/nXG+b EbEkox73K7EEx1PNHsHea6VdcpZYThs+0OKRGsXdRFTOlYEMj4Im6lb/hUi3Q3P9 1rgrM3Y2EtAZ81z+1opkooIaVyjwCNw+cjNJUtZhblBMX6w7MtFkir2ne7oegCh4 CjiqQ++PaP1AT5Z7jz2cwvD6yKEVdoHG3JXQE3LaH4sFSDZaFLbKp9I9kNgutEk= =fUJh -----END PGP SIGNATURE----- _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Le 23/11/2011 17:02, Marcel de Rooy a écrit :
Quote from Bugzilla #7167 Here is how it works: * each database update is stored in a numbered file, in installer/data/mysql/versions * database updates can be .sql or .pl files. 2 skeletons are provided to explain how it works * there is no more automatic checking of the database update on each page. The librarian/sysadmin must go to the admin/updatedatabase.pl page on each update. * the updatedatabase.pl script keep track of all updates that have been applied, how it went. And it's non-linear: 3.7.2.3 can be applied after 3.7.2.4 * The about.pl will display the highest db update applied, but maybe there are some missing, it's only the highest applied [End of quote]
My questions are now: How do I submit a new db update? just create a file under installer/data/mysql/versions
I cannot give it a number myself; in that case somebody else could have a patch pending using that number already. So I assume that the RM renames the file. What convention do we use? yes you can : as it's not linear, you can. To know which numbers have been "reserved", we could have a wiki page. If you've reserved 3.07.01.017 and someone has a 3.07.01.018, the 018 can be pushed *before* your 017, there's no problem with that.
Is there still a check at login time if all updates have been run which redirects to the update screen? no. Because it takes a long time to check all versions, and, as it's unlinearized, you must check each version, not just check you've the "highest number"
I would say that we do not need that check everywhere, but I would keep it at login time. This idea could be investigated for someone that log-in with admin permission, for example. Does others think it's a must-have ? Here at BibLibre, most of us think checking the database update is a part of an update, so if it's not made, it means you're doing a poor job.
Would you allow an admin to install only partially the updates? Isn't that asking for trouble? yes, you could. Even if, I agree, that would be a strange idea. And that's why there is a [UPDATE] on the top that update everything
It is not linear, but some patches with db updates will be sensitive to the order applied. Is there any logic to prevent problems in that area? Nope, and we discussed a lot of this during hackfest, there is no solution for this. Plus, investigating actual updatedatabase show only a few cases that could cause a problem. 90% of the updates are syspref/index/foreign keys related.
HTH - -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOzSVIAAoJEK81SonuhyGo4JAIAJWxr3IMX8OgxDU2a4QNgwQD BRcKqNQDTG3YJneW7CRh959wglzk2gABQdiKcZqoO0XrAKNnqxaEinrpWbbTq63p 4yev1h8A1RFYwMYjtHSsFtdJFDWBf6XWPzHkl5gscPohPbFWteLBqeNePgTxXkh+ D4dPi6PG1HPK3iAt75tv26zruLA3Zn9DsML1Zhv0hvLqgsuFVobmXFA8R4OGRawm CGaPFD5NZW1PfHjqNdAC4Bfb0hrqd59q1+bTlsDcd2gJrePKelDmC8Yge2S9n3GN v/g+vWq0Vmmd1fdEfFWkui60iw9XJizKA0JFOpcq1Dzc5Z+OzQMelkxyHcYLtkM= =xN9m -----END PGP SIGNATURE-----
Paul Poulain <paul.poulain@biblibre.com>
I would say that we do not need that check everywhere, but I would keep it at login time. This idea could be investigated for someone that log-in with admin
Le 23/11/2011 17:02, Marcel de Rooy a écrit : permission, for example. Does others think it's a must-have ? Here at BibLibre, most of us think checking the database update is a part of an update, so if it's not made, it means you're doing a poor job.
I think it may be a good idea. Not only experts like you and me install Koha. Some hobbyists and accidental tech workers are doing it for themselves. What would be very cool is if the version number was computed somehow from the updates that had been applied (which may make it go nonlinear) in a way that we could see what been applied. One possibility would be for the RM to give each update a prime number and we just sum them, but the version number might get verrrry big. Can anyone do better or should we junk that idea? Regards, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. Webmaster, Debian Developer, Past Koha RM, statistician, former lecturer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for various work through http://www.software.coop/
MJ Ray schreef op wo 23-11-2011 om 22:46 [+0000]:
Can anyone do better or should we junk that idea?
I tend to agree with you here, I think. Perhaps a quick hash/CRC of the filenames of the patches that have been applied? Should be quick to generated and test, and will tell us immediately if we need to run again. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D
You know what this thread is missing? 2 cents from yours truly!
Perhaps a quick hash/CRC of the filenames of the patches that have been applied? Should be quick to generated and test, and will tell us immediately if we need to run again.
+1 from me My vote is for using a hash for identifying updates in every situation. Preferable to file names or hardcoded version numbers, in my view. -- Jared Camins-Esakov Bibliographer, C & P Bibliography Services, LLC (phone) +1 (917) 727-3445 (e-mail) jcamins@cpbibliography.com (web) http://www.cpbibliography.com/
2011/11/23 Jared Camins-Esakov <jcamins@cpbibliography.com>:
You know what this thread is missing? 2 cents from yours truly!
Perhaps a quick hash/CRC of the filenames of the patches that have been applied? Should be quick to generated and test, and will tell us immediately if we need to run again.
+1 from me
+1 here as well. Kind Regards, Chris
Robin Sheat <robin@catalyst.net.nz>
Perhaps a quick hash/CRC of the filenames of the patches that have been applied? Should be quick to generated and test, and will tell us immediately if we need to run again.
That was another idea, but I thought a sufficient hash would be a very long version number from the start and also would not necessarily increase as updates are applied, or tell us simply which updates have been applied when people ask for help, so I went with multiplying primes. But both have their merits. Hash, CRC or primes are fine by me. Regards, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. Webmaster, Debian Developer, Past Koha RM, statistician, former lecturer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for various work through http://www.software.coop/
We were discussing the following at hackfest: a directory of atomic database updates. While my intent was for .pl files, we could adapt to include .sql, as well. I just really like the idea of a CHECK/DO/UNDO interface A YAML file, edited by the RM only, that assigns those update files to a specific database revision number. This gives us a quick and ordered means of determining what has been "blessed" by the release team, and allow us to continue a quick check of "is DB version equal to software version?" on every page. If a user needs to make a local change, they can either apply the atomic update file directly (for testing), or add it to the YAML file, but with a placeholder string like "LOCALNUMBER" instead of an actual DB rev number. This would allow people to put together their own database update combinations for local developments, without getting their DB version number out of whack. The script that reads the YAML file and applies the update can be called just in the event that DB number != Code number. This would require that the sysadmin manually add the atomic update the first time, but if they're applying patches that require DB updates, they should be savvy enough to complete that small additional step. I do like the idea of hashes... but looking at it, we'd essentially be mimicking the data structure we already have with Git commits. Plus, readability goes down. Cheers, -Ian On Thu, Nov 24, 2011 at 5:37 AM, MJ Ray <mjr@phonecoop.coop> wrote:
Robin Sheat <robin@catalyst.net.nz>
Perhaps a quick hash/CRC of the filenames of the patches that have been applied? Should be quick to generated and test, and will tell us immediately if we need to run again.
That was another idea, but I thought a sufficient hash would be a very long version number from the start and also would not necessarily increase as updates are applied, or tell us simply which updates have been applied when people ask for help, so I went with multiplying primes.
But both have their merits. Hash, CRC or primes are fine by me.
Regards, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. Webmaster, Debian Developer, Past Koha RM, statistician, former lecturer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for various work through http://www.software.coop/ _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Ian Walls Lead Development Specialist ByWater Solutions Phone # (888) 900-8944 http://bywatersolutions.com ian.walls@bywatersolutions.com Twitter: @sekjal
Op 25-11-11 00:49, Ian Walls schreef:
I do like the idea of hashes... but looking at it, we'd essentially be mimicking the data structure we already have with Git commits. Plus, readability goes down.
Is readability an issue here? I was thinking a process like: 1) on each page (and this'll improve heaps when we get persistence :) the list of patch files is build, concated, CRCed. 2) if that value is different from what's stored in the DB, we can then do a more expensive process to work out what has and hasn't been applied. 3) apply the ones we need to, note that however (exercise for reader:) 4) save the hash generated in (1) into the DB 5) ??? 6) Profit! Just an idea, please take and rework as you see fit, or not. I'm a fan of some kind of check for every page load (or, perhaps just every "main" page load, e.g. the front page of each module - no check on the speed critical pages would be good) though, otherwise people _will_ get caught out and confused. Another option may be saving the date of the most recent patch file, however I can see issues there with local customisations on top of a tarball or something. Also, git isn't good at keeping timestamps intact. But basically, a quick check to see "do we need to?" followed by "yes we do, lets spend 10 seconds working out the details because it's OK to do that now" kind of thing. Robin.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Le 24/11/2011 13:22, Robin Sheat a écrit : > Op 25-11-11 00:49, Ian Walls schreef: >> I do like the idea of hashes... but looking at it, we'd >> essentially be mimicking the data structure we already have with >> Git commits. Plus, readability goes down. > > Is readability an issue here? I was thinking a process like: > > 1) on each page (and this'll improve heaps when we get persistence > :) the list of patch files is build, concated, CRCed. There are 2 strong opinions here: * upgrade is a process you know you're doing, so no need to check on every page * upgrade is safer if you check often if you're uptodate: the computer must fix human mistakes. Maybe a middle solution would be to have do a check on the login page only (or on mainpage.pl ?). As it's mandatory to log-in on the staff interface, that seems fair. > Perhaps a quick hash/CRC of the filenames of the patches that have > been applied? Should be quick to generated and test, and will tell > us immediately if we need to run again. In a few months, we will have 40 different files. And hashing/CRCing 40 files is not negligible. We want to improve perfs, so I think it's a wrong idea. On the login page though, I think it's something we can afford. (for those who are not reading Bug 6328 (fines in days) and updatedatabase, I'll also add comments here, and both threads are related) - -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJOzmCvAAoJEK81SonuhyGoizkH/RLfsfNzC/KaWCHk37YUXI8q Hq86OuwibnT1pdgnn3XY9QkDEiwJRXIh8kjZDQNJJhdJhqwIZvL27vnwon/ZBMJT No8RM8efuTYKTDvCbNDRSlMtqLXgRhjzfDPJygPZEcYipgRqfq/z1nDvB1dHmnTT eWNQpQpLu2KX/s0Oduj3PPzln8Ri/nWsWQDgFSTn0/62IQ+ibmSo98TjyjYWZ2Kb NotSe+MGLLfIHGaf7WpZt6GitYzJA26sWiGFYJI4wd55bm/dapwB5r1ML30WRTm7 6xTCmjRyX/0wZt2gpI9TMmoL7ZKF17TryAWSgNK5/L/Ub86QKfbcNG525CZAR3w= =/Cjt -----END PGP SIGNATURE-----
On Thu, Nov 24, 2011 at 10:20 AM, Paul Poulain <paul.poulain@biblibre.com> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Le 24/11/2011 13:22, Robin Sheat a écrit : >> Op 25-11-11 00:49, Ian Walls schreef: >>> I do like the idea of hashes... but looking at it, we'd >>> essentially be mimicking the data structure we already have with >>> Git commits. Plus, readability goes down. >> >> Is readability an issue here? I was thinking a process like: >> >> 1) on each page (and this'll improve heaps when we get persistence >> :) the list of patch files is build, concated, CRCed. > There are 2 strong opinions here: > * upgrade is a process you know you're doing, so no need to check on > every page > * upgrade is safer if you check often if you're uptodate: the computer > must fix human mistakes. > > Maybe a middle solution would be to have do a check on the login page > only (or on mainpage.pl ?). As it's mandatory to log-in on the staff > interface, that seems fair. I tend to agree with Paul here. Checking at login for up-to-date deps seems to be the best way to go, introducing the least overall latency from the user's prospective. > >> Perhaps a quick hash/CRC of the filenames of the patches that have >> been applied? Should be quick to generated and test, and will tell >> us immediately if we need to run again. > > In a few months, we will have 40 different files. And hashing/CRCing > 40 files is not negligible. We want to improve perfs, so I think it's > a wrong idea. On the login page though, I think it's something we can > afford. Of course, we could always relegate the hashing/CRCing *and* update check to a cron job which runs once very <a-time-with-low-system-load> and gives an appropriate status indication on mainpage.pl. Kind Regards, Chris
Op 25-11-11 04:20, Paul Poulain schreef:
In a few months, we will have 40 different files. And hashing/CRCing 40 files is not negligible.
File_names_, not files. And CRCing on string of 40 filenames is pretty negligible. I just want to avoid people trucking along with new code on an old database schema. I think there's a risk of something bad happening. Robin.
Op 26-11-11 11:15, BWS Johnson schreef:
Any reason not to do it at logoff in the background?
People may not logoff, also they may not see errors/warnings if they go away immediately. That said, if you do it on the login screen, which is where the logoff screen takes you, it's really the same thing :) Robin.
Ian Walls <ian.walls@bywatersolutions.com> [...]
We were discussing the following at hackfest:
:-( but thank you for documenting it now. [...]
A YAML file, edited by the RM only, that assigns those update files to a specific database revision number. This gives us a quick and ordered means of determining what has been "blessed" by the release team, and allow us to continue a quick check of "is DB version equal to software version?" on every page.
I don't understand what this gains us. What does the revision number mean? That a particular update has been applied, or that all updates up to and including that point in the numbering file have been applied?
If a user needs to make a local change, they can either apply the atomic update file directly (for testing), or add it to the YAML file, but with a placeholder string like "LOCALNUMBER" instead of an actual DB rev number. This would allow people to put together their own database update combinations for local developments, without getting their DB version number out of whack.
So how could we infer anything from the DB version number in problem reports? Thanks for any answers anyone could give. Confused, -- MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op. Webmaster, Debian Developer, Past Koha RM, statistician, former lecturer. In My Opinion Only: see http://mjr.towers.org.uk/email.html Available for hire for various work through http://www.software.coop/
I would say that we do not need that check everywhere, but I would keep it at login time. This idea could be investigated for someone that log-in with admin permission, for example. Does others think it's a must-have ? Here at BibLibre, most of us think checking the database update is a part of an update, so if it's not made, it means you're doing a poor job.
I understand that (poor job-argument) for a production database, but for a test db on latest master (fetching new commits all the time) it would be very comfortable if Koha could check if I am missing an update instead of checking it manually and forgetting to do so.. If the check is done via some kind of hash function, it should not take long time.
On 2011-11-21, at 8:52 PM, Paul Poulain wrote:
Le 20/11/2011 01:09, Mason James a écrit :
now.. how could an .SQL only solution achieve this? - its impossible, right?
- using .pl patches, with a *combination* of perl and sql, would allow us the best of both options - using .sql patches alone, means a patch won't be able to use any internal Koha subroutines?
Ian (& all): see my mail about what Jonathan has already written and that I should submit quickly (today ?) : he deals with both .sql and .pl, so we get the best of both worlds !!! (the updater detect the extension and work accordingly)
hi Paul ok, this is perfect! - now i shut up :)
Hello, I just have submitted on bug 7167 3 patches that are working as expected. You can also get a document with screenshots at https://depot.biblibre.com/ppoulain/updatedb%20for%203.8.odt (can't upload it on bugzilla, it's too large: 1.6MB, vs a 1.0MB limit) (Note that the "testing patch" is here just for testing & validating) I think this code handle all the cases (ppl running 3.6, ppl upgrading to 3.8 when it's released, ppl running master), and is a major improvement for everybody: * for developers, it will be easier to submit patches * for users, they'll keep track of what has been applied and what has not. And get feedback if there is a problem Please have a look, test, and, if you think it's OK, sign-off. I worked hard on this in the last days. Also notice that I won't push any patch with an updatedatabase that should be in 3.8 until we've reached an agreement on this improvement (and that's why I considered this as needing to be done urgently) I'll push bugfixes that are for 3.6 without any problem, as the proposed new system won't change anything for 3.6 related db updates. Waiting for your feedback. -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
participants (12)
-
BWS Johnson -
Chris Nighswonger -
Edgar Fuß -
Ian Walls -
Jared Camins-Esakov -
Jonathan Druart -
Julian Maurice -
Marcel de Rooy -
Mason James -
MJ Ray -
Paul Poulain -
Robin Sheat