There's been talk of making a few conventions for the parameters to command-line scripts. The initial discussion was dealing with "test mode" where it runs through all the actions but doesn't actually do anything, i.e. you know if it'll complete successfully. 1. There are two directions for this: I. By default the action will be performed, unless you add an argument (--dry-run for example.) II. By default the action will not be performed, unless you add an argument (--commit for example.) 2. What do we want to call the options, whichever way is picked? I. Keeping in mind that some things (-c for example) can conflict with other conventions. 3. Are there any other conventions we should deal with at the same time? While I have your attention, we should be more careful about how the dry run stuff is actually done. I've found one script that gives you a different count if you're in dry run mode (e.g. it'll say "0 actions done") which is strictly correct, but means you don't know if it's 0 because something's wrong, or because it's in dry run mode. Unless there's a good reason, it's best to do everything in a commit, so something like: $dbi->begin_work; ...do stuff... print "$count actions done.\n"; if ($dry_run) { $dbi->rollback; print "Changes NOT committed.\n"; } else { $dbi->commit; } This also has the side effect that if your program crashes in any mode, you're not left with an inconsistent result. But mostly it means that /everything/ will happen in exactly the same way no matter what, except the results just won't be saved. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF
Greetings,
1. There are two directions for this: I. By default the action will be performed, unless you add an argument (--dry-run for example.) II. By default the action will not be performed, unless you add an argument (--commit for example.)
If you run a script called DestroyTheDatabase, you would expect the default action to be complete and utter deletion. I think a dry-run flag would make sense for this type of script. The problem is those scripts whose name is not indicative of DB modifications. I suspect you are thinking about checkNonIndexedBiblios.pl which only runs if you include -c, much like point 1 subsection II. The name doesn't suggest default modifications. In this case, you have to add the -z which will then insert the non-indexed Biblios into the zebra queue. This is like the second method. Can we have a mix? Does a mix make any sense? This kind of reminds me of the help for scripts issues I was wondering about. I see a script name, other than reading the code, how do I get a sample use page? Do we force help on users if no parameters are passed? Do we expect a particular letter or phrase? --help -h --usage --examples --something-else? checkNonIndexedBiblios.pl does it both ways.
2. What do we want to call the options, whichever way is picked? I. Keeping in mind that some things (-c for example) can conflict with other conventions.
Are there different conventions to choose from? How does Debian (Ubuntu, etc.) do it?
3. Are there any other conventions we should deal with at the same time?
None of which I am aware. [SNIP] +1 on the turning off auto-commit, so dry runs do the same as actual runs. +1 on the explicit rollback or commit.
$dbi->begin_work;
Actually, I guess that is a nicer way of doing this? # Start transaction $dbh->{AutoCommit} = 0; $dbh->{RaiseError} = 1; GPML, Mark Tompsett
Do we force help on users if no parameters are passed?
As a non-expert in command line things, this is what I would expect. At the very least a brief help, with the suggestion that passing --help would give more. -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org
Mark Tompsett schreef op do 18-09-2014 om 09:29 [-0400]:
not indicative of DB modifications. I suspect you are thinking about checkNonIndexedBiblios.pl which only runs if you include -c, much like point 1 subsection II. The name doesn't suggest default modifications.
Well the solution to that is to fix the name. It's not checking, it's modifying. To be fair, fsck (filesystem check) also modifies :)
In this case, you have to add the -z which will then insert the non-indexed Biblios into the zebra queue. This is like the second method. Can we have a mix? Does a mix make any sense?
A mix sounds like the worst of both possible worlds. I don't have strong opinions about it (though ceteris paribus would prefer having "commit" by default), but it's the sort of case where consistency would be good.
This kind of reminds me of the help for scripts issues I was wondering about. I see a script name, other than reading the code, how do I get a sample use page? Do we force help on users if no parameters are passed? Do we expect a particular letter or phrase? --help -h --usage --examples --something-else? checkNonIndexedBiblios.pl does it both ways.
If a script requires arguments in order to do its function, then it should present with a help synopsis. If it doesn't, then it might go forth and do its thing. This is the UNIX way. If you want help, you use -h or --help. This is the GNU way, which is mostly the POSIX way, which is mostly the UNIX way. I also tend to use --man for providing the full manual, but I'm not sure where I got that idea from. --version is also more or less a GNU requirement, but it's perhaps not so important for us.
2. What do we want to call the options, whichever way is picked? I. Keeping in mind that some things (-c for example) can conflict with other conventions.
Are there different conventions to choose from? How does Debian (Ubuntu, etc.) do it?
I did a bit of pottering around, and while there are many different things, the most common I saw was -c for config file. So it'd probably be best to avoid that. Other things are fair game (except --help and --version I guess.) My suggestion is that we use --dry-run|-d if we have an option to activate that mode, or --run|-r if we require an option to commit changes. Note that no matter what is decided, the "delete_everything_and_burn_it_all_down" script should require a parameter to commit changes. Perhaps '--screw-it-lets-do-it-live'.
$dbi->begin_work;
Actually, I guess that is a nicer way of doing this?
# Start transaction $dbh->{AutoCommit} = 0; $dbh->{RaiseError} = 1;
It wasn't really a complete example. RaiseError should always be 1 unless you have a reason, I think starting a transaction turns off AutoCommit for the duration of the transaction anyway. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF
Le 18/09/2014 03:51, Robin Sheat a écrit :
There's been talk of making a few conventions for the parameters to command-line scripts. (a little bit late answer...)
I've 2 strong opinions: * if the script does something that can't be rolled back, there *must* be a of validation. for example "delete_bibs_without_items.pl" must not delete anything without warning. This warning can be something like "you're about to delete 100000 bibliographic records. Are you sure Y/N", unless the script is run with a parameter (delete_bibs_without_items.pl -y) * we must choose an option, and make it valid for all scripts. Outside from those 2 things, I've no preference. I tend to be on Owen side : no parameter = small help -ie: explicit parameter needed to run the script) Additional question: some CLI are made for being run from cron. In this case, no information should be printed unless something goes wrong. SO: by default, should scripts be silent: * no parameters = silent mode * -v = some informations * -vv (or -v 2, or --very-verbose, I don't care ;-) ) = verbose mode or should there be an explicit "silent mode": * -s parameter = silent mode * no parameter = some informations * -v = verbose mode something for a future technical meeting and improved coding guidelines ?
The initial discussion was dealing with "test mode" where it runs through all the actions but doesn't actually do anything, i.e. you know if it'll complete successfully.
1. There are two directions for this: I. By default the action will be performed, unless you add an argument (--dry-run for example.) II. By default the action will not be performed, unless you add an argument (--commit for example.) 2. What do we want to call the options, whichever way is picked? I. Keeping in mind that some things (-c for example) can conflict with other conventions. 3. Are there any other conventions we should deal with at the same time?
While I have your attention, we should be more careful about how the dry run stuff is actually done. I've found one script that gives you a different count if you're in dry run mode (e.g. it'll say "0 actions done") which is strictly correct, but means you don't know if it's 0 because something's wrong, or because it's in dry run mode. Unless there's a good reason, it's best to do everything in a commit, so something like:
$dbi->begin_work; ...do stuff... print "$count actions done.\n"; if ($dry_run) { $dbi->rollback; print "Changes NOT committed.\n"; } else { $dbi->commit; }
This also has the side effect that if your program crashes in any mode, you're not left with an inconsistent result. But mostly it means that /everything/ will happen in exactly the same way no matter what, except the results just won't be saved.
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Paul Poulain, Associé-gérant / co-owner BibLibre, expert du logiciel libre pour les bibliothèques BibLibre, Open Source software for libraries expert
Paul Poulain schreef op di 21-10-2014 om 09:53 [+0200]:
I've 2 strong opinions: * if the script does something that can't be rolled back, there *must* be a of validation. for example "delete_bibs_without_items.pl" must not delete anything without warning. This warning can be something like "you're about to delete 100000 bibliographic records. Are you sure Y/N", unless the script is run with a parameter (delete_bibs_without_items.pl -y)
I quite like that idea actually. We'd also want some tests for running within a TTY etc so we don't end up blocking forever, but that's not hard.
Additional question: some CLI are made for being run from cron. In this case, no information should be printed unless something goes wrong. SO: by default, should scripts be silent: * no parameters = silent mode * -v = some informations * -vv (or -v 2, or --very-verbose, I don't care ;-) ) = verbose mode
-vvvvv is the standard way of doing such things, and very easy to do with Getopt::Long. I think this scheme is the best idea for things that are designed to be cron'ed, as opposed to:
or should there be an explicit "silent mode": * -s parameter = silent mode * no parameter = some informations * -v = verbose mode
-- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF
Le 18/09/2014 03:51, Robin Sheat a écrit :
While I have your attention, we should be more careful about how the dry run stuff is actually done. I've found one script that gives you a different count if you're in dry run mode (e.g. it'll say "0 actions done") which is strictly correct, but means you don't know if it's 0 because something's wrong, or because it's in dry run mode. Unless there's a good reason, it's best to do everything in a commit, so something like:
$dbi->begin_work; ...do stuff... print "$count actions done.\n"; if ($dry_run) { $dbi->rollback; print "Changes NOT committed.\n"; } else { $dbi->commit; } mmm... I agree with this point, BUT, for large changes, MySQL really does really not well with a huge single transactions. In this case, the only way is to do smaller transactions.
-- Paul Poulain, Associé-gérant / co-owner BibLibre, expert du logiciel libre pour les bibliothèques BibLibre, Open Source software for libraries expert
Paul Poulain schreef op di 21-10-2014 om 09:56 [+0200]:
mmm... I agree with this point, BUT, for large changes, MySQL really does really not well with a huge single transactions. In this case, the only way is to do smaller transactions.
I've never noticed that, but perhaps the things I'm usually doing aren't quite big enough. However, it's not a big deal to batch up (say) 1,000 operations and then commit/not commit them if necessary. It does lose you some element of error recovery, but at least you can still do a real dry-run test and it'll fail safely if something is wrong. Or we could run postgres :) -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF
Le 23/10/2014 01:07, Robin Sheat a écrit :
Paul Poulain schreef op di 21-10-2014 om 09:56 [+0200]:
mmm... I agree with this point, BUT, for large changes, MySQL really does really not well with a huge single transactions. In this case, the only way is to do smaller transactions.
I've never noticed that, but perhaps the things I'm usually doing aren't quite big enough. However, it's not a big deal to batch up (say) 1,000 operations and then commit/not commit them if necessary. It does lose you some element of error recovery, but at least you can still do a real dry-run test and it'll fail safely if something is wrong. Yep, 1000 is not a big deal. I'm talking here of 1 000 000 of mysql operations (on a large catalogue, it's easy to reach, with constraints. Something like DELETE from biblio WHERE title LIKE "A%" will also have an impact on biblioitems, which has an impact on items, which has an impact on issues, which has ...)
Or we could run postgres :) Troll detected :D :D
-- Paul Poulain, Associé-gérant / co-owner BibLibre, expert du logiciel libre pour les bibliothèques BibLibre, Open Source software for libraries expert
participants (4)
-
Mark Tompsett -
Owen Leonard -
Paul Poulain -
Robin Sheat