Inconsistent Perl shebangs (new coding guideline?)
Hi all, I was looking at "Bug 20582 - Turn Koha into a Mojolicious application" when I noticed that some of the scripts were using "#!/usr/bin/env perl" instead of "#!/usr/bin/perl". While there is no requirement in the Coding Guidelines to use "#!/usr/bin/perl", I thought it would be best for all our Perl scripts to be consistent. However, now I'm weighing up the options. Conceptually, using "#!/usr/bin/env perl" means that we can use perlbrew/plenv (or manually provide a different Perl in our $PATH) when running these scripts, so that we don't have to use the system Perl located at /usr/bin/perl. That said, I was concerned about how most CGI scripts have "#!/usr/bin/perl" in them and the implications that might have if we invoked Starman/Morbo/Hypnotoad with a different Perl, but looking at Plack::App::CGIBin, CGI::Emulate::PSGI, Plack::App::WrapCGI, CGI::Compile, and Koha::App::Plugin::CGIBinKoha makes me think that the shebang in CGI scripts is ignored when using these Perl application servers, since they're just read as Perl code and not handed to the exec() family of kernel syscalls. Starman/Morbo/Hypnotoad have been installed with /usr/bin/perl shebangs, so they'll use the system Perl by default, but I think if they were installed using plenv and Carton for instance that they'd have their shebangs rewritten appropriately. We could also invoke Starman/Morbo/Hypnotoad with the version of Perl that we desire. So ultimately. I suppose the most reasonable thing to do is probably to mandate "#!/usr/bin/perl" for all Koha Perl scripts? If someone does want to use a different Perl, I suppose the onus is on them to make sure they execute Koha Perl scripts with a different Perl binary? Also, do people think there is a point in repairing existing Koha Perl scripts that have '#!/usr/bin/env perl' instead of '#!/usr/bin/perl'? Cheers, David Cook Software Engineer Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Online: 02 8005 0595
On 13/10/20 12:19 pm, dcook@prosentient.com.au wrote:
Hi all,
I was looking at “Bug 20582 - Turn Koha into a Mojolicious application” when I noticed that some of the scripts were using “#!/usr/bin/env perl” instead of “#!/usr/bin/perl”. While there is no requirement in the Coding Guidelines to use “#!/usr/bin/perl”, I thought it would be best for all our Perl scripts to be consistent.
...
Also, do people think there is a point in repairing existing Koha Perl scripts that have ‘#!/usr/bin/env perl’ instead of ‘#!/usr/bin/perl’?
+1 for /usr/bin/env
+1 Whould be nice for non-Debian OS possibility, and multi-perl installs. On 13/10/2020 01:19, dcook@prosentient.com.au wrote:
Hi all,
I was looking at “Bug 20582 - Turn Koha into a Mojolicious application” when I noticed that some of the scripts were using “#!/usr/bin/env perl” instead of “#!/usr/bin/perl”. While there is no requirement in the Coding Guidelines to use “#!/usr/bin/perl”, I thought it would be best for all our Perl scripts to be consistent.
However, now I’m weighing up the options.
Conceptually, using “#!/usr/bin/env perl” means that we can use perlbrew/plenv (or manually provide a different Perl in our $PATH) when running these scripts, so that we don’t have to use the system Perl located at /usr/bin/perl.
That said, I was concerned about how most CGI scripts have “#!/usr/bin/perl” in them and the implications that might have if we invoked Starman/Morbo/Hypnotoad with a different Perl, but looking at Plack::App::CGIBin, CGI::Emulate::PSGI, Plack::App::WrapCGI, CGI::Compile, and Koha::App::Plugin::CGIBinKoha makes me think that the shebang in CGI scripts is ignored when using these Perl application servers, since they’re just read as Perl code and not handed to the exec() family of kernel syscalls.
Starman/Morbo/Hypnotoad have been installed with /usr/bin/perl shebangs, so they’ll use the system Perl by default, but I think if they were installed using plenv and Carton for instance that they’d have their shebangs rewritten appropriately. We could also invoke Starman/Morbo/Hypnotoad with the version of Perl that we desire.
So ultimately… I suppose the most reasonable thing to do is probably to mandate “#!/usr/bin/perl” for all Koha Perl scripts? If someone does want to use a different Perl, I suppose the onus is on them to make sure they execute Koha Perl scripts with a different Perl binary?
Also, do people think there is a point in repairing existing Koha Perl scripts that have ‘#!/usr/bin/env perl’ instead of ‘#!/usr/bin/perl’?
Cheers,
David Cook
Software Engineer
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Fridolin SOMERS <fridolin.somers@biblibre.com> Software and system maintainer 🦄 BibLibre, France
I vaguely remember I had this discussion with someone, somewhere. And I have been told it was not a good idea. But I cannot remember why, who, where... Le mar. 13 oct. 2020 à 01:19, <dcook@prosentient.com.au> a écrit :
Hi all,
I was looking at “Bug 20582 - Turn Koha into a Mojolicious application” when I noticed that some of the scripts were using “#!/usr/bin/env perl” instead of “#!/usr/bin/perl”. While there is no requirement in the Coding Guidelines to use “#!/usr/bin/perl”, I thought it would be best for all our Perl scripts to be consistent.
However, now I’m weighing up the options.
Conceptually, using “#!/usr/bin/env perl” means that we can use perlbrew/plenv (or manually provide a different Perl in our $PATH) when running these scripts, so that we don’t have to use the system Perl located at /usr/bin/perl.
That said, I was concerned about how most CGI scripts have “#!/usr/bin/perl” in them and the implications that might have if we invoked Starman/Morbo/Hypnotoad with a different Perl, but looking at Plack::App::CGIBin, CGI::Emulate::PSGI, Plack::App::WrapCGI, CGI::Compile, and Koha::App::Plugin::CGIBinKoha makes me think that the shebang in CGI scripts is ignored when using these Perl application servers, since they’re just read as Perl code and not handed to the exec() family of kernel syscalls.
Starman/Morbo/Hypnotoad have been installed with /usr/bin/perl shebangs, so they’ll use the system Perl by default, but I think if they were installed using plenv and Carton for instance that they’d have their shebangs rewritten appropriately. We could also invoke Starman/Morbo/Hypnotoad with the version of Perl that we desire.
So ultimately… I suppose the most reasonable thing to do is probably to mandate “#!/usr/bin/perl” for all Koha Perl scripts? If someone does want to use a different Perl, I suppose the onus is on them to make sure they execute Koha Perl scripts with a different Perl binary?
Also, do people think there is a point in repairing existing Koha Perl scripts that have ‘#!/usr/bin/env perl’ instead of ‘#!/usr/bin/perl’?
Cheers,
David Cook
Software Engineer
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
My vague recollection of such a conversation was summarized in this post: https://jmmv.dev/2016/09/env-considered-harmful.html I'm generally pro /usr/bin/env but they do make some good points in the above that are worth consideration. *Martin Renvoize* <https://www.ptfs-europe.com> Development Team Manager Community Release Manager (19.11, 20.05) *Phone:* +44 (0) 1483 378728 *Mobile:* +44 (0) 7725 985 636 *Email:* martin.renvoize@ptfs-europe.com *Fax:* +44 (0) 800 756 6384 www.ptfs-europe.com Registered in the United Kingdom No. 06416372 VAT Reg No. 925 7211 30 The information contained in this email message may be privileged, confidential and protected from disclosure. If you are not the intended recipient, any dissemination, distribution or copying is strictly prohibited. If you think that you have received this email message in error, please email the sender at info@ptfs-europe.com On Tue, 13 Oct 2020 at 10:09, Jonathan Druart < jonathan.druart@bugs.koha-community.org> wrote:
I vaguely remember I had this discussion with someone, somewhere. And I have been told it was not a good idea. But I cannot remember why, who, where...
Le mar. 13 oct. 2020 à 01:19, <dcook@prosentient.com.au> a écrit :
Hi all,
I was looking at “Bug 20582 - Turn Koha into a Mojolicious application”
when I noticed that some of the scripts were using “#!/usr/bin/env perl” instead of “#!/usr/bin/perl”. While there is no requirement in the Coding Guidelines to use “#!/usr/bin/perl”, I thought it would be best for all our Perl scripts to be consistent.
However, now I’m weighing up the options.
Conceptually, using “#!/usr/bin/env perl” means that we can use
perlbrew/plenv (or manually provide a different Perl in our $PATH) when running these scripts, so that we don’t have to use the system Perl located at /usr/bin/perl.
That said, I was concerned about how most CGI scripts have
“#!/usr/bin/perl” in them and the implications that might have if we invoked Starman/Morbo/Hypnotoad with a different Perl, but looking at Plack::App::CGIBin, CGI::Emulate::PSGI, Plack::App::WrapCGI, CGI::Compile, and Koha::App::Plugin::CGIBinKoha makes me think that the shebang in CGI scripts is ignored when using these Perl application servers, since they’re just read as Perl code and not handed to the exec() family of kernel syscalls.
Starman/Morbo/Hypnotoad have been installed with /usr/bin/perl shebangs,
so they’ll use the system Perl by default, but I think if they were installed using plenv and Carton for instance that they’d have their shebangs rewritten appropriately. We could also invoke Starman/Morbo/Hypnotoad with the version of Perl that we desire.
So ultimately… I suppose the most reasonable thing to do is probably to
mandate “#!/usr/bin/perl” for all Koha Perl scripts? If someone does want to use a different Perl, I suppose the onus is on them to make sure they execute Koha Perl scripts with a different Perl binary?
Also, do people think there is a point in repairing existing Koha Perl
scripts that have ‘#!/usr/bin/env perl’ instead of ‘#!/usr/bin/perl’?
Cheers,
David Cook
Software Engineer
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
"Renvoize, Martin" <martin.renvoize@ptfs-europe.com> wrote:
My vague recollection of such a conversation was summarized in this post: https://jmmv.dev/2016/09/env-considered-harmful.html
I'm generally pro /usr/bin/env but they do make some good points in the above that are worth consideration.
Let me consider those six points (converting roman numerals to arabic): 1-3. basically variations on using env might not find the version you want - but that applies to /usr/bin/perl too; 4. cannot pass flags to the interpreter - do we want to? 5. affected by user environment - I think we use some environment variables anyway, so we already need to control the environment, so this is not a problem for Koha. 6. "not suitable to being installed" due to 1-5 plus some arbitrary rules about "the installed script must see the exact same environment that was specified at installation time" which is not true of any system getting security updates, is it? Maybe I don't understand this point. So I don't think I see a compelling reason against env there. Have I missed something? Regards, -- MJR http://mjr.towers.org.uk/ Member of http://www.software.coop/ (but this email is my personal view only)
Good point. I have to say managing things with ENV is a good path for packaging Koha in different ways (flatpak, snap, docker, etc) On Tue, Oct 13, 2020, 18:20 MJ Ray <mjr@phonecoop.coop> wrote:
"Renvoize, Martin" <martin.renvoize@ptfs-europe.com> wrote:
My vague recollection of such a conversation was summarized in this post: https://jmmv.dev/2016/09/env-considered-harmful.html
I'm generally pro /usr/bin/env but they do make some good points in the above that are worth consideration.
Let me consider those six points (converting roman numerals to arabic):
1-3. basically variations on using env might not find the version you want - but that applies to /usr/bin/perl too;
4. cannot pass flags to the interpreter - do we want to?
5. affected by user environment - I think we use some environment variables anyway, so we already need to control the environment, so this is not a problem for Koha.
6. "not suitable to being installed" due to 1-5 plus some arbitrary rules about "the installed script must see the exact same environment that was specified at installation time" which is not true of any system getting security updates, is it? Maybe I don't understand this point.
So I don't think I see a compelling reason against env there.
Have I missed something?
Regards, --
MJR http://mjr.towers.org.uk/ Member of http://www.software.coop/ (but this email is my personal view only)
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
My point is that I find 1217 instances of '#!/usr/bin/perl' but only 38 instances of '#!/usr/bin/env perl' in the Koha master codebase. It seems to me that we should change those 38 instances of '#!/usr/bin/env perl' to adhere to '#!/usr/bin/perl' so that we're 100% consistent with our Perl shebangs. Since we're moving away from CGI and over to Plack/Mojo, we really don't need to worry about shebangs for the web app too much. Starman/Morbo/Hypnotoad should have the same shebang as the Perl that was used to compile them (thanks to this aspect of ExtUtils::MakeMaker: https://metacpan.org/pod/ExtUtils%3a%3aMakeMaker#EXE_FILES). In Debian packages, they'll use /usr/bin/perl, but if you installed them in a "virtual environment", they should point to whatever Perl interpreter you're using there. At a glance, that leaves a few potential issues: 1. koha-shell has a #!/usr/bin/perl shebang. This might need to be #!/usr/bin/env perl in order to work in a "virtual environment", or somehow rewritten to the desired Perl binary during build. (Miyagawa rewrites the cpanm shebang in Makefile.PL although for a different reason: https://github.com/miyagawa/cpanminus/blob/devel/App-cpanminus/Makefile.PL#L.... In our case, since debian/ isn't affected by Makefile.PL, we'd probably need to do this in debian/rules...). 2. koha-z3950-responder and koha-plack make references to #!/usr/bin/perl in them. They probably should just try to call "perl" and expect that the caller has set their PATH correctly. Notes: 1. t/ and xt/ scripts use #!/usr/bin/perl. I think prove will use the Perl used to invoke "prove". The shebang is just used by Test::Harness to determine if it's a Perl script. It looks like prove uses $ENV{HARNESS_PERL}, which is undocumented, or $^X (https://perldoc.perl.org/perlvar#$%5EX) according to https://metacpan.org/source/TAP::Parser::SourceHandler::Perl#L324. So I don't think this is a drama. 2. Obviously lots of cronjobs and other scripts have #!/usr/bin/perl shebangs, but that shouldn't matter so long as they're called by a Perl interpreter rather than being executed by a shell. 3. Noticing a few debian/* files using #!/usr/bin/perl, but they're developer scripts, and I imagine they're invoked via "perl" rather than by the shell. Shouldn't be an issue. For other words on #!/usr/bin/env perl, take a look at the following: https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/issues/58 (Discussion about #!/usr/bin/env in the context of EXE_FILES amongst Perl big shots like Miyagawa, Karen Etheridge, Sebastian Riedel, Leon Timmermans, Tim Bunce, etc.) https://www.python.org/dev/peps/pep-0394/ (Python's own statements about shebang usage for Python scripts.)("For scripts that are only expected to be run in an activated virtual environment, shebang lines can be written as #!/usr/bin/env python, as this instructs the script to respect the active virtual environment.")("When potentially targeting [other] environments, developers may either use a Python package installation tool that rewrites shebang lines for the installed environment, provide instructions on updating shebang lines interactively, or else use more specific shebang lines that are tailored to the target environment.") Of course, everything after the first sentence of my email is a bit moot, as at the moment we only support Koha in a non-virtual environment scenario. David Cook Software Engineer Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Online: 02 8005 0595 -----Original Message----- From: Koha-devel <koha-devel-bounces@lists.koha-community.org> On Behalf Of MJ Ray Sent: Wednesday, 14 October 2020 8:19 AM To: koha-devel@lists.koha-community.org Subject: Re: [Koha-devel] Inconsistent Perl shebangs (new coding guideline?) "Renvoize, Martin" <martin.renvoize@ptfs-europe.com> wrote:
My vague recollection of such a conversation was summarized in this post: https://jmmv.dev/2016/09/env-considered-harmful.html
I'm generally pro /usr/bin/env but they do make some good points in the above that are worth consideration.
Let me consider those six points (converting roman numerals to arabic): 1-3. basically variations on using env might not find the version you want - but that applies to /usr/bin/perl too; 4. cannot pass flags to the interpreter - do we want to? 5. affected by user environment - I think we use some environment variables anyway, so we already need to control the environment, so this is not a problem for Koha. 6. "not suitable to being installed" due to 1-5 plus some arbitrary rules about "the installed script must see the exact same environment that was specified at installation time" which is not true of any system getting security updates, is it? Maybe I don't understand this point. So I don't think I see a compelling reason against env there. Have I missed something? Regards, -- MJR http://mjr.towers.org.uk/ Member of http://www.software.coop/ (but this email is my personal view only) _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Robin helped me and found it! That's the one I had in mind: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5617#c7 Le mar. 13 oct. 2020 à 11:09, Jonathan Druart <jonathan.druart@bugs.koha-community.org> a écrit :
I vaguely remember I had this discussion with someone, somewhere. And I have been told it was not a good idea. But I cannot remember why, who, where...
Le mar. 13 oct. 2020 à 01:19, <dcook@prosentient.com.au> a écrit :
Hi all,
I was looking at “Bug 20582 - Turn Koha into a Mojolicious application” when I noticed that some of the scripts were using “#!/usr/bin/env perl” instead of “#!/usr/bin/perl”. While there is no requirement in the Coding Guidelines to use “#!/usr/bin/perl”, I thought it would be best for all our Perl scripts to be consistent.
However, now I’m weighing up the options.
Conceptually, using “#!/usr/bin/env perl” means that we can use perlbrew/plenv (or manually provide a different Perl in our $PATH) when running these scripts, so that we don’t have to use the system Perl located at /usr/bin/perl.
That said, I was concerned about how most CGI scripts have “#!/usr/bin/perl” in them and the implications that might have if we invoked Starman/Morbo/Hypnotoad with a different Perl, but looking at Plack::App::CGIBin, CGI::Emulate::PSGI, Plack::App::WrapCGI, CGI::Compile, and Koha::App::Plugin::CGIBinKoha makes me think that the shebang in CGI scripts is ignored when using these Perl application servers, since they’re just read as Perl code and not handed to the exec() family of kernel syscalls.
Starman/Morbo/Hypnotoad have been installed with /usr/bin/perl shebangs, so they’ll use the system Perl by default, but I think if they were installed using plenv and Carton for instance that they’d have their shebangs rewritten appropriately. We could also invoke Starman/Morbo/Hypnotoad with the version of Perl that we desire.
So ultimately… I suppose the most reasonable thing to do is probably to mandate “#!/usr/bin/perl” for all Koha Perl scripts? If someone does want to use a different Perl, I suppose the onus is on them to make sure they execute Koha Perl scripts with a different Perl binary?
Also, do people think there is a point in repairing existing Koha Perl scripts that have ‘#!/usr/bin/env perl’ instead of ‘#!/usr/bin/perl’?
Cheers,
David Cook
Software Engineer
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
Oh that's interesting! Yeah I reckon we just use #!/usr/bin/perl and let fix-perl-path.PL sort things out from there. David Cook Software Engineer Prosentient Systems 72/330 Wattle St Ultimo, NSW 2007 Australia Office: 02 9212 0899 Online: 02 8005 0595 -----Original Message----- From: Jonathan Druart <jonathan.druart@bugs.koha-community.org> Sent: Wednesday, 14 October 2020 7:52 PM To: David Cook <dcook@prosentient.com.au> Cc: koha-devel <koha-devel@lists.koha-community.org> Subject: Re: [Koha-devel] Inconsistent Perl shebangs (new coding guideline?) Robin helped me and found it! That's the one I had in mind: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=5617#c7 Le mar. 13 oct. 2020 à 11:09, Jonathan Druart <jonathan.druart@bugs.koha-community.org> a écrit :
I vaguely remember I had this discussion with someone, somewhere. And I have been told it was not a good idea. But I cannot remember why, who, where...
Le mar. 13 oct. 2020 à 01:19, <dcook@prosentient.com.au> a écrit :
Hi all,
I was looking at “Bug 20582 - Turn Koha into a Mojolicious application” when I noticed that some of the scripts were using “#!/usr/bin/env perl” instead of “#!/usr/bin/perl”. While there is no requirement in the Coding Guidelines to use “#!/usr/bin/perl”, I thought it would be best for all our Perl scripts to be consistent.
However, now I’m weighing up the options.
Conceptually, using “#!/usr/bin/env perl” means that we can use perlbrew/plenv (or manually provide a different Perl in our $PATH) when running these scripts, so that we don’t have to use the system Perl located at /usr/bin/perl.
That said, I was concerned about how most CGI scripts have “#!/usr/bin/perl” in them and the implications that might have if we invoked Starman/Morbo/Hypnotoad with a different Perl, but looking at Plack::App::CGIBin, CGI::Emulate::PSGI, Plack::App::WrapCGI, CGI::Compile, and Koha::App::Plugin::CGIBinKoha makes me think that the shebang in CGI scripts is ignored when using these Perl application servers, since they’re just read as Perl code and not handed to the exec() family of kernel syscalls.
Starman/Morbo/Hypnotoad have been installed with /usr/bin/perl shebangs, so they’ll use the system Perl by default, but I think if they were installed using plenv and Carton for instance that they’d have their shebangs rewritten appropriately. We could also invoke Starman/Morbo/Hypnotoad with the version of Perl that we desire.
So ultimately… I suppose the most reasonable thing to do is probably to mandate “#!/usr/bin/perl” for all Koha Perl scripts? If someone does want to use a different Perl, I suppose the onus is on them to make sure they execute Koha Perl scripts with a different Perl binary?
Also, do people think there is a point in repairing existing Koha Perl scripts that have ‘#!/usr/bin/env perl’ instead of ‘#!/usr/bin/perl’?
Cheers,
David Cook
Software Engineer
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia
Office: 02 9212 0899
Online: 02 8005 0595
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
participants (7)
-
dcook＠prosentient.com.au -
Fridolin SOMERS -
Jonathan Druart -
Mason James -
MJ Ray -
Renvoize, Martin -
Tomas Cohen Arazi