[Bug 23653] New: Plack stopped working - JSON validation timeout
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Bug ID: 23653 Summary: Plack stopped working - JSON validation timeout Change sponsored?: --- Product: Koha Version: 19.05 Hardware: Other OS: Linux Status: NEW Severity: major Priority: P5 - low Component: Architecture, internals, and plumbing Assignee: koha-bugs@lists.koha-community.org Reporter: rick_conrad@sil.org QA Contact: testopia@bugs.koha-community.org I had plack working fine before this morning, but now if I have plack enabled, Koha does not respond to web requests, and apache eventually shows me a 502. These are the entries from plack-error.log. I don't know what swagger.io is - seems there is such a website, but I can't `curl` it from the webserver, at least right now. plack-error.log ```` Error while loading /etc/koha/plack.psgi: Can't load application from file "/usr/share/koha/api/v1/app.pl": [JSON::Validator] GET http://swagger.io/v2/schema.json == Inactivity timeout at /usr/share/perl5/JSON/Validator.pm line 274. Compilation failed in require at (eval 389) line 1. Child process 3159 exited with status 255 ```` The only change I made this morning was changing the server's timezone, but I doubt that's related. I did put it back to UTC, but no improvement. Disabling plack works for me, but my site is a lot slower (and it wasn't real quick to begin with). -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #1 from Rick Conrad <rick_conrad@sil.org> --- I observed later that I was able to reach swagger.io from the server, so I tried reenabling plack and now it's working. So, this may properly be a plack issue and not a koha issue, but it seems like my library site ought to work whether or not I can connect to swagger.io, whatever that is. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Component|Architecture, internals, |REST api |and plumbing | CC| |katrin.fischer@bsz-bw.de, | |tomascohen@gmail.com QA Contact|testopia@bugs.koha-communit | |y.org | -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #2 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Can you give us more information on your setup? The 'swagger file' is bundled in Koha and there shouldn't be any external system reached out. It would help if you additionally: $ sudo apt install pmtools and then we need the output for: $ pmvers JSON::Validator $ pmvers Mojolicious $ pmvers Mojolicious::Plugin::OpenAPI -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|koha-bugs@lists.koha-commun |tomascohen@gmail.com |ity.org | -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #3 from Rick Conrad <rick_conrad@sil.org> --- ```` rick@koha:~$ pmvers JSON::Validator 1.08 rick@koha:~$ pmvers Mojolicious 7.59 rick@koha:~$ pmvers Mojolicious::Plugin::OpenAPI 1.15 ```` Also ```` rick@koha:~$ cd / rick@koha:/$ sudo find . -name swagger ./usr/share/koha/api/v1/swagger rick@koha:/$ cd /usr/share/koha/api/v1/swagger/ rick@koha:/usr/share/koha/api/v1/swagger$ ls definitions parameters paths swagger.json definitions.json parameters.json paths.json x-primitives.json ``` -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 chad@pennmanor.net changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |chad@pennmanor.net --- Comment #4 from chad@pennmanor.net --- I was able to reproduce this on a fresh installation of koha 20.05 from packages on debian buster. I have the webserver behind a firewall blocking all outbound traffic and could not get the koha installer to launch. After digging into the logs I saw the same connection timeouts to swagger.io As soon as I opened up our firewall the installer loaded. You can also reproduce this by dropping something bogus in your /etc/hosts file for swagger.io and restarting plack For example - 127.0.0.1 swagger.io -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtj@kohaaloha.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #5 from Mason James <mtj@kohaaloha.com> --- (In reply to Rick Conrad from comment #3)
```` rick@koha:~$ pmvers JSON::Validator 1.08 rick@koha:~$ pmvers Mojolicious 7.59 rick@koha:~$ pmvers Mojolicious::Plugin::OpenAPI 1.15 ````
hi Rick which debian version is this output from? buster? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #6 from Rick Conrad <rick_conrad@sil.org> --- I don't have it on-hand, but I believe it was Ubuntu 18.04. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #7 from Mason James <mtj@kohaaloha.com> --- (In reply to Rick Conrad from comment #6)
I don't have it on-hand, but I believe it was Ubuntu 18.04.
for 18.04, try running these commands $ sudo echo 'deb http://debian.koha-community.org/koha 20.05 main bionic' > /etc/apt/sources.list.d/koha.list $ sudo apt-get update; apt-get -y upgrade $ sudo apt install pmtools you should have... $ pmvers JSON::Validator 3.06 $ pmvers Mojolicious 8.12 $ pmvers Mojolicious::Plugin::OpenAPI 2.12 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #8 from Tomás Cohen Arazi <tomascohen@gmail.com> --- This looks really bad if Koha is trying to contact an external site on start-up -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #9 from chad@pennmanor.net --- Here is the output from my fresh buster server with koha-common 20.05.00-1 $ pmvers JSON::Validator 3.06 $ pmvers Mojolicious 8.12 $ pmvers Mojolicious::Plugin::OpenAPI 2.12
From plack-error.og
Compilation failed in require at (eval 410) line 1. Child process 25034 exited with status 255 Error while loading /etc/koha/plack.psgi: Can't load application from file "/usr/share/koha/api/v1/app.pl": [JSON::Validator] GET http://swagger.io/v2/schema.json == Not Found at /usr/share/perl5/JSON/Validator.pm line 343. Compilation failed in require at (eval 410) line 1. Child process 25042 exited with status 255 Error while loading /etc/koha/plack.psgi: Can't load application from file "/usr/share/koha/api/v1/app.pl": [JSON::Validator] GET http://swagger.io/v2/schema.json == Not Found at /usr/share/perl5/JSON/Validator.pm line 343. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #10 from Mason James <mtj@kohaaloha.com> --- hmm, this does look like a legitimate bug, caused by 22522 the 'try' block always fails with... Can't locate object method "bundle" via package "JSON::Validator::OpenAPI::Mojolicious" at /kohadevbox/koha/Koha/REST/V1.pm line 76. ...and the 'catch' block calls validator->load_and_validate_schema() which does a lookup to swagger.io ./Koha/REST/V1.pm 72 73 # Try to load and merge all schemas first and validate the result just once. 74 my $spec; 75 try { 76 $spec = $validator->bundle( 77 { 78 replace => 1, 79 schema => $self->home->rel_file("api/v1/swagger/swagger.json") 80 } 81 ); 82 100 } 101 catch { 102 # Validation of the complete spec failed. Resort to validation one-by-one 103 # to catch bad ones. 104 $validator->load_and_validate_schema( 105 $self->home->rel_file("api/v1/swagger/swagger.json"), 106 { 107 allow_invalid_ref => 1, 108 } 109 ); -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=22522 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |In Discussion --- Comment #11 from Mason James <mtj@kohaaloha.com> --- (In reply to Mason James from comment #10)
hmm, this does look like a legitimate bug, caused by 22522
^ oops, ignore that - i was testing on jessie :) the problem seems to be that the debian libjson-validator-perl package has the 'cache' dir omitted, which contains local copies of the various schemas Rick, a quick solution to the problem is to restore the cached schemas $ git clone https://github.com/mojolicious/json-validator $ cd json-validator $ git reset --hard 3.06 HEAD is now at 8b19ba0 Released version 3.06 $ sudo cp -a ./lib/JSON/Validator/cache /usr/share/perl5/JSON/Validator/ -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #12 from chad@pennmanor.net --- I followed the steps Jason provided to restore the cache folder on debian buster. Plack was able to start even when access to swagger.io was blocked. Thanks! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #13 from Mason James <mtj@kohaaloha.com> --- (In reply to Mason James from comment #11)
(In reply to Mason James from comment #10)
hmm, this does look like a legitimate bug, caused by 22522
^ oops, ignore that - i was testing on jessie :)
the problem seems to be that the debian libjson-validator-perl package has the 'cache' dir omitted, which contains local copies of the various schemas
looks like the schemas were removed due to a license issue... Source: https://metacpan.org/release/JSON-Validator Upstream-Contact: Jan Henning Thorsen <jhthorsen@cpan.org> Upstream-Name: JSON-Validator Files-Excluded: lib/JSON/Validator/cache Comment: The +dfsg version is created by removing the cache files from the lib/JSON/Validator directory, where the license is unknown. https://salsa.debian.org/perl-team/modules/packages/libjson-validator-perl/-... i'll attempt to build a new package with the schemas included -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #14 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to Mason James from comment #13)
(In reply to Mason James from comment #11)
(In reply to Mason James from comment #10)
hmm, this does look like a legitimate bug, caused by 22522
^ oops, ignore that - i was testing on jessie :)
the problem seems to be that the debian libjson-validator-perl package has the 'cache' dir omitted, which contains local copies of the various schemas
looks like the schemas were removed due to a license issue...
Source: https://metacpan.org/release/JSON-Validator Upstream-Contact: Jan Henning Thorsen <jhthorsen@cpan.org> Upstream-Name: JSON-Validator Files-Excluded: lib/JSON/Validator/cache Comment: The +dfsg version is created by removing the cache files from the lib/JSON/Validator directory, where the license is unknown.
https://salsa.debian.org/perl-team/modules/packages/libjson-validator-perl/-... raw/master/debian/copyright
i'll attempt to build a new package with the schemas included
That's weird. The schema is clearly licensed on the OpenAPI repository. It maybe I got it is about the process to acknowledge that on Debian side: https://github.com/OAI/OpenAPI-Specification/blob/master/LICENSE -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |victor@tuxayo.net --- Comment #15 from Victor Grousset/tuxayo <victor@tuxayo.net> --- This bug is marked as only affecting version 19.05.x, but it seem to affect 20.05 (as likely master) also, right? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #16 from Victor Grousset/tuxayo <victor@tuxayo.net> --- == Test plan == 1. In /etc/hosts , add «127.0.0.1 swagger.io» 2. try to load a page 3. it works 4. koha-plack --restart kohadev 5. try to load a page 6. endless or timeout loading 7. remove the sabotage from /etc/hosts 8. after a few seconds, the page will load Confirmed on master. It works on 19.05 (In reply to Mason James from comment #10)
hmm, this does look like a legitimate bug, caused by 22522
Possibly. Because 19.05.x doesn't have bug 22522 patches -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|19.05 |master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 ian.gordon@strath.ac.uk changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ian.gordon@strath.ac.uk --- Comment #17 from ian.gordon@strath.ac.uk --- Hi, At about 21:00UTC today we also started to have this problem. We're running 19.05.12 on Ubuntu 18.04. The solution in comment 11 seems to work for now. % pmvers JSON::Validator 1.08 % pmvers Mojolicious 7.59 % pmvers Mojolicious::Plugin::OpenAPI 1.15 Error message in plack-error.log is: Error while loading /etc/koha/plack.psgi: Can't load application from file "/usr/share/koha/api/v1/app.pl": [JSON::Validator] GET http://swagger.io/v2/schema.json == Connect timeout at /usr/share/perl5/JSON/Validator.pm line 274. Compilation failed in require at (eval 389) line 1. Regards, Ian Gordon. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #18 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Issue reported to upstream's upstream: https://github.com/swagger-api/swagger.io/issues/281 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Hayley Mapley <hayleymapley@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|major |blocker CC| |hayleymapley@catalyst.net.n | |z -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #19 from David Cook <dcook@prosentient.com.au> --- A solution to this problem is to provide our own argument "schema" argument to the load_and_validate_schema() method. https://metacpan.org/pod/JSON::Validator#load_and_validate_schema I've looked through the code and there's a variety of ways of providing the schema, which we can source from https://raw.githubusercontent.com/mojolicious/json-validator/3.06/lib/JSON/V... I'd say we should provide it as a local file, so that we're not relying on any web servers to provide this crucial file. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Federico Paiz <federicoantoniopaiz@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |federicoantoniopaiz@gmail.c | |om --- Comment #20 from Federico Paiz <federicoantoniopaiz@gmail.com> --- The same thing happened to me, I solved it by doing
cpan install Mojolicious::Plugin::OpenAPI
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #21 from David Cook <dcook@prosentient.com.au> --- (In reply to Federico Paiz from comment #20)
The same thing happened to me, I solved it by doing
cpan install Mojolicious::Plugin::OpenAPI
Yeah, the CPAN version of that module comes with the "cache" which the Debian package excludes, so you'll be fine with that. But it's a short-term solution. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #22 from David Cook <dcook@prosentient.com.au> --- I'm about to start on a patch for this. Hopefully will have something very soon. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #23 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to David Cook from comment #22)
I'm about to start on a patch for this. Hopefully will have something very soon.
We should just package the library with the correct cache -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #24 from David Cook <dcook@prosentient.com.au> --- Note that this should only affect plugin-enabled instances -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #25 from David Cook <dcook@prosentient.com.au> --- (In reply to Tomás Cohen Arazi from comment #23)
(In reply to David Cook from comment #22)
I'm about to start on a patch for this. Hopefully will have something very soon.
We should just package the library with the correct cache
Maybe. I think there is more to this than we think. Consider ./Koha/REST/V1.pm: $validator->bundle uses a schema of $self->home->rel_file("api/v1/swagger/swagger.json"), which is a drastically reduced schema file. Calls to $validator->load_and_validate_schema (in ./Koha/REST/V1.pm and ./Koha/REST/Plugin/PluginRoutes.pm) don't use that same schema. They try to fetch http://swagger.io/v2/schema.json. The question I have is whether they should all be using $self->home->rel_file("api/v1/swagger/swagger.json"), or if *some* calls should use our local swagger.json and others should use the full swagger 2.0 schema. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #26 from David Cook <dcook@prosentient.com.au> --- Oh wait I'm misunderstanding... $validator->bundle is just loading the schema... and it doesn't validate it. Whereas $validator->load_and_validate_schema loads the schema, but then validates it against the Swagger 2.0 API schema. Wow, JSON::Validator really overloads the word "schema" in its code... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #27 from David Cook <dcook@prosentient.com.au> ---
From what I can see, we're not actually validating the schema in ./Koha/REST/V1.pm when we use bundle().
We only validate it in ./Koha/REST/V1.pm when we do the catch{} block. And we always validate ./Koha/REST/Plugin/PluginRoutes.pm. Ok... yeah re-packaging libjson-validator-perl to include the "cache" probably makes the most sense. I'm going to post a code-level patch anyway (partially just because I'm already this far anyway) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #28 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #27)
I'm going to post a code-level patch anyway (partially just because I'm already this far anyway)
Plus, if I do that, then we wouldn't have to maintain another custom package dependency... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #29 from David Cook <dcook@prosentient.com.au> --- Actually, it looks like Koha::REST::Plugin::PluginRoutes doesn't receive the validator. It looks like the OpenAPI plugin must be calling a validator and that's where it's failing... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #30 from Victor Grousset/tuxayo <victor@tuxayo.net> --- (In reply to Tomás Cohen Arazi from comment #23)
We should just package the library with the correct cache
Indeed, that way, deployers only have to update the perl-deps, no patch to apply. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #31 from David Cook <dcook@prosentient.com.au> --- Created attachment 106444 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106444&action=edit Bug 23653: use local copy of swagger v2 schema By default, JSON::Validator::OpenAPI tries to fetch the swagger v2 schema from http://swagger.io/v2/schema.json. If you've installed from CPAN, JSON::Validator::OpenAPI will come with a cached copy, so it won't try to fetch it over HTTP. However, if you've installed from libjson-validator-perl from Debian/Ubuntu, the Debian package excludes the cached copy, so JSON::Validator::OpenAPI tries to fetch it over HTTP. Unfortunately, today and other days in the past, the file at http://swagger.io/v2/schema.json has been unavailable, and this causes Koha to crash in a perpetual loop. This patch includes a copy of the swagger v2 schema, and it loads it locally rather than fetching over HTTP. The changes to Koha/REST/Plugin/PluginRoutes.pm are not required, since the validator isn't currently called there, but I've added a patch to future proof it. To Test: 0a) Remove /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df if it exists 0b) Block external access to http://swagger.io/v2/schema.json or test during an outage when it's unavailable 0c) Do not apply patch 1) koha-plack --restart kohadev 2) Note that it crashes in a loop and is unavailable in web browser 3) Apply patch 4) koha-plack --restart kohadev 5) Note that Koha comes up and there are no errors in the Plack logs -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #106444|0 |1 is obsolete| | --- Comment #32 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Created attachment 106445 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106445&action=edit Bug 23653: use local copy of swagger v2 schema By default, JSON::Validator::OpenAPI tries to fetch the swagger v2 schema from http://swagger.io/v2/schema.json. If you've installed from CPAN, JSON::Validator::OpenAPI will come with a cached copy, so it won't try to fetch it over HTTP. However, if you've installed from libjson-validator-perl from Debian/Ubuntu, the Debian package excludes the cached copy, so JSON::Validator::OpenAPI tries to fetch it over HTTP. Unfortunately, today and other days in the past, the file at http://swagger.io/v2/schema.json has been unavailable, and this causes Koha to crash in a perpetual loop. This patch includes a copy of the swagger v2 schema, and it loads it locally rather than fetching over HTTP. The changes to Koha/REST/Plugin/PluginRoutes.pm are not required, since the validator isn't currently called there, but I've added a patch to future proof it. To Test: 0a) Remove /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df if it exists 0b) Block external access to http://swagger.io/v2/schema.json or test during an outage when it's unavailable 0c) Do not apply patch 1) koha-plack --restart kohadev 2) Note that it crashes in a loop and is unavailable in web browser 3) Apply patch 4) koha-plack --restart kohadev 5) Note that Koha comes up and there are no errors in the Plack logs Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #33 from Victor Grousset/tuxayo <victor@tuxayo.net> --- It works! :D (In reply to Victor Grousset/tuxayo from comment #30)
Indeed, that way, deployers only have to update the perl-deps, no patch to apply.
Should we actually expect a significant number of instances to have more difficulty applying such a simple patch (just this patch, not updating Koha) compared to updating the perl deps? The question is whether or not packaging libjson-validator-perl during some time is worth it. Not sure now. Also, if http://swagger.io/v2/schema.json would work again tomorrow that would help letting those instances count on that until their next Koha update. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #34 from David Cook <dcook@prosentient.com.au> --- (In reply to Victor Grousset/tuxayo from comment #33)
Should we actually expect a significant number of instances to have more difficulty applying such a simple patch (just this patch, not updating Koha) compared to updating the perl deps?
The question is whether or not packaging libjson-validator-perl during some time is worth it. Not sure now.
Also, if http://swagger.io/v2/schema.json would work again tomorrow that would help letting those instances count on that until their next Koha update.
I'm curious about this as well. While we felt this in production in Australia, I'm guessing the rest of the world might not feel it for another few hours. I don't know where Jonathan is physically located these days, but I'm guessing that we may have just missed him as well, so the Koha world is in for an interesting day... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #35 from David Cook <dcook@prosentient.com.au> --- Mason announced 30 minutes ago on koha-devel that he's updated the libjson-validator-perl package. As per his email: hi Folks i've updated the 'libjson-validator-perl' package to include the cache dir to update the package... $ sudo apt update $ sudo apt install libjson-validator-perl run the following command, and confirm the result... $ apt-cache policy libjson-validator-perl libjson-validator-perl: Installed: 3.06+dfsg-1+koha1 Candidate: 3.06+dfsg-1+koha1 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #36 from David Cook <dcook@prosentient.com.au> --- I've tested the new package in koha-testing-docker and it looks good. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #37 from David Cook <dcook@prosentient.com.au> --- I'd recommend to people starting their day to look at https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653#c11 for a quick fix to get through the worst of things right now. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|tomascohen@gmail.com |dcook@prosentient.com.au -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #38 from David Cook <dcook@prosentient.com.au> --- I think Mason found that newer versions of Ubuntu weren't installing 3.06 as they were using newer versions, but it looks like he's now released a 3.23+dfsg-1+koha1 version. I've installed that on Debian Jessie in koha-testing-docker, and it looks like it's working successfully. So folk using the Debian packages shoudl be able to just do the following: apt-get update apt-get install libjson-validator-perl -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #39 from David Cook <dcook@prosentient.com.au> --- Ok that 3.23 is still in the staging repo. If you're running a Debian/Ubuntu package, you'll be vulnerable, but not if you're running a Koha Community package. So you should be fine on Debian Jessie and Debian Stretch and Ubuntu Xenial. If you're on anything newer than those, you'll need to upgrade to the Koha Community 3.06 (or 3.23 when it is available) as per the table below: __Koha Community Apt Repo__ libjson-validator-perl_0.67+dfsg-1~koha1_all.deb libjson-validator-perl_0.97-1~kohadev1_all.deb libjson-validator-perl_3.06+dfsg-1+koha1_all.deb __Debian__ Stretch 9: libjson-validator-perl (0.92+dfsg-1) Buster 10: libjson-validator-perl (3.06+dfsg-1) Bullseye 11: libjson-validator-perl (3.25+dfsg-1) __Ubuntu__ Xenial 16.04: libjson-validator-perl (0.66+dfsg-1) Bionic 18.04: libjson-validator-perl (1.08+dfsg-1) Eoan 19.10: libjson-validator-perl (3.14+dfsg-1) Focal 20.04: libjson-validator-perl (3.23+dfsg-1) Groovy 20.10: libjson-validator-perl (3.25+dfsg-1) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #40 from David Cook <dcook@prosentient.com.au> --- Here's a condensed version of the above: Good (don't worry): - Debian Jessie 8 - Debian Stretch 9 - Ubuntu Xenial 16.04 Bad (do worry): - Debian Buster 10 (and above) - Ubuntu Bionic 18.04 (and above) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Fridolin SOMERS <fridolin.somers@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m --- Comment #41 from Fridolin SOMERS <fridolin.somers@biblibre.com> --- For Ubuntu Debian, we at biblibre installed from CPAN JSON::Validator 4.01 and added the cache file. It works without patch on sources. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #42 from Victor Grousset/tuxayo <victor@tuxayo.net> --- The schema hosting at swagger.io is back https://github.com/swagger-api/swagger.io/issues/282 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #43 from David Cook <dcook@prosentient.com.au> --- (In reply to Victor Grousset/tuxayo from comment #42)
The schema hosting at swagger.io is back https://github.com/swagger-api/swagger.io/issues/282
This is great! Although I hope this doesn't cause us to forget about the issue... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|Plack stopped working - |Plack fails when |JSON validation timeout |http://swagger.io/v2/schema | |.json is unavailable and | |schema cache missing -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #44 from Mason James <mtj@kohaaloha.com> --- (In reply to David Cook from comment #35)
Mason announced 30 minutes ago on koha-devel that he's updated the libjson-validator-perl package.
As per his email:
hi Folks i've updated the 'libjson-validator-perl' package to include the cache dir
to update the package... $ sudo apt update $ sudo apt install libjson-validator-perl
run the following command, and confirm the result... $ apt-cache policy libjson-validator-perl libjson-validator-perl: Installed: 3.06+dfsg-1+koha1 Candidate: 3.06+dfsg-1+koha1
I've updated the libjson-validator-perl and libmojolicious-plugin-openapi-perl packages again, to make them work on debian/ubuntu 'unstable' $ apt-cache policy libjson-validator-perl libmojolicious-plugin-openapi-perl libjson-validator-perl: Installed: 3.25+dfsg-1+koha3 Candidate: 3.25+dfsg-1+koha3 Version table: *** 3.25+dfsg-1+koha3 500 500 http://debian.koha-community.org/koha stable/main amd64 Packages 100 /var/lib/dpkg/status .. libmojolicious-plugin-openapi-perl: Installed: 2.16-1+koha1 Candidate: 2.16-1+koha1 Version table: *** 2.16-1+koha1 500 500 http://debian.koha-community.org/koha stable/main amd64 Packages 100 /var/lib/dpkg/status -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Andrew Nugged <nugged@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nugged@gmail.com --- Comment #45 from Andrew Nugged <nugged@gmail.com> --- (In reply to Fridolin SOMERS from comment #41)
For Ubuntu Debian, we at biblibre installed from CPAN JSON::Validator 4.01 and added the cache file. It works without patch on sources.
Confirming the same: just using cpanm with only one this updated all quickfixed, temporarily for libs, but now I am learning how to make this solution stable, reading all above & tracking :). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Marcel de Rooy <m.de.rooy@rijksmuseum.nl> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |m.de.rooy@rijksmuseum.nl --- Comment #46 from Marcel de Rooy <m.de.rooy@rijksmuseum.nl> --- How often this schema changes ? Need to check for updates ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #47 from Victor Grousset/tuxayo <victor@tuxayo.net> ---
From what I understand, it's a schema from a version of the OpenAPI Specification. Something like the draft 4 of the version 2.
Here we can compare a version from 2014 and from now. https://web.archive.org/web/20141019005411/https://swagger.io/v2/schema.json vs https://web.archive.org/web/20200702172005/https://schema.swagger.io/v2/sche... Here is the actual source it seems: https://github.com/OAI/OpenAPI-Specification/blob/c196cac8191069fa97028c2f1a... vs https://github.com/OAI/OpenAPI-Specification/blob/master/schemas/v2.0/schema... I would *guess* it will stay backward compatible. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #48 from Andrew Nugged <nugged@gmail.com> --- Note by the way, these changes with dependencies brought issues with fresh installations: on Ubuntu 18.04 with those regular steps which worked everywhere usually before, (preps + "apt-get install koha-common" and voila) now fails with: install 'koha-common' failed: Unable to correct problems, you have held broken packages. Depends: libmojolicious-plugin-openapi-perl but it is not going to be installed Depends: libnet-oauth2-authorizationserver-perl but it is not going to be installed This was when: "deb http://debian.koha-community.org/koha 20.05 main" (https://wiki.koha-community.org/wiki/Koha_on_Debian#Follow_a_codename_.28ver...) but adding mentioned above explicit release name: "deb http://debian.koha-community.org/koha 20.05 main bionic" made install seamless. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #49 from Julian Maurice <julian.maurice@biblibre.com> --- Comment on attachment 106445 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=106445 Bug 23653: use local copy of swagger v2 schema Review of attachment 106445: --> (https://bugs.koha-community.org/bugzilla3/page.cgi?id=splinter.html&bug=23653&attachment=106445) ----------------------------------------------------------------- ::: Koha/REST/Plugin/PluginRoutes.pm @@ +49,5 @@
{ + my $schema = $app->home->rel_file("api/swagger-v2-schema.json"); + if ($schema){ + $self->{'swagger-v2-schema'} = $schema; + }
What is the test for ? rel_file returns something even if the file does not exist -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julian.maurice@biblibre.com --- Comment #50 from Julian Maurice <julian.maurice@biblibre.com> --- The cached copy of the schema is slightly different from the one at https://schema.swagger.io/v2/schema.json Where does it come from ? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #51 from Victor Grousset/tuxayo <victor@tuxayo.net> --- (In reply to Julian Maurice from comment #50)
The cached copy of the schema is slightly different from the one at https://schema.swagger.io/v2/schema.json Where does it come from ?
From what seems to be the canonical source: https://github.com/OAI/OpenAPI-Specification/blob/master/schemas/v2.0/schema...
-- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@bugs.koha-c | |ommunity.org --- Comment #52 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Do we need a unit test to compare the local and remote files? (ofc it won't fail if the remove file is not reachable) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #53 from Julian Maurice <julian.maurice@biblibre.com> --- (In reply to Jonathan Druart from comment #52)
Do we need a unit test to compare the local and remote files? (ofc it won't fail if the remove file is not reachable)
I think it's not worth it. Swagger/OpenAPI 2.0 doesn't change anymore (last change was in 2017) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #54 from Victor Grousset/tuxayo <victor@tuxayo.net> --- (In reply to Jonathan Druart from comment #52)
Do we need a unit test to compare the local and remote files? (ofc it won't fail if the remove file is not reachable)
my 2 cents: don't think it's worth it also. Because here we can see migration paths from the different draft versions: https://json-schema.org/specification.html#migrating-from-older-drafts So backward compatibility seem to be assured. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #55 from David Cook <dcook@prosentient.com.au> --- (In reply to Julian Maurice from comment #49)
What is the test for ? rel_file returns something even if the file does not exist
That's interesting. I didn't think it did for me when I tested. I'd have to look gain. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #56 from David Cook <dcook@prosentient.com.au> --- (In reply to Jonathan Druart from comment #52)
Do we need a unit test to compare the local and remote files? (ofc it won't fail if the remove file is not reachable)
I wouldn't think so. If our other API unit tests are passing, I'd say that's what would matter most. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #57 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Ticket opened upstream (Debian) to have the cache (not really cache) files included in the package. As upstream's(Debian) upstream's(json-validator) upstream (OpenAPI/Swagger) confirmed the license of these files. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=964496 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #58 from David Cook <dcook@prosentient.com.au> --- (In reply to Victor Grousset/tuxayo from comment #57)
Ticket opened upstream (Debian) to have the cache (not really cache) files included in the package. As upstream's(Debian) upstream's(json-validator) upstream (OpenAPI/Swagger) confirmed the license of these files.
Nice one, Victor! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Julian Maurice <julian.maurice@biblibre.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Failed QA --- Comment #59 from Julian Maurice <julian.maurice@biblibre.com> --- Failing QA to get an answer to comment 49 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|blocker |major --- Comment #60 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- IIUC this is no longer blocker, lowering severity. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #61 from David Cook <dcook@prosentient.com.au> --- (In reply to Julian Maurice from comment #59)
Failing QA to get an answer to comment 49
Yeah that's fair. I'll put it on my TODO list. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #62 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Upstream answer https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=964496#10
Indeed, the OpenAPI schema files are Apache2-licensed [3], thus at least them could be retained. However, copies of these schema files are already around in Debian [4], thus I would suggest packaging OpenAPI schema files in a separate Debian package, and symlinking them to JSON::Validator cache location.
Help welcome to try to find arguments to stop removing the schema files. The duplication concern is fair but I don't think someone here has the resources to push towards a common schema Debian package. Or maybe I'm wrong and it's not much work? hmmmm ^^" -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Failed QA |Signed Off -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #63 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 107984 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=107984&action=edit Bug 23653: Remove uneeded cond test rel_file returns the path anyway -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #64 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to Julian Maurice from comment #49)
Comment on attachment 106445 [details] [review] Bug 23653: use local copy of swagger v2 schema
Review of attachment 106445 [details] [review]: -----------------------------------------------------------------
::: Koha/REST/Plugin/PluginRoutes.pm @@ +49,5 @@
{ + my $schema = $app->home->rel_file("api/swagger-v2-schema.json"); + if ($schema){ + $self->{'swagger-v2-schema'} = $schema; + }
What is the test for ? rel_file returns something even if the file does not exist
I removed the test. Ready for QA. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #65 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- (In reply to Victor Grousset/tuxayo from comment #62)
Upstream answer https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=964496#10
Indeed, the OpenAPI schema files are Apache2-licensed [3], thus at least them could be retained. However, copies of these schema files are already around in Debian [4], thus I would suggest packaging OpenAPI schema files in a separate Debian package, and symlinking them to JSON::Validator cache location.
Help welcome to try to find arguments to stop removing the schema files. The duplication concern is fair but I don't think someone here has the resources to push towards a common schema Debian package. Or maybe I'm wrong and it's not much work? hmmmm ^^"
We don't want to rely on a remote file/server to start up correctly anyway. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |RM_priority --- Comment #66 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- What are we waiting for exactly? Can I get feedback ASAP please? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #107984|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #106445|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #67 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 109095 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=109095&action=edit Bug 23653: use local copy of swagger v2 schema By default, JSON::Validator::OpenAPI tries to fetch the swagger v2 schema from http://swagger.io/v2/schema.json. If you've installed from CPAN, JSON::Validator::OpenAPI will come with a cached copy, so it won't try to fetch it over HTTP. However, if you've installed from libjson-validator-perl from Debian/Ubuntu, the Debian package excludes the cached copy, so JSON::Validator::OpenAPI tries to fetch it over HTTP. Unfortunately, today and other days in the past, the file at http://swagger.io/v2/schema.json has been unavailable, and this causes Koha to crash in a perpetual loop. This patch includes a copy of the swagger v2 schema, and it loads it locally rather than fetching over HTTP. The changes to Koha/REST/Plugin/PluginRoutes.pm are not required, since the validator isn't currently called there, but I've added a patch to future proof it. To Test: 0a) Remove /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df if it exists 0b) Block external access to http://swagger.io/v2/schema.json or test during an outage when it's unavailable 0c) Do not apply patch 1) koha-plack --restart kohadev 2) Note that it crashes in a loop and is unavailable in web browser 3) Apply patch 4) koha-plack --restart kohadev 5) Note that Koha comes up and there are no errors in the Plack logs Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #68 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 109096 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=109096&action=edit Bug 23653: Remove uneeded cond test rel_file returns the path anyway Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |tomascohen@gmail.com --- Comment #69 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Do we need to acknowledge the file is from another project, and its license? -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #70 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 109113 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=109113&action=edit Bug 23653: Add license information in about.pl Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #109113|0 |1 is obsolete| | --- Comment #71 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Created attachment 109180 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=109180&action=edit Bug 23653: Add license information in about.pl Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> JD Amended patch <p>The included <a href="https://github.com/OAI/OpenAPI-Specification/blob/master/schemas/v2.0/schema.json">api/swagger-v2-schema.json</a> file is licensed under the[-the-] <a href="https://github.com/OAI/OpenAPI-Specification/tree/master/schemas/v2.0">Apache License, Version 2.0</a>, by the <a href="https://www.openapis.org/about">OpenAPI Initiative [-(OAI)</a></p>-]{+(OAI)</a>.</p>+} -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to master Version(s)| |20.11.00 released in| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #72 from Jonathan Druart <jonathan.druart@bugs.koha-community.org> --- Pushed to master for 20.11, thanks to everybody involved! -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords|RM_priority | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #73 from Victor Grousset/tuxayo <victor@tuxayo.net> --- (In reply to Jonathan Druart from comment #65)
(In reply to Victor Grousset/tuxayo from comment #62)
Upstream answer https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=964496#10
[...]
Help welcome to try to find arguments to stop removing the schema files. The duplication concern is fair but I don't think someone here has the resources to push towards a common schema Debian package. Or maybe I'm wrong and it's not much work? hmmmm ^^"
We don't want to rely on a remote file/server to start up correctly anyway.
The request was to include again the schema file into the Debian package to precisely not depend on a remote file. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #74 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Should this patch be backported? I just had a test failure on Ubuntu 18.04 due to not being able to reach the schema file at swagger.io https://jenkins.koha-community.org/view/19.05/job/Koha_19.05_U18/320/console... koha_1 | [JSON::Validator] GET http://swagger.io/v2/schema.json == Inactivity timeout at /usr/share/perl5/JSON/Validator.pm line 274. koha_1 | # Looks like your test exited with 255 before it could output anything. koha_1 | [10:16:53] t/db_dependent/api/v1/illrequests.t ..................................... koha_1 | Dubious, test returned 255 (wstat 65280, 0xff00) koha_1 | Failed 1/1 subtests -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #75 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Two other same in this run: https://jenkins.koha-community.org/view/19.05/job/Koha_19.05_U18/322/console... -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Andrew Nugged <nugged@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC|nugged@gmail.com | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #76 from Mason James <mtj@kohaaloha.com> --- (In reply to Victor Grousset/tuxayo from comment #74)
Should this patch be backported? I just had a test failure on Ubuntu 18.04 due to not being able to reach the schema file at swagger.io
hi Victor, we really need this backported, back to 19.11.x https://swagger.io/v2/schema.json was down recently and still causing breakage i'll set the ticket back to open -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|FIXED |--- Status|RESOLVED |REOPENED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|REOPENED |ASSIGNED -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Victor Grousset/tuxayo <victor@tuxayo.net> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Passed QA |Pushed to oldoldstable Version(s)|20.11.00 |20.11.00,20.05.13 released in| | --- Comment #77 from Victor Grousset/tuxayo <victor@tuxayo.net> --- Backported: Pushed to 20.05.x branch for 20.05.13 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wainuiwitikapark@catalyst.n | |et.nz --- Comment #78 from wainuiwitikapark@catalyst.net.nz --- This does not apply cleanly to 19.11.x please submit one formatted for 19.11.x The merge conflict in Koha/REST/V1.pm is quite large and complex so we weren't confident trying to fix it. (I did fix the merge conflict in Koha/REST/Plugin/PluginRoutes.pm okay though) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #79 from wainuiwitikapark@catalyst.net.nz --- The above comment refers to the first patch: Bug 23653: use local copy of swagger v2 schema -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #80 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 122333 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122333&action=edit Bug 23653: [19.11] use local copy of swagger v2 schema By default, JSON::Validator::OpenAPI tries to fetch the swagger v2 schema from http://swagger.io/v2/schema.json. If you've installed from CPAN, JSON::Validator::OpenAPI will come with a cached copy, so it won't try to fetch it over HTTP. However, if you've installed from libjson-validator-perl from Debian/Ubuntu, the Debian package excludes the cached copy, so JSON::Validator::OpenAPI tries to fetch it over HTTP. Unfortunately, today and other days in the past, the file at http://swagger.io/v2/schema.json has been unavailable, and this causes Koha to crash in a perpetual loop. This patch includes a copy of the swagger v2 schema, and it loads it locally rather than fetching over HTTP. The changes to Koha/REST/Plugin/PluginRoutes.pm are not required, since the validator isn't currently called there, but I've added a patch to future proof it. To Test: 0a) Remove /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df if it exists 0b) Block external access to http://swagger.io/v2/schema.json or test during an outage when it's unavailable 0c) Do not apply patch 1) koha-plack --restart kohadev 2) Note that it crashes in a loop and is unavailable in web browser 3) Apply patch 4) koha-plack --restart kohadev 5) Note that Koha comes up and there are no errors in the Plack logs Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #81 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 122334 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122334&action=edit Bug 23653: [19.11] Remove uneeded cond test rel_file returns the path anyway Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #82 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Created attachment 122335 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122335&action=edit Bug 23653: [19.11] Add license information in about.pl Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> JD Amended patch <p>The included <a href="https://github.com/OAI/OpenAPI-Specification/blob/master/schemas/v2.0/schema.json">api/swagger-v2-schema.json</a> file is licensed under the[-the-] <a href="https://github.com/OAI/OpenAPI-Specification/tree/master/schemas/v2.0">Apache License, Version 2.0</a>, by the <a href="https://www.openapis.org/about">OpenAPI Initiative [-(OAI)</a></p>-]{+(OAI)</a>.</p>+} -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #83 from Jonathan Druart <jonathan.druart+koha@gmail.com> --- Wainui, I rebased the patches for 19.11.x but I'd like someone with a plugin injecting REST API route to test the patches before you push them. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #84 from wainuiwitikapark@catalyst.net.nz --- (In reply to Jonathan Druart from comment #83)
Wainui, I rebased the patches for 19.11.x but I'd like someone with a plugin injecting REST API route to test the patches before you push them.
Thanks Jonathan. They applied cleanly to my branch, but I won't test and push them up until the next release (19.11.20). -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #85 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to wainuiwitikapark from comment #84)
(In reply to Jonathan Druart from comment #83)
Wainui, I rebased the patches for 19.11.x but I'd like someone with a plugin injecting REST API route to test the patches before you push them.
Thanks Jonathan. They applied cleanly to my branch, but I won't test and push them up until the next release (19.11.20).
I can test in the morning. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #86 from wainuiwitikapark@catalyst.net.nz --- (In reply to Tomás Cohen Arazi from comment #85)
(In reply to wainuiwitikapark from comment #84)
(In reply to Jonathan Druart from comment #83)
Wainui, I rebased the patches for 19.11.x but I'd like someone with a plugin injecting REST API route to test the patches before you push them.
Thanks Jonathan. They applied cleanly to my branch, but I won't test and push them up until the next release (19.11.20).
I can test in the morning.
Hi Tomas, I'm just wondering if you have had a chance to test this yet? Thanks, Wainui :) -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #87 from Tomás Cohen Arazi <tomascohen@gmail.com> --- (In reply to wainuiwitikapark from comment #86)
(In reply to Tomás Cohen Arazi from comment #85)
(In reply to wainuiwitikapark from comment #84)
(In reply to Jonathan Druart from comment #83)
Wainui, I rebased the patches for 19.11.x but I'd like someone with a plugin injecting REST API route to test the patches before you push them.
Thanks Jonathan. They applied cleanly to my branch, but I won't test and push them up until the next release (19.11.20).
I can test in the morning.
Hi Tomas,
I'm just wondering if you have had a chance to test this yet?
Thanks,
Wainui :)
Doing it right now -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #122333|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #122334|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #122335|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #88 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 122650 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122650&action=edit Bug 23653: [19.11] use local copy of swagger v2 schema By default, JSON::Validator::OpenAPI tries to fetch the swagger v2 schema from http://swagger.io/v2/schema.json. If you've installed from CPAN, JSON::Validator::OpenAPI will come with a cached copy, so it won't try to fetch it over HTTP. However, if you've installed from libjson-validator-perl from Debian/Ubuntu, the Debian package excludes the cached copy, so JSON::Validator::OpenAPI tries to fetch it over HTTP. Unfortunately, today and other days in the past, the file at http://swagger.io/v2/schema.json has been unavailable, and this causes Koha to crash in a perpetual loop. This patch includes a copy of the swagger v2 schema, and it loads it locally rather than fetching over HTTP. The changes to Koha/REST/Plugin/PluginRoutes.pm are not required, since the validator isn't currently called there, but I've added a patch to future proof it. To Test: 0a) Remove /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df if it exists 0b) Block external access to http://swagger.io/v2/schema.json or test during an outage when it's unavailable 0c) Do not apply patch 1) koha-plack --restart kohadev 2) Note that it crashes in a loop and is unavailable in web browser 3) Apply patch 4) koha-plack --restart kohadev 5) Note that Koha comes up and there are no errors in the Plack logs Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #89 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 122651 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122651&action=edit Bug 23653: [19.11] Remove uneeded cond test rel_file returns the path anyway Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #90 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 122652 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122652&action=edit Bug 23653: [19.11] Add license information in about.pl Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> JD Amended patch <p>The included <a href="https://github.com/OAI/OpenAPI-Specification/blob/master/schemas/v2.0/schema.json">api/swagger-v2-schema.json</a> file is licensed under the[-the-] <a href="https://github.com/OAI/OpenAPI-Specification/tree/master/schemas/v2.0">Apache License, Version 2.0</a>, by the <a href="https://www.openapis.org/about">OpenAPI Initiative [-(OAI)</a></p>-]{+(OAI)</a>.</p>+} Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 --- Comment #91 from Tomás Cohen Arazi <tomascohen@gmail.com> --- The patches work as expected. Tests pass, and I tested the GOBI plugin which adds API routes. The routes are listed in /api/v1/.html as expected, and the plugin is functional. Thanks for the rebase, Jonathan. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #122650|0 |1 is obsolete| | --- Comment #92 from wainuiwitikapark@catalyst.net.nz --- Created attachment 122662 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122662&action=edit Bug 23653: [19.11] use local copy of swagger v2 schema By default, JSON::Validator::OpenAPI tries to fetch the swagger v2 schema from http://swagger.io/v2/schema.json. If you've installed from CPAN, JSON::Validator::OpenAPI will come with a cached copy, so it won't try to fetch it over HTTP. However, if you've installed from libjson-validator-perl from Debian/Ubuntu, the Debian package excludes the cached copy, so JSON::Validator::OpenAPI tries to fetch it over HTTP. Unfortunately, today and other days in the past, the file at http://swagger.io/v2/schema.json has been unavailable, and this causes Koha to crash in a perpetual loop. This patch includes a copy of the swagger v2 schema, and it loads it locally rather than fetching over HTTP. The changes to Koha/REST/Plugin/PluginRoutes.pm are not required, since the validator isn't currently called there, but I've added a patch to future proof it. To Test: 0a) Remove /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df if it exists 0b) Block external access to http://swagger.io/v2/schema.json or test during an outage when it's unavailable 0c) Do not apply patch 1) koha-plack --restart kohadev 2) Note that it crashes in a loop and is unavailable in web browser 3) Apply patch 4) koha-plack --restart kohadev 5) Note that Koha comes up and there are no errors in the Plack logs Signed-off-by: Victor Grousset/tuxayo <victor@tuxayo.net> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #122651|0 |1 is obsolete| | --- Comment #93 from wainuiwitikapark@catalyst.net.nz --- Created attachment 122663 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122663&action=edit Bug 23653: [19.11] Remove uneeded cond test rel_file returns the path anyway Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #122652|0 |1 is obsolete| | --- Comment #94 from wainuiwitikapark@catalyst.net.nz --- Created attachment 122664 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=122664&action=edit Bug 23653: [19.11] Add license information in about.pl Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> JD Amended patch <p>The included <a href="https://github.com/OAI/OpenAPI-Specification/blob/master/schemas/v2.0/schema.json">api/swagger-v2-schema.json</a> file is licensed under the[-the-] <a href="https://github.com/OAI/OpenAPI-Specification/tree/master/schemas/v2.0">Apache License, Version 2.0</a>, by the <a href="https://www.openapis.org/about">OpenAPI Initiative [-(OAI)</a></p>-]{+(OAI)</a>.</p>+} Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> Signed-off-by: Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 wainuiwitikapark@catalyst.net.nz changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)|20.11.00,20.05.13 |20.11.00,20.05.13,19.11.20 released in| | Status|Pushed to oldoldstable |Pushed to oldoldoldstable --- Comment #95 from wainuiwitikapark@catalyst.net.nz --- Backported to 19.11.x for 19.11.20 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=23653 Wainui Witika-Park <wainuiwitikapark@catalyst.net.nz> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Pushed to oldoldoldstable |RESOLVED Resolution|--- |FIXED -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon@bugs.koha-community.org