[Bug 36722] New: REST API won't work if json-schema.org is down
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 Bug ID: 36722 Summary: REST API won't work if json-schema.org is down Change sponsored?: --- Product: Koha Version: 22.05 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 - low Component: REST API Assignee: koha-bugs@lists.koha-community.org Reporter: dcook@prosentient.com.au CC: tomascohen@gmail.com This is arguably the same issue as bug 23653 except that it's a different website this time. Also, Koha won't crash if json-schema.org is down, but the API will return a 404 for all endpoints. Like bug 23653, I think that we should use a local copy of the specification. At the moment, the way to do that is to download http://json-schema.org/draft-04/schema#, save it into our source tree, and then run the following: my $schema = JSON::Validator::Schema::OpenAPIv2->new; $schema->specification($saved_schema_spec); In theory, that's it. I'll look at providing a patch shortly... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #1 from David Cook <dcook@prosentient.com.au> --- Note also that JSON::Validator::Schema::Draft4 used by JSON::Validator::Schema::OpenAPIv2 is set to http://json-schema.org/draft-04/schema# which redirects to the HTTPS version, so every worker does 2 web requests to it on startup. Yikes... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |30194 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30194 [Bug 30194] Update required JSON::Validator version -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #2 from David Cook <dcook@prosentient.com.au> --- Note an easy way to prove this is to run the following: Window 1: sudo tcpdump -A -i any 'host json-schema.org' Window 2: sudo koha-plack --restart kohadev -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- Looks like JSON::Validator isn't caching files unless JSON_VALIDATOR_CACHE_PATH is defined... It did cache when I changed koha-plack to do this: if JSON_VALIDATOR_CACHE_PATH=/opt/cache ${STARMAN} ${STARMANOPTS}; then -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #4 from David Cook <dcook@prosentient.com.au> --- And now I'm extra confused... because JSON::Validator::Schema::OpenAPIv2 has a specification of http://swagger.io/v2/schema.json but http://json-schema.org/draft-04/schema# is what's being downloaded... I suppose because the $schema for http://swagger.io/v2/schema.json is http://json-schema.org/draft-04/schema# -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #5 from David Cook <dcook@prosentient.com.au> --- If I force JSON::Validator::Store into debug mode, it does look like it's downloading it during the resolving process... [JSON::Validator] Resolve curr: ref=http://json-schema.org/draft-04/schema#/properties/description,base_url=http... fragment= id=http://swagger.io/v2/schema.json root=HASH(0x55d20f974060) schema=HASH(0x55d20f9740 60) source=load [JSON::Validator] Resolve state: base_url=http://json-schema.org/draft-04/schema fragment=/definitions/stringArray id=http://json-schema.org/draft-04/schema root=HASH(0x55d2057e4a40) schema=HASH(0x55d20f702f40) source=schema/base_url I'm starting to think that bug 30194 actually made no difference here... -- [JSON::Validator] Load from cache /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df [JSON::Validator] Load from URL http://json-schema.org/draft-04/schema And Koha can't add json-schema.org to the cache, because Koha doesn't have write access to /usr/share/perl5/JSON/Validator/cache... 36d1bd12eeed51e86c8695bd8876a9df is the cached version of the http://swagger.io/v2/schema.json file. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #6 from David Cook <dcook@prosentient.com.au> --- Note that /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df and api/swagger-v2-schema.json are the same... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #7 from David Cook <dcook@prosentient.com.au> --- I think the shortest solution for this one is going to be to add the contents of http://json-schema.org/draft-04/schema# as /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8 And that can be done via the libjson-validator-perl package that the Koha community puts together. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mtj@kohaaloha.com Component|REST API |Packaging QA Contact| |testopia@bugs.koha-communit | |y.org --- Comment #8 from David Cook <dcook@prosentient.com.au> --- Changing this to a "Packaging" bug I guess. Technically, we could point the cache paths elsewhere and provide our own cached files but happy enough to be continuing as we are... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|REST API won't work if |Koha not caching |json-schema.org is down |http://json-schema.org/draf | |t-04/schema# -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #9 from Tomás Cohen Arazi <tomascohen@gmail.com> --- At some point I did ```shell ktd --shell sudo vim /usr/share/perl5/JSON/Validator/Schema/Draft4.pm ``` and replaced `http` with `https`on line 12. And it started working. But I'm not sure it is really it, as I cannot make it fail anymore :-P Just noting in case it is useful. I think the solution is on packaging as well. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #10 from David Cook <dcook@prosentient.com.au> --- When I reported and analyzed this one, I wasn't having any errors. I just wanted to make things faster/more efficient. The error for me was hypothetical only, so that's interesting that you did experience the error! To make it reliably fail, you can add the following to your /etc/hosts: 127.0.0.1 json-schema.org -- But yeah we should be able to fix this one by packaging (or by changing the cache paths and providing our own cached files). -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=36546 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #11 from David Cook <dcook@prosentient.com.au> --- (In reply to David Cook from comment #7)
I think the shortest solution for this one is going to be to add the contents of http://json-schema.org/draft-04/schema# as /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8
And that can be done via the libjson-validator-perl package that the Koha community puts together.
/usr/share/perl5/JSON/Validator/Store.pm removes the fragment from the URL, so you end up with the following: echo -n "http://json-schema.org/draft-04/schema" | md5sum 49c95b866e40f788892a7fb3c816b0e8 - -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #12 from David Cook <dcook@prosentient.com.au> --- Ok now I'm super curious... My Koha 23.11 dev system on Ubuntu 22.04 libjson-validator-perl 5.99+really5.08-koha1 /usr/share/perl5/JSON/Validator/cache/089e74a6d17f64af17a9efd6d0fa0de6 /usr/share/perl5/JSON/Validator/cache/10a5eeb37fcd5d829449028f7ceb0774 /usr/share/perl5/JSON/Validator/cache/33912dbbde6e1d936140f1c82b283d01 /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df /usr/share/perl5/JSON/Validator/cache/3be3f46eb248daf48925640f8ef057e8 /usr/share/perl5/JSON/Validator/cache/3d35aac549d951f4cf9182ff47bff0b4 /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8 /usr/share/perl5/JSON/Validator/cache/4a31fe43be9e23ca9eb8d9e9faba8892 /usr/share/perl5/JSON/Validator/cache/546acf85ddc442761c18517490215b90 /usr/share/perl5/JSON/Validator/cache/630949337805585c8e52deea27d11419 /usr/share/perl5/JSON/Validator/cache/7fe97ed1a4c3fac607dd276b2b298275 /usr/share/perl5/JSON/Validator/cache/a0f5b4b4e75ea17fc09e88ec0343d148 /usr/share/perl5/JSON/Validator/cache/a516498b60c53096b2ce2cd83ebe0abc /usr/share/perl5/JSON/Validator/cache/c6f188eb288cf986f23db49297b25e83 /usr/share/perl5/JSON/Validator/cache/d18065ce8fb1f748e766b2737bae5200 /usr/share/perl5/JSON/Validator/cache/d8cf7ae7a0fd14accadf5d18bc84d14f /usr/share/perl5/JSON/Validator/cache/ea34d47d4e060a1c3b12d2287aff89a7 /usr/share/perl5/JSON/Validator/cache/eaa832720f36cff0abc20c05236a9cd9 My Koha main dev system on Debian 12 libjson-validator-perl 5.99+really5.14-koha1 /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df /usr/share/perl5/JSON/Validator/cache/a516498b60c53096b2ce2cd83ebe0abc -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Version|22.05 |Main Depends on|30194 | Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=30194 [Bug 30194] Update required JSON::Validator version -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #13 from David Cook <dcook@prosentient.com.au> --- So not a problem in prod for people using the "koha" apt repo. But it is a problem in the "koha-staging" apt repo. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=37361 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 --- Comment #14 from David Cook <dcook@prosentient.com.au> --- Ok and now it looks like it's a problem in "koha"! dpkg -l libjson-validator-perl Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-======================-=====================-============-============================================= ii libjson-validator-perl 5.99+really5.14-koha2 all module to validate data against a JSON schema dpkg -L libjson-validator-perl /. /usr /usr/share /usr/share/doc /usr/share/doc/libjson-validator-perl /usr/share/doc/libjson-validator-perl/NEWS.Debian.gz /usr/share/doc/libjson-validator-perl/changelog.Debian.gz /usr/share/doc/libjson-validator-perl/changelog.gz /usr/share/doc/libjson-validator-perl/copyright /usr/share/man /usr/share/man/man3 /usr/share/man/man3/JSON::Validator.3pm.gz /usr/share/man/man3/JSON::Validator::Error.3pm.gz /usr/share/man/man3/JSON::Validator::Formats.3pm.gz /usr/share/man/man3/JSON::Validator::Joi.3pm.gz /usr/share/man/man3/JSON::Validator::Schema.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft201909.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft4.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft6.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::Draft7.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::OpenAPIv2.3pm.gz /usr/share/man/man3/JSON::Validator::Schema::OpenAPIv3.3pm.gz /usr/share/man/man3/JSON::Validator::Store.3pm.gz /usr/share/man/man3/JSON::Validator::URI.3pm.gz /usr/share/man/man3/JSON::Validator::Util.3pm.gz /usr/share/perl5 /usr/share/perl5/JSON /usr/share/perl5/JSON/Validator /usr/share/perl5/JSON/Validator/Error.pm /usr/share/perl5/JSON/Validator/Formats.pm /usr/share/perl5/JSON/Validator/Joi.pm /usr/share/perl5/JSON/Validator/Schema /usr/share/perl5/JSON/Validator/Schema/Draft201909.pm /usr/share/perl5/JSON/Validator/Schema/Draft4.pm /usr/share/perl5/JSON/Validator/Schema/Draft6.pm /usr/share/perl5/JSON/Validator/Schema/Draft7.pm /usr/share/perl5/JSON/Validator/Schema/OpenAPIv2.pm /usr/share/perl5/JSON/Validator/Schema/OpenAPIv3.pm /usr/share/perl5/JSON/Validator/Schema.pm /usr/share/perl5/JSON/Validator/Store.pm /usr/share/perl5/JSON/Validator/URI.pm /usr/share/perl5/JSON/Validator/Util.pm /usr/share/perl5/JSON/Validator/cache /usr/share/perl5/JSON/Validator.pm /usr/share/perl5/JSON/Validator/cache/36d1bd12eeed51e86c8695bd8876a9df /usr/share/perl5/JSON/Validator/cache/a516498b60c53096b2ce2cd83ebe0abc -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36722 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |DUPLICATE Status|NEW |RESOLVED --- Comment #15 from David Cook <dcook@prosentient.com.au> --- It looks like my warning wasn't heeded and now it's affecting at least 2 people in prod... *** This bug has been marked as a duplicate of bug 37361 *** -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org