[Bug 37361] New: REST API: api/v1/ does not work with proxy an plack
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 Bug ID: 37361 Summary: REST API: api/v1/ does not work with proxy an plack Change sponsored?: --- Product: Koha Version: 24.05 Hardware: All OS: Linux Status: NEW Severity: major Priority: P5 - low Component: REST API Assignee: koha-bugs@lists.koha-community.org Reporter: tomiak@helmholtz-berlin.de QA Contact: testopia@bugs.koha-community.org CC: tomascohen@gmail.com https://servername/api/v1/.html and any call to the API, even when done by Koha internally always shows error 404 or the error page. starman/plack tries to contact external web servers for the schema files, as seen in plack-api-error.log: [2024/07/15 14:47:45] [ERROR] Warning: Could not load REST API spec bundle: GET http://json-schema.org/draft-04/schema: Connection refused at /usr/share/perl5/JSON/Validator/Store.pm line 190. The external http-access will require a proxy server and these are defined in /etc/profile.d/proxy.sh but not used by starman, which can be proved with Window 1: sudo tcpdump -A -i any 'host json-schema.org' Window 2: sudo koha-plack --restart koha-instance the error messages will result. The file environ of the starman processes in the linux process file system are empty. Solution (dirty): Source /etc/profile.d/proxy.sh with the definition of the proxies in /usr/sbin/koha-plack first. The proxy environment variables will be used then and all will work as wanted. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 Andreas Tomiak <tomiak@helmholtz-berlin.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |tomiak@helmholtz-berlin.de Patch complexity|--- |String patch -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 Andreas Tomiak <tomiak@helmholtz-berlin.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|REST API: api/v1/ does not |REST API: api/v1/ does not |work with proxy an plack |work with active proxy and | |plack -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 Andreas Tomiak <tomiak@helmholtz-berlin.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|major |critical Priority|P5 - low |P3 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 Michalis Zisis <mixasgr@greeklug.gr> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mixasgr@greeklug.gr --- Comment #1 from Michalis Zisis <mixasgr@greeklug.gr> --- Greetings, i am facing the same issue in a similar case. Likewise, the installation is connected to a restricted network, which is routed through a proxy. In my case, the proxy settings are defined in "/etc/environment". Moreover the library (due to being an old instance) has no active plack service. After upgrading from 23.11 to 24.05 the same issue occurs, with the api address (https://servername/api/v1/) not responding to requests and returning a 404 error. In "/var/log/koha/library/api-error.log" the same message appears "[ERROR] Warning: Could not load REST API spec bundle: GET http://json-schema.org/draft-04/schema: Connection refused at /usr/share /perl5/JSON/Validator/Store.pm line 190." To verify, I tried connecting the server through an other network that does have any network restrictions, and koha functioned without an error. So it seems that the api internally relies on retrieving the address "http://json-schema.org/draft-04/schema" and when this is not available, eg in the case of using a restricted network/proxy, there is an internal error that leads to generating a "404 not found" in the api. In turn this causes errors in various sections, such as the patrons list failing to load, due to api returning the 404 response. In regards for a possible solution, i would propose the api to not fail to run when it can't communicate in order to retrieve the needed (json) schema. Possibly, there could be a check to the api call in case of network error that would continue the api respose or some kind of fallback to a local storage of the schema/standards, so that even in case the network is not accessible, the api service would work. Unfortunately, i do not have knowledge of the code the api service users in order to suggest a specific fix. Thank you -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #2 from David Cook <dcook@prosentient.com.au> --- Thanks for the report and the additional comment. How have you installed Koha? And what OS are you using? If you've used Debian packages, there should be a cached copy of the JSON schema on the server (probably at /usr/share/perl5/JSON/Validator/cache/49c95b866e40f788892a7fb3c816b0e8), which should prevent Starman from trying to fetch http://json-schema.org/draft-04/schema Recently, I did notice a particular version of libjson-validator-perl which was missing the cached file, so it might be related to that -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.koha-community | |.org/bugzilla3/show_bug.cgi | |?id=36722 -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 --- Comment #3 from David Cook <dcook@prosentient.com.au> --- As I note on bug 36722, it looks like libjson-validator-perl version 5.99+really5.14-koha2 does not provide the cached file that it should. I flagged this back in April, but it looks like I wasn't noisy enough about the problem... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 --- Comment #4 from David Cook <dcook@prosentient.com.au> --- *** Bug 36722 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|REST API: api/v1/ does not |REST API won't start |work with active proxy and |properly without Internet |plack |access due to missing cache | |files -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Component|REST API |Packaging CC| |mtj@kohaaloha.com --- Comment #5 from David Cook <dcook@prosentient.com.au> --- Noting that this is a packaging bug because it's due to the Koha community packaging of libjson-validator-perl -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 --- Comment #6 from Andreas Tomiak <tomiak@helmholtz-berlin.de> --- Hello, I´ve installed it on Debian 11 with the proposed Koah sources. It was upgraded several times. The problem appears in April, so this may be a hint to a solution. The cache displays only these files: ls -la /usr/share/perl5/JSON/Validator/cache/ total 8 drwxr-xr-x 2 root root 4096 Jun 12 10:04 . drwxr-xr-x 4 root root 4096 Jun 12 10:04 .. lrwxrwxrwx 1 root root 58 Apr 3 14:16 36d1bd12eeed51e86c8695bd8876a9df -> ../../../../openapi-specification/schemas/v2.0/schema.json lrwxrwxrwx 1 root root 58 Apr 3 14:16 a516498b60c53096b2ce2cd83ebe0abc -> ../../../../openapi-specification/schemas/v3.0/schema.json -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 --- Comment #7 from Andreas Tomiak <tomiak@helmholtz-berlin.de> --- Some more information: dpkg-query --list | grep libjson-validator-perl ii libjson-validator-perl 5.99+really5.14-koha2 all module to validate data against a JSON schema The links are correct: xxx@yyy:/usr/share/perl5/JSON/Validator/cache$ ls -la ../../../../openapi-specification/schemas/v2.0/schema.json -rw-r--r-- 1 root root 40247 Feb 16 2021 ../../../../openapi-specification/schemas/v2.0/schema.json xxx@yyy:/usr/share/perl5/JSON/Validator/cache$ ls -la ../../../../openapi-specification/schemas/v3.0/schema.json -rw-r--r-- 1 root root 35456 Feb 16 2021 ../../../../openapi-specification/schemas/v3.0/schema.json Our installation sources: cat /etc/apt/sources.list.d/koha.list deb [signed-by=/usr/share/keyrings/koha-keyring.gpg] http://debian.koha-community.org/koha stable main bullseye I´ve changed rights on /etc/apt/sources.list.d/koha.list and /etc/apt/apt.conf.d/ from 600 to 644 now. Wrong umask setting due to a weird security theatre here :-( -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 --- Comment #8 from David Cook <dcook@prosentient.com.au> --- (In reply to Andreas Tomiak from comment #7)
Some more information:
dpkg-query --list | grep libjson-validator-perl ii libjson-validator-perl 5.99+really5.14-koha2 all module to validate data against a JSON schema
Yeah, that's a bad package. I'm trying to get the package maintainer to fix it. An "apt upgrade" or "apt-get upgrade" would've updated the libjson-validator-perl from the previously working version to this bad version. As a short term measure, you could try downgrading this package, although 24.05 might need the newer one. I haven't tried out 24.05 yet, so I'm not sure. But working on an update to the cached files provided with this package... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 --- Comment #9 from Mason James <mtj@kohaaloha.com> --- (In reply to David Cook from comment #8)
(In reply to Andreas Tomiak from comment #7)
Some more information:
dpkg-query --list | grep libjson-validator-perl ii libjson-validator-perl 5.99+really5.14-koha2 all module to validate data against a JSON schema
Yeah, that's a bad package. I'm trying to get the package maintainer to fix it.
i've added a corrected package to the koha-staging repo, for CI testing the package includes the missing cached schema files in /usr/share/perl5/JSON/Validator/cache the package is located at... https://debian.koha-community.org/koha-staging/pool/main/libj/libjson-valida... -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 --- Comment #10 from Andreas Tomiak <tomiak@helmholtz-berlin.de> --- Hello David, I´ve installed the debian package, removed my patch and restarted koha-common. The API now works as expected. Thanks a lot, fine work. Andreas -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 --- Comment #11 from David Cook <dcook@prosentient.com.au> --- (In reply to Mason James from comment #9)
i've added a corrected package to the koha-staging repo, for CI testing
the package includes the missing cached schema files in /usr/share/perl5/JSON/Validator/cache
the package is located at... https://debian.koha-community.org/koha-staging/pool/main/libj/libjson- validator-perl/libjson-validator-perl_5.99+really5.14-koha3_all.deb
Thanks, Mason. I really appreciate it. I figure once it goes through the CI testing we'll get it out on the stable apt repo? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 Mason James <mtj@kohaaloha.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |In Discussion --- Comment #12 from Mason James <mtj@kohaaloha.com> --- (In reply to David Cook from comment #11)
(In reply to Mason James from comment #9)
i've added a corrected package to the koha-staging repo, for CI testing
the package includes the missing cached schema files in /usr/share/perl5/JSON/Validator/cache
the package is located at... https://debian.koha-community.org/koha-staging/pool/main/libj/libjson- validator-perl/libjson-validator-perl_5.99+really5.14-koha3_all.deb
Thanks, Mason. I really appreciate it.
I figure once it goes through the CI testing we'll get it out on the stable apt repo?
yes! hopefully will push today after next jenkins build -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 --- Comment #13 from Mason James <mtj@kohaaloha.com> ---
I figure once it goes through the CI testing we'll get it out on the stable apt repo?
yes! hopefully will push today after next jenkins build
jenkins build is passing, so i've pushed this package live to 24.05/stable repo $ apt policy libjson-validator-perl libjson-validator-perl: Installed: 5.99+really5.14-koha3 Candidate: 5.99+really5.14-koha3 Version table: *** 5.99+really5.14-koha3 500 500 http://debian.koha-community.org/koha stable/main amd64 Packages 100 /var/lib/dpkg/status 5.14+dfsg-1 500 500 http://ftp.nz.debian.org/debian bookworm/main amd64 Packages -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 --- Comment #14 from David Cook <dcook@prosentient.com.au> --- (In reply to Mason James from comment #13)
jenkins build is passing, so i've pushed this package live to 24.05/stable repo
Huzzah! I think we can probably close this report then? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=37361 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Status|In Discussion |RESOLVED -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org