[Bug 24700] New: Improve Mojo startup speed for REST APIs
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Bug ID: 24700 Summary: Improve Mojo startup speed for REST APIs Change sponsored?: --- Product: Koha Version: master Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: REST API Assignee: koha-bugs@lists.koha-community.org Reporter: ere.maijala@helsinki.fi Currently the API spec is validated multiple times, and the whole spec loaded so far is re-validated for each (if any) REST API plugin. Coming up with a patch to avoid multiple validation unless the complete spec doesn't validate. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Patch complexity|--- |Small patch Assignee|koha-bugs@lists.koha-commun |ere.maijala@helsinki.fi |ity.org | Change sponsored?|--- |Sponsored -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |Needs Signoff -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 --- Comment #1 from Ere Maijala <ere.maijala@helsinki.fi> --- Created attachment 99314 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=99314&action=edit Bug 24700: Avoid validating REST API spec multiple times This patch changes the spec loading so that it is fully loaded and merged with any plugin routes before validation. The individual parts are separately validated only if the resulting complete spec is invalid. Note: the OpenAPI plugin validates the spec given to it, so normally no manual validation is necessary. Test plan: 1. prove -v t/db_dependent/Koha/REST/* 2. prove -v t/db_dependent/api/v1/* Sponsored-by: The National Library of Finland -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Ere Maijala <ere.maijala@helsinki.fi> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |joonas.kylmala@helsinki.fi, | |josef.moravec@gmail.com, | |martin.renvoize@ptfs-europe | |.com, tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |RM_priority -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #99314|0 |1 is obsolete| | --- Comment #2 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 99319 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=99319&action=edit Bug 24700: Avoid validating REST API spec multiple times This patch changes the spec loading so that it is fully loaded and merged with any plugin routes before validation. The individual parts are separately validated only if the resulting complete spec is invalid. Note: the OpenAPI plugin validates the spec given to it, so normally no manual validation is necessary. Test plan: 1. prove -v t/db_dependent/Koha/REST/* 2. prove -v t/db_dependent/api/v1/* Sponsored-by: The National Library of Finland Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #99319|0 |1 is obsolete| | --- Comment #3 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Created attachment 99320 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=99320&action=edit Bug 24700: Avoid validating REST API spec multiple times This patch changes the spec loading so that it is fully loaded and merged with any plugin routes before validation. The individual parts are separately validated only if the resulting complete spec is invalid. Note: the OpenAPI plugin validates the spec given to it, so normally no manual validation is necessary. Test plan: 1. prove -v t/db_dependent/Koha/REST/* 2. prove -v t/db_dependent/api/v1/* Sponsored-by: The National Library of Finland Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs Signoff |Signed Off --- Comment #4 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- This is a nice improvement and the code is fairly clear. I'm struggling a little to find where this is tested to date though.. so I haven't run tests against it yet to fully prove there are no regressions.. human testing gave me no issues though so I'm signing off. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 --- Comment #5 from Ere Maijala <ere.maijala@helsinki.fi> --- t/db_dependent/Koha/REST/Plugin/PluginRoutes.t contains the tests for PluginRoutes, but I don't think there's anything for V1.pm's startup. I can try to create something, but it's pretty huge if all that the startup method does is to be tested. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 --- Comment #6 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- I'll just verify for myself that the pluginroutes tests cover us.. thanks for the pointer. -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julian.maurice@biblibre.com | |, katrin.fischer@bsz-bw.de --- Comment #7 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Don't understand the code well enough to QA here - still gave it a go to provoke some movement. Might be my data, but I have failing tests with and without the patch: Test Summary Report ------------------- t/db_dependent/Koha/REST/Plugin/PluginRoutes.t (Wstat: 65280 Tests: 2 Failed: 1) Failed test: 2 Non-zero exit status: 255 # Failed test 'No tests run for subtest "Disabled plugins tests"' # at t/db_dependent/Koha/REST/Plugin/PluginRoutes.t line 121. Can't locate Koha/Plugin/Com/ByWaterSolutions/KitchenSink.pm in @INC (you may need to install the Koha::Plugin::Com::ByWaterSolutions::KitchenSink module) (@INC contains: /home/vagrant/kohaclone /home/vagrant/qa-test-tools /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base t/db_dependent/Koha/REST/Plugin/../../../../lib) at /usr/share/perl/5.24/Module/Load.pm line 77. Can't locate Koha/Plugin/Com/ByWaterSolutions/KitchenSink in @INC (@INC contains: /home/vagrant/kohaclone /home/vagrant/qa-test-tools /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.24.1 /usr/local/share/perl/5.24.1 /usr/lib/x86_64-linux-gnu/perl5/5.24 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.24 /usr/share/perl/5.24 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base t/db_dependent/Koha/REST/Plugin/../../../../lib) at /usr/share/perl/5.24/Module/Load.pm line 77. # Looks like your test exited with 255 just after 2. Dubious, test returned 255 (wstat 65280, 0xff00) Failed 1/2 subtests t/db_dependent/api/v1/acquisitions_orders.t (Wstat: 256 Tests: 5 Failed: 1) Failed test: 1 Non-zero exit status: 1 [Sun Mar 22 13:31:12 2020] [debug] GET "/api/v1/acquisitions/orders" [Sun Mar 22 13:31:12 2020] [debug] Routing to controller "Koha::REST::V1::Auth" and action "under" [Sun Mar 22 13:31:12 2020] [debug] 400 Bad Request (0.027112s, 36.884/s) ok 6 - GET //W0txdhLojIJjsOm41yB0E6aNafhS1vdorHkDUn9Z17KvscSt:thePassword123@/api/v1/acquisitions/orders?order_blah=blah ok 7 - 400 Bad Request ok 8 - exact match for JSON Pointer "" # Looks like you failed 2 tests of 8. not ok 1 - list() tests -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- QA Contact| |tomascohen@gmail.com -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Signed Off |Passed QA -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Tomás Cohen Arazi <tomascohen@gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #99320|0 |1 is obsolete| | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 --- Comment #8 from Tomás Cohen Arazi <tomascohen@gmail.com> --- Created attachment 101906 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=101906&action=edit Bug 24700: Avoid validating REST API spec multiple times This patch changes the spec loading so that it is fully loaded and merged with any plugin routes before validation. The individual parts are separately validated only if the resulting complete spec is invalid. Note: the OpenAPI plugin validates the spec given to it, so normally no manual validation is necessary. Test plan: 1. prove -v t/db_dependent/Koha/REST/* 2. prove -v t/db_dependent/api/v1/* Sponsored-by: The National Library of Finland Signed-off-by: Martin Renvoize <martin.renvoize@ptfs-europe.com> Signed-off-by: Tomas Cohen Arazi <tomascohen@theke.io> -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Martin Renvoize <martin.renvoize@ptfs-europe.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Version(s)| |20.05.00 released in| | Status|Passed QA |Pushed to master -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 --- Comment #9 from Martin Renvoize <martin.renvoize@ptfs-europe.com> --- Nice work everyone! Pushed to master for 20.05 -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Joy Nelson <joy@bywatersolutions.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |joy@bywatersolutions.com --- Comment #10 from Joy Nelson <joy@bywatersolutions.com> --- enhancement not backported to 19.11.x -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jonathan.druart@bugs.koha-c | |ommunity.org Keywords|RM_priority | -- You are receiving this mail because: You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700 Jonathan Druart <jonathan.druart@bugs.koha-community.org> changed: What |Removed |Added ---------------------------------------------------------------------------- Blocks| |25504 Referenced Bugs: https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=25504 [Bug 25504] Wrong API spec breaks plack without meaningful error -- You are receiving this mail because: You are watching all bug changes.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org