<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body>A new request with request id 18552 has been created by koha-devel-request@lists.koha-community.org. Short info on the request is : <br><br>Title : Koha-devel Digest, Vol 185, Issue 31<br>Category : <br>Description : <div>Send Koha-devel mailing list submissions to<br> koha-devel@lists.koha-community.org<br><br>To subscribe or unsubscribe via the World Wide Web, visit<br> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>or, via email, send a message with subject or body 'help' to<br> koha-devel-request@lists.koha-community.org<br><br>You can reach the person managing the list at<br> koha-devel-owner@lists.koha-community.org<br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of Koha-devel digest..."<br><br><br>Today's Topics:<br><br> 1. Re: Optimizing Starman startup (dcook@prosentient.com.au)<br> 2. Re: Optimizing Starman startup (dcook@prosentient.com.au)<br><br><br>----------------------------------------------------------------------<br><br>Message: 1<br>Date: Wed, 28 Apr 2021 14:29:56 +1000<br>From: <dcook@prosentient.com.au><br>To: "'Ere Maijala'" <ere.maijala@helsinki.fi>, "'Renvoize, Martin'"<br> <martin.renvoize@ptfs-europe.com><br>Cc: "'Koha Devel'" <koha-devel@lists.koha-community.org><br>Subject: Re: [Koha-devel] Optimizing Starman startup<br>Message-ID: <044a01d73be7$24a22810$6de67830$@prosentient.com.au><br>Content-Type: text/plain; charset="utf-8"<br><br>Based on my request, Jan has added an option to skip validation (https://github.com/jhthorsen/mojolicious-plugin-openapi/commit/673079d19f827ce8c8ab3a2943a4abc798fa1e18). <br><br>However, he doesn't seem to want to implement my cache idea. He has said that he's willing to consider a PR, so I'll look at doing that some evening after work.<br><br>He did propose freezing a JSON::Validator object and passing that between processes, but I don't really like that idea. <br><br>David Cook<br>Software Engineer<br>Prosentient Systems<br>Suite 7.03<br>6a Glen St<br>Milsons Point NSW 2061<br>Australia<br><br>Office: 02 9212 0899<br>Online: 02 8005 0595<br><br>-----Original Message-----<br>From: Ere Maijala <ere.maijala@helsinki.fi> <br>Sent: Tuesday, 27 April 2021 3:28 PM<br>To: dcook@prosentient.com.au; 'Renvoize, Martin' <martin.renvoize@ptfs-europe.com><br>Cc: 'Koha Devel' <koha-devel@lists.koha-community.org><br>Subject: Re: [Koha-devel] Optimizing Starman startup<br><br>Thanks for digging up all the information. I think adding a caching validator would provide a nice improvement, so I'm in favor of it, and if I were to decide, I'd also include it in Koha.<br><br>Still, getting it upstream would be great in the long run, so I'd support that as well.<br><br>--Ere<br><br>dcook@prosentient.com.au kirjoitti 27.4.2021 klo 5.34:<br>> After more experimenting, I have concluded that<br>> JSON::Validator->validate() is the culprit in terms of CPU time and <br>> memory usage.<br>> <br>> Fortunately, I’ve determined that Mojo::JSON and Digest::MD5 can be <br>> used together to create consistent reproducible checksums, which could <br>> be used for caching validated schemas.<br>> <br>> Of course, a solution would involve changes to JSON::Validator (and <br>> possibly Mojolicious::Plugin::OpenAPI depending on the chosen <br>> solution), and then we’d have to wait for the new and improved version <br>> to come downstream, so we wouldn’t see the benefit of this for years.<br>> <br>> That said… we could always roll our own JSON::Validator. And if we <br>> don’t want to do it as a community, I could always just do it myself.<br>> <br>> In terms of testing… with 18 CPUs I can restart 60 instances (120<br>> processes) and get through the app setup in about 60 seconds with <br>> significant server load, when using validation. Without validation, I <br>> can do it in 20 seconds without significant server load (beyond a few <br>> short-lived CPU spikes).<br>> <br>> I’m thinking about writing a patch and sending a pull request for <br>> JSON::Validator, but also really thinking about implementing it <br>> locally too at least in the meantime.<br>> <br>> I haven’t heard from the author of JSON::Validator for a little while <br>> now, but I hope I do hear back from him. I think it would be a great <br>> addition to the library.<br>> <br>> David Cook<br>> <br>> Software Engineer<br>> <br>> Prosentient Systems<br>> <br>> Suite 7.03<br>> <br>> 6a Glen St<br>> <br>> Milsons Point NSW 2061<br>> <br>> Australia<br>> <br>> Office: 02 9212 0899<br>> <br>> Online: 02 8005 0595<br>> <br>> *From:*Koha-devel <koha-devel-bounces@lists.koha-community.org> *On <br>> Behalf Of *dcook@prosentient.com.au<br>> *Sent:* Monday, 26 April 2021 7:01 PM<br>> *To:* 'Renvoize, Martin' <martin.renvoize@ptfs-europe.com><br>> *Cc:* 'Koha Devel' <koha-devel@lists.koha-community.org><br>> *Subject:* Re: [Koha-devel] Optimizing Starman startup<br>> <br>> After some more experimenting, it’s clear that the problem isn’t <br>> JSON::Validator::OpenAPI::Mojolicious or <br>> Koha::REST::Plugin::PluginRoutes. If you exclude <br>> Mojolicious::Plugin::OpenAPI, the startup is very fast. It’s 30 <br>> seconds start to finish to restart 60 instances and each instance <br>> restarts very quickly.<br>> <br>> When using Mojolicious::Plugin::OpenAPI, it takes about 3 minutes and <br>> there’s a fair bit of downtime during that time.<br>> <br>> When I do a strace, I’m noticing that a process can spend 30 seconds <br>> just allocating memory for Mojolicious::Plugin::OpenAPI, but it only <br>> happens once you hit a certain volume of processes. If you’re just <br>> starting up 1 or 2, then it’s only a couple seconds. But if you have <br>> say<br>> 60-120 processes, it can take up to 30 seconds for <br>> Mojolicious::Plugin::OpenAPI to do its work. I’m putting 10 CPUs to <br>> this work, but clearly that’s not enough. I imagine there may be other <br>> bottlenecks accessing the memory as well.<br>> <br>> Has anyone profiled Mojolicious before? I’m guessing maybe Martin?<br>> <br>> I suspect that this is just a problem that I’m going to have to live <br>> with but maybe it is a case where I can find a way to optimize <br>> Mojolicious::Plugin::OpenAPI.<br>> <br>> David Cook<br>> <br>> Software Engineer<br>> <br>> Prosentient Systems<br>> <br>> Suite 7.03<br>> <br>> 6a Glen St<br>> <br>> Milsons Point NSW 2061<br>> <br>> Australia<br>> <br>> Office: 02 9212 0899<br>> <br>> Online: 02 8005 0595<br>> <br>> *From:*Koha-devel <koha-devel-bounces@lists.koha-community.org<br>> <mailto:koha-devel-bounces@lists.koha-community.org>> *On Behalf Of <br>> *dcook@prosentient.com.au <mailto:dcook@prosentient.com.au><br>> *Sent:* Monday, 26 April 2021 5:12 PM<br>> *To:* 'Renvoize, Martin' <martin.renvoize@ptfs-europe.com <br>> <mailto:martin.renvoize@ptfs-europe.com>><br>> *Cc:* 'Koha Devel' <koha-devel@lists.koha-community.org<br>> <mailto:koha-devel@lists.koha-community.org>><br>> *Subject:* Re: [Koha-devel] Optimizing Starman startup<br>> <br>> So I just tried the following…<br>> <br>> --<br>> <br>> root@kohadevbox:koha(master)$ npm install -g swagger-cli<br>> <br>> /usr/bin/swagger-cli -> <br>> /usr/lib/node_modules/swagger-cli/swagger-cli.js<br>> <br>> npm WARN @apidevtools/swagger-parser@10.0.2 requires a peer of<br>> openapi-types@>=7 but none is installed. You must install peer <br>> dependencies yourself.<br>> <br>> + swagger-cli@4.0.4 <mailto:swagger-cli@4.0.4><br>> <br>> added 46 packages from 27 contributors in 8.203s<br>> <br>> --<br>> <br>> root@kohadevbox:koha(master)$ time swagger-cli bundle <br>> api/v1/swagger/swagger.json --outfile api/v1/swagger/openapi.json <br>> --type json<br>> <br>> Created api/v1/swagger/openapi.json from api/v1/swagger/swagger.json<br>> <br>> real 0m0.296s<br>> <br>> user 0m0.346s<br>> <br>> sys 0m0.032s<br>> <br>> openapi.json is 10891 lines long but it actually contains 741 $ref <br>> lines like "$ref": "#/definitions/error" and "$ref":<br>> "#/definitions/patron_extended_attribute".<br>> <br>> --<br>> <br>> Now to do some benchmarking… I ran the following code:<br>> <br>> #!/usr/bin/perl<br>> <br>> use JSON::Validator::OpenAPI::Mojolicious;<br>> <br>> my $validator = JSON::Validator::OpenAPI::Mojolicious->new;<br>> <br>> my $spec = $validator->bundle({<br>> <br>> replace => 1,<br>> <br>> schema => "api/v1/swagger/swagger.json",<br>> <br>> });<br>> <br>> The first time I ran it… it took 1.343 seconds. The second time and <br>> subsequent times it took .354 seconds. (That’s using Ubuntu 20.04 and <br>> JSON Validator 3.14.) That suggests caching although I’m not sure where.<br>> I don’t see anything obvious in /usr/share/perl5/JSON/Validator/cache.<br>> <br>> Trying with openapi.json yields .280 seconds instead of .354 seconds. <br>> It’s faster, but not significantly.<br>> <br>> So that suggests that the problem is actually with <br>> Koha::REST::Plugin::PluginRoutes or Mojolicious::Plugin::OpenAPI more <br>> specifically…<br>> <br>> David Cook<br>> <br>> Software Engineer<br>> <br>> Prosentient Systems<br>> <br>> Suite 7.03<br>> <br>> 6a Glen St<br>> <br>> Milsons Point NSW 2061<br>> <br>> Australia<br>> <br>> Office: 02 9212 0899<br>> <br>> Online: 02 8005 0595<br>> <br>> *From:*dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> <br>> <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au>><br>> *Sent:* Monday, 26 April 2021 11:24 AM<br>> *To:* 'Renvoize, Martin' <martin.renvoize@ptfs-europe.com <br>> <mailto:martin.renvoize@ptfs-europe.com>><br>> *Cc:* 'Koha Devel' <koha-devel@lists.koha-community.org<br>> <mailto:koha-devel@lists.koha-community.org>><br>> *Subject:* RE: [Koha-devel] Optimizing Starman startup<br>> <br>> I think that I accidentally offended him, as he hasn’t responded to me <br>> since his initial response.<br>> <br>> I do wonder if reducing the number of references would help, although <br>> I wonder how easy that would be to do in practice. It looks like we <br>> have about 5767 lines of JSON all up as is… so it would probably get <br>> even bigger if we dereferenced them.<br>> <br>> Oh… here’s a thought. Why don’t we compile it? According to <br>> https://davidgarcia.dev/posts/how-to-split-open-api-spec-into-multiple<br>> -files/ <br>> <https://davidgarcia.dev/posts/how-to-split-open-api-spec-into-multipl<br>> e-files/>, you can maintain many different files, and then use <br>> something like swagger-cli to create a single built/compiled OpenAPI <br>> file.<br>> <br>> That way JSON::Validator wouldn’t need to resolve any references for <br>> the core API. I don’t know if the plugins have any $ref in them but <br>> I’m guessing not (just based on Coverflow). So that could be a big win.<br>> <br>> I’m working on other things at the moment, but I’m going to put that <br>> on my eternal list.<br>> <br>> David Cook<br>> <br>> Software Engineer<br>> <br>> Prosentient Systems<br>> <br>> Suite 7.03<br>> <br>> 6a Glen St<br>> <br>> Milsons Point NSW 2061<br>> <br>> Australia<br>> <br>> Office: 02 9212 0899<br>> <br>> Online: 02 8005 0595<br>> <br>> *From:*Renvoize, Martin <martin.renvoize@ptfs-europe.com <br>> <mailto:martin.renvoize@ptfs-europe.com>><br>> *Sent:* Friday, 23 April 2021 5:24 PM<br>> *To:* David Cook <dcook@prosentient.com.au <br>> <mailto:dcook@prosentient.com.au>><br>> *Cc:* Koha Devel <koha-devel@lists.koha-community.org<br>> <mailto:koha-devel@lists.koha-community.org>><br>> *Subject:* Re: [Koha-devel] Optimizing Starman startup<br>> <br>> Jan's code is certainly challenging to read and understand at times I <br>> agree.. I used to contribute to the plugin a number of years ago now..<br>> but the project that gave me time to play with that has since been <br>> sold on so I'm not involved at the level I used to be.. he uses lots <br>> of Perl foo which often takes me a long time to wrap my head around.<br>> <br>> As for the refs, I think we split our spec up too much in all honesty.. <br>> even the swagger spec suggest we went too far.. I think I might have <br>> been unclear when I first pushed for a split from one massive file. <br>> We could/should certainly reduce that somewhat.. it'll be <br>> interesting to see if it makes much difference.. that could be a fairly quick win.<br>> <br>> On Thu, 22 Apr 2021, 12:32 am , <dcook@prosentient.com.au <br>> <mailto:dcook@prosentient.com.au>> wrote:<br>> <br>> Hi Ere,<br>> <br>> I think you're right about the refs. While they get resolved by the<br>> OpenAPI plugin, you probably have to resolve them before trying to<br>> dynamically inject the routes from plugins.<br>> <br>> Jan Thorsen (the author of Mojolicious::Plugin::OpenAPI and<br>> JSON::Validator) thinks that the ref resolution is actually what's<br>> taking so long. I looked it up and I think we have over 400<br>> different references in the main OpenAPI spec alone. I haven't<br>> profiled it but something to think about.<br>> <br>> At some point, I'm going to have a play with newer versions of the<br>> modules. I'm gong to look at Ubuntu 20.04 and newer Debian versions<br>> to see what I can get away with in terms of newness. Needs more<br>> investigation, but I am really hoping that this is an issue that can<br>> be solved by just upgrading the OS.<br>> <br>> I find Jan's code to be unnecessarily opaque (could use more<br>> descriptive comments and function naming) but... I'll investigate.<br>> Probably not right away as I have a bunch of other priorities that I<br>> have to address but... this is on my mind.<br>> <br>> Starman startup time is probably the thing about Koha annoying me<br>> the most right now and probably the most practical thing I can<br>> improve at the moment...<br>> <br>> David Cook<br>> Software Engineer<br>> Prosentient Systems<br>> Suite 7.03<br>> 6a Glen St<br>> Milsons Point NSW 2061<br>> Australia<br>> <br>> Office: 02 9212 0899<br>> Online: 02 8005 0595<br>> <br>> -----Original Message-----<br>> From: Ere Maijala <ere.maijala@helsinki.fi<br>> <mailto:ere.maijala@helsinki.fi>><br>> Sent: Wednesday, 21 April 2021 6:31 PM<br>> To: dcook@prosentient.com.au <mailto:dcook@prosentient.com.au>;<br>> koha-devel@lists.koha-community.org<br>> <mailto:koha-devel@lists.koha-community.org><br>> Subject: Re: [Koha-devel] Optimizing Starman startup<br>> <br>> Hi David,<br>> <br>> I wish I'd remember all the details, but my memory fails me. I think<br>> not using JSON had something to do with how the refs are resolved.<br>> That may or may not have been the reason, but if everything works<br>> with JSON module, I can't think of a reason not to use it.<br>> <br>> Thanks for taking a look!<br>> <br>> --Ere<br>> <br>> dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> kirjoitti<br>> 21.4.2021 klo 3.28:<br>> > Hi Ere,<br>> ><br>> > Thanks for your reply. 24700 looks much better. I'll look at<br>> backporting it locally.<br>> ><br>> > Although I'm looking at JSON::Validator::OpenAPI::Mojolicious at<br>> https://metacpan.org/pod/release/JHTHORSEN/Mojolicious-Plugin-OpenAPI-2.19/lib/JSON/Validator/OpenAPI/Mojolicious.pm<br>> <https://metacpan.org/pod/release/JHTHORSEN/Mojolicious-Plugin-OpenAPI-2.19/lib/JSON/Validator/OpenAPI/Mojolicious.pm><br>> and it says "Do not use this module directly. Use<br>> Mojolicious::Plugin::OpenAPI instead." I notice that you're using<br>> the "bundle" method. Do we really need that there? Why don't we just<br>> load the JSON using the JSON module, merge with the plugin spec<br>> files, and then pass it to the OpenAPI plugin? Shouldn't the plugin<br>> take care of the $ref replacement?<br>> ><br>> > Hmm... I didn't realize until now that the OpenAPI plugin was<br>> doing a validate behind the scenes. That's tricky.<br>> ><br>> > At a glance, we might be able to pre-load the app into the Starman<br>> > master process pre-fork. There are warnings about doing that with<br>> open<br>> > database connections, so we'd need to review plack.psgi, but a quick<br>> > glance suggests it might be OK. (Alternatively, I have wondered<br>> about<br>> > running the REST API as a separate process apart from Starman using<br>> > hypnotoad. According to<br>> > https://docs.mojolicious.org/Mojolicious/Guides/Cookbook<br>> <https://docs.mojolicious.org/Mojolicious/Guides/Cookbook>,<br>> > Mojo::Server::Prefork preloads the application in the manager/master<br>> > process, and Hypnotoad is based off that, so that would help.)<br>> ><br>> > It does seem like changes to the OpenAPI plugin would be needed<br>> for caching.<br>> ><br>> > I'm going to try backporting your change and try pre-loading and<br>> see how far that gets me.<br>> ><br>> > David Cook<br>> > Software Engineer<br>> > Prosentient Systems<br>> > Suite 7.03<br>> > 6a Glen St<br>> > Milsons Point NSW 2061<br>> > Australia<br>> ><br>> > Office: 02 9212 0899<br>> > Online: 02 8005 0595<br>> ><br>> > -----Original Message-----<br>> > From: Koha-devel <koha-devel-bounces@lists.koha-community.org<br>> <mailto:koha-devel-bounces@lists.koha-community.org>> On<br>> > Behalf Of Ere Maijala<br>> > Sent: Tuesday, 20 April 2021 4:48 PM<br>> > To: koha-devel@lists.koha-community.org<br>> <mailto:koha-devel@lists.koha-community.org><br>> > Subject: Re: [Koha-devel] Optimizing Starman startup<br>> ><br>> > Hi,<br>> ><br>> > I did some work on improving it here:<br>> ><br>> > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700<br>> <https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700><br>> ><br>> > That shaved a good bit of time from it, but it's still a heavy<br>> > operation, and it would make sense to<br>> ><br>> > 1.) avoid doing it too often<br>> ><br>> > 2.) cache the results and avoid doing it if results are cached<br>> ><br>> > If you could address the first one, that'd go a long way. I'm<br>> afraid the second one would require changes to the OpenAPI plugin to<br>> support caching.<br>> ><br>> > --Ere<br>> ><br>> > dcook@prosentient.com.au <mailto:dcook@prosentient.com.au><br>> kirjoitti 20.4.2021 klo 6.15:<br>> >> Hi all,<br>> >><br>> >> Do you despair when you see the following periodically in “top”<br>> when<br>> >> a starman worker is recreated ?<br>> >><br>> >> PID USER PR NI VIRT RES SHR S %CPU %MEM <br>> TIME+<br>> >> COMMAND<br>> >><br>> >> 9529 my-koha 20 0 460108 197212 17172 R 100.0 0.4 0:03.41<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> Or the following in top when you install koha-common package or<br>> >> restart the koha-common service?<br>> >><br>> >> 11101 1-koha 20 0 447232 193320 16076 R 10.6 0.4 0:09.09<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11168 1-koha 20 0 447240 193264 16056 R 10.6 0.4 0:08.72<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11306 2-koha 20 0 447220 193148 16000 R 10.6 0.4 0:08.07<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11543 2-koha 20 0 447232 193036 15828 R 10.6 0.4 0:07.07<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11784 3-koha 20 0 441536 189664 16172 R 10.6 0.4 0:06.04<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11830 3-koha 20 0 439548 187212 15748 R 10.6 0.4 0:05.82<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11831 4-koha 20 0 438620 186344 15748 R 10.6 0.4 0:05.81<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11853 4-koha 20 0 437680 185672 16000 R 10.6 0.4 0:05.79<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> Well, I still have a lot of investigation left to do, but I<br>> notice 1<br>> >> place that a lot of time taken is here (per worker):<br>> >><br>> >> my $validator = JSON::Validator::OpenAPI::Mojolicious->new;<br>> >><br>> >> $validator->load_and_validate_schema(<br>> >><br>> >> $self->home->rel_file("api/v1/swagger/swagger.json"),<br>> >><br>> >> {<br>> >><br>> >> allow_invalid_ref => 1,<br>> >><br>> >> }<br>> >><br>> >> );<br>> >><br>> >> push @{$self->routes->namespaces}, 'Koha::Plugin';<br>> >><br>> >> my $spec = $validator->schema->data;<br>> >><br>> >> $self->plugin(<br>> >><br>> >> 'Koha::REST::Plugin::PluginRoutes' => {<br>> >><br>> >> spec => $spec,<br>> >><br>> >> validator => $validator<br>> >><br>> >> }<br>> >><br>> >> );<br>> >><br>> >> $self->plugin(<br>> >><br>> >> OpenAPI => {<br>> >><br>> >> spec => $spec,<br>> >><br>> >> route =><br>> >> $self->routes->under('/api/v1')->to('Auth#under'),<br>> >><br>> >> allow_invalid_ref =><br>> >><br>> >> 1, # required by our spec because $ref directly<br>> >> under<br>> >><br>> >> # Paths-, Parameters-, Definitions- &<br>> >> Info-object<br>> >><br>> >> # is not allowed by the OpenAPI specification.<br>> >><br>> >> }<br>> >><br>> >> );<br>> >><br>> >> Anyone have ideas for improving this? Do we have to validate the<br>> >> schema every time? Can we move the schema validation into a<br>> different<br>> >> module and preload it into Starman using the -M flag so that it’s<br>> >> done<br>> >> 1 time per Starman master instance rather than 1 time per<br>> Starman worker instance?<br>> >><br>> >> I find “/usr/share/koha/api/v1/app.pl <http://app.pl>” to be the<br>> bane of deployments,<br>> >> as it puts a massive load on a server, when you have multiple Koha<br>> >> instances on the server.<br>> >><br>> >> David Cook<br>> >><br>> >> Software Engineer<br>> >><br>> >> Prosentient Systems<br>> >><br>> >> Suite 7.03<br>> >><br>> >> 6a Glen St<br>> >><br>> >> Milsons Point NSW 2061<br>> >><br>> >> Australia<br>> >><br>> >> Office: 02 9212 0899<br>> >><br>> >> Online: 02 8005 0595<br>> >><br>> >><br>> >> _______________________________________________<br>> >> Koha-devel mailing list<br>> >> Koha-devel@lists.koha-community.org<br>> <mailto:Koha-devel@lists.koha-community.org><br>> >><br>> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>> <https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel><br>> >> website : https://www.koha-community.org/<br>> <https://www.koha-community.org/> git :<br>> >> https://git.koha-community.org/<br>> <https://git.koha-community.org/> bugs :<br>> >> https://bugs.koha-community.org/ <https://bugs.koha-community.org/><br>> >><br>> ><br>> > --<br>> > Ere Maijala<br>> > Kansalliskirjasto / The National Library of Finland<br>> > _______________________________________________<br>> > Koha-devel mailing list<br>> > Koha-devel@lists.koha-community.org<br>> <mailto:Koha-devel@lists.koha-community.org><br>> ><br>> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>> <https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel><br>> > website : https://www.koha-community.org/<br>> <https://www.koha-community.org/> git :<br>> > https://git.koha-community.org/ <https://git.koha-community.org/><br>> bugs :<br>> > https://bugs.koha-community.org/ <https://bugs.koha-community.org/><br>> ><br>> ><br>> <br>> --<br>> Ere Maijala<br>> Kansalliskirjasto / The National Library of Finland<br>> <br>> <br>> _______________________________________________<br>> Koha-devel mailing list<br>> Koha-devel@lists.koha-community.org<br>> <mailto:Koha-devel@lists.koha-community.org><br>> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>> <https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel><br>> website : https://www.koha-community.org/<br>> <https://www.koha-community.org/><br>> git : https://git.koha-community.org/ <https://git.koha-community.org/><br>> bugs : https://bugs.koha-community.org/<br>> <https://bugs.koha-community.org/><br>> <br><br>--<br>Ere Maijala<br>Kansalliskirjasto / The National Library of Finland<br><br><br><br><br>------------------------------<br><br>Message: 2<br>Date: Wed, 28 Apr 2021 14:40:09 +1000<br>From: <dcook@prosentient.com.au><br>To: "'Ere Maijala'" <ere.maijala@helsinki.fi>, "'Renvoize, Martin'"<br> <martin.renvoize@ptfs-europe.com><br>Cc: "'Koha Devel'" <koha-devel@lists.koha-community.org><br>Subject: Re: [Koha-devel] Optimizing Starman startup<br>Message-ID: <044d01d73be8$91960b20$b4c22160$@prosentient.com.au><br>Content-Type: text/plain; charset="utf-8"<br><br>But now that I think about it again... Jan's change is actually all we need.<br><br>Within Koha, we could do the checksum calculation, and then skip the validation if we have a local record that our schema has already been validated. (I guess we'd set the cache after Mojoliciou::Plugin::OpenAPI successfully registers without dying from errors.)<br><br>So we don't need to change JSON::Validator at all. We just need a Mojolicious::Plugin::OpenAPI that allows us to skip validation. We then handle the checksum and caching within our application. <br><br>*mindblown*<br><br>Now we just need for Jan to release the new version and for Koha to catch up to the point of being able to use that version hehe. <br><br>David Cook<br>Software Engineer<br>Prosentient Systems<br>Suite 7.03<br>6a Glen St<br>Milsons Point NSW 2061<br>Australia<br><br>Office: 02 9212 0899<br>Online: 02 8005 0595<br><br>-----Original Message-----<br>From: dcook@prosentient.com.au <dcook@prosentient.com.au> <br>Sent: Wednesday, 28 April 2021 2:30 PM<br>To: 'Ere Maijala' <ere.maijala@helsinki.fi>; 'Renvoize, Martin' <martin.renvoize@ptfs-europe.com><br>Cc: 'Koha Devel' <koha-devel@lists.koha-community.org><br>Subject: RE: [Koha-devel] Optimizing Starman startup<br><br>Based on my request, Jan has added an option to skip validation (https://github.com/jhthorsen/mojolicious-plugin-openapi/commit/673079d19f827ce8c8ab3a2943a4abc798fa1e18). <br><br>However, he doesn't seem to want to implement my cache idea. He has said that he's willing to consider a PR, so I'll look at doing that some evening after work.<br><br>He did propose freezing a JSON::Validator object and passing that between processes, but I don't really like that idea. <br><br>David Cook<br>Software Engineer<br>Prosentient Systems<br>Suite 7.03<br>6a Glen St<br>Milsons Point NSW 2061<br>Australia<br><br>Office: 02 9212 0899<br>Online: 02 8005 0595<br><br>-----Original Message-----<br>From: Ere Maijala <ere.maijala@helsinki.fi><br>Sent: Tuesday, 27 April 2021 3:28 PM<br>To: dcook@prosentient.com.au; 'Renvoize, Martin' <martin.renvoize@ptfs-europe.com><br>Cc: 'Koha Devel' <koha-devel@lists.koha-community.org><br>Subject: Re: [Koha-devel] Optimizing Starman startup<br><br>Thanks for digging up all the information. I think adding a caching validator would provide a nice improvement, so I'm in favor of it, and if I were to decide, I'd also include it in Koha.<br><br>Still, getting it upstream would be great in the long run, so I'd support that as well.<br><br>--Ere<br><br>dcook@prosentient.com.au kirjoitti 27.4.2021 klo 5.34:<br>> After more experimenting, I have concluded that<br>> JSON::Validator->validate() is the culprit in terms of CPU time and <br>> memory usage.<br>> <br>> Fortunately, I’ve determined that Mojo::JSON and Digest::MD5 can be <br>> used together to create consistent reproducible checksums, which could <br>> be used for caching validated schemas.<br>> <br>> Of course, a solution would involve changes to JSON::Validator (and <br>> possibly Mojolicious::Plugin::OpenAPI depending on the chosen <br>> solution), and then we’d have to wait for the new and improved version <br>> to come downstream, so we wouldn’t see the benefit of this for years.<br>> <br>> That said… we could always roll our own JSON::Validator. And if we <br>> don’t want to do it as a community, I could always just do it myself.<br>> <br>> In terms of testing… with 18 CPUs I can restart 60 instances (120<br>> processes) and get through the app setup in about 60 seconds with <br>> significant server load, when using validation. Without validation, I <br>> can do it in 20 seconds without significant server load (beyond a few <br>> short-lived CPU spikes).<br>> <br>> I’m thinking about writing a patch and sending a pull request for <br>> JSON::Validator, but also really thinking about implementing it <br>> locally too at least in the meantime.<br>> <br>> I haven’t heard from the author of JSON::Validator for a little while <br>> now, but I hope I do hear back from him. I think it would be a great <br>> addition to the library.<br>> <br>> David Cook<br>> <br>> Software Engineer<br>> <br>> Prosentient Systems<br>> <br>> Suite 7.03<br>> <br>> 6a Glen St<br>> <br>> Milsons Point NSW 2061<br>> <br>> Australia<br>> <br>> Office: 02 9212 0899<br>> <br>> Online: 02 8005 0595<br>> <br>> *From:*Koha-devel <koha-devel-bounces@lists.koha-community.org> *On <br>> Behalf Of *dcook@prosentient.com.au<br>> *Sent:* Monday, 26 April 2021 7:01 PM<br>> *To:* 'Renvoize, Martin' <martin.renvoize@ptfs-europe.com><br>> *Cc:* 'Koha Devel' <koha-devel@lists.koha-community.org><br>> *Subject:* Re: [Koha-devel] Optimizing Starman startup<br>> <br>> After some more experimenting, it’s clear that the problem isn’t <br>> JSON::Validator::OpenAPI::Mojolicious or <br>> Koha::REST::Plugin::PluginRoutes. If you exclude <br>> Mojolicious::Plugin::OpenAPI, the startup is very fast. It’s 30 <br>> seconds start to finish to restart 60 instances and each instance <br>> restarts very quickly.<br>> <br>> When using Mojolicious::Plugin::OpenAPI, it takes about 3 minutes and <br>> there’s a fair bit of downtime during that time.<br>> <br>> When I do a strace, I’m noticing that a process can spend 30 seconds <br>> just allocating memory for Mojolicious::Plugin::OpenAPI, but it only <br>> happens once you hit a certain volume of processes. If you’re just <br>> starting up 1 or 2, then it’s only a couple seconds. But if you have <br>> say<br>> 60-120 processes, it can take up to 30 seconds for <br>> Mojolicious::Plugin::OpenAPI to do its work. I’m putting 10 CPUs to <br>> this work, but clearly that’s not enough. I imagine there may be other <br>> bottlenecks accessing the memory as well.<br>> <br>> Has anyone profiled Mojolicious before? I’m guessing maybe Martin?<br>> <br>> I suspect that this is just a problem that I’m going to have to live <br>> with but maybe it is a case where I can find a way to optimize <br>> Mojolicious::Plugin::OpenAPI.<br>> <br>> David Cook<br>> <br>> Software Engineer<br>> <br>> Prosentient Systems<br>> <br>> Suite 7.03<br>> <br>> 6a Glen St<br>> <br>> Milsons Point NSW 2061<br>> <br>> Australia<br>> <br>> Office: 02 9212 0899<br>> <br>> Online: 02 8005 0595<br>> <br>> *From:*Koha-devel <koha-devel-bounces@lists.koha-community.org<br>> <mailto:koha-devel-bounces@lists.koha-community.org>> *On Behalf Of <br>> *dcook@prosentient.com.au <mailto:dcook@prosentient.com.au><br>> *Sent:* Monday, 26 April 2021 5:12 PM<br>> *To:* 'Renvoize, Martin' <martin.renvoize@ptfs-europe.com <br>> <mailto:martin.renvoize@ptfs-europe.com>><br>> *Cc:* 'Koha Devel' <koha-devel@lists.koha-community.org<br>> <mailto:koha-devel@lists.koha-community.org>><br>> *Subject:* Re: [Koha-devel] Optimizing Starman startup<br>> <br>> So I just tried the following…<br>> <br>> --<br>> <br>> root@kohadevbox:koha(master)$ npm install -g swagger-cli<br>> <br>> /usr/bin/swagger-cli -><br>> /usr/lib/node_modules/swagger-cli/swagger-cli.js<br>> <br>> npm WARN @apidevtools/swagger-parser@10.0.2 requires a peer of<br>> openapi-types@>=7 but none is installed. You must install peer <br>> dependencies yourself.<br>> <br>> + swagger-cli@4.0.4 <mailto:swagger-cli@4.0.4><br>> <br>> added 46 packages from 27 contributors in 8.203s<br>> <br>> --<br>> <br>> root@kohadevbox:koha(master)$ time swagger-cli bundle <br>> api/v1/swagger/swagger.json --outfile api/v1/swagger/openapi.json <br>> --type json<br>> <br>> Created api/v1/swagger/openapi.json from api/v1/swagger/swagger.json<br>> <br>> real 0m0.296s<br>> <br>> user 0m0.346s<br>> <br>> sys 0m0.032s<br>> <br>> openapi.json is 10891 lines long but it actually contains 741 $ref <br>> lines like "$ref": "#/definitions/error" and "$ref":<br>> "#/definitions/patron_extended_attribute".<br>> <br>> --<br>> <br>> Now to do some benchmarking… I ran the following code:<br>> <br>> #!/usr/bin/perl<br>> <br>> use JSON::Validator::OpenAPI::Mojolicious;<br>> <br>> my $validator = JSON::Validator::OpenAPI::Mojolicious->new;<br>> <br>> my $spec = $validator->bundle({<br>> <br>> replace => 1,<br>> <br>> schema => "api/v1/swagger/swagger.json",<br>> <br>> });<br>> <br>> The first time I ran it… it took 1.343 seconds. The second time and <br>> subsequent times it took .354 seconds. (That’s using Ubuntu 20.04 and <br>> JSON Validator 3.14.) That suggests caching although I’m not sure where.<br>> I don’t see anything obvious in /usr/share/perl5/JSON/Validator/cache.<br>> <br>> Trying with openapi.json yields .280 seconds instead of .354 seconds. <br>> It’s faster, but not significantly.<br>> <br>> So that suggests that the problem is actually with <br>> Koha::REST::Plugin::PluginRoutes or Mojolicious::Plugin::OpenAPI more <br>> specifically…<br>> <br>> David Cook<br>> <br>> Software Engineer<br>> <br>> Prosentient Systems<br>> <br>> Suite 7.03<br>> <br>> 6a Glen St<br>> <br>> Milsons Point NSW 2061<br>> <br>> Australia<br>> <br>> Office: 02 9212 0899<br>> <br>> Online: 02 8005 0595<br>> <br>> *From:*dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> <br>> <dcook@prosentient.com.au <mailto:dcook@prosentient.com.au>><br>> *Sent:* Monday, 26 April 2021 11:24 AM<br>> *To:* 'Renvoize, Martin' <martin.renvoize@ptfs-europe.com <br>> <mailto:martin.renvoize@ptfs-europe.com>><br>> *Cc:* 'Koha Devel' <koha-devel@lists.koha-community.org<br>> <mailto:koha-devel@lists.koha-community.org>><br>> *Subject:* RE: [Koha-devel] Optimizing Starman startup<br>> <br>> I think that I accidentally offended him, as he hasn’t responded to me <br>> since his initial response.<br>> <br>> I do wonder if reducing the number of references would help, although <br>> I wonder how easy that would be to do in practice. It looks like we <br>> have about 5767 lines of JSON all up as is… so it would probably get <br>> even bigger if we dereferenced them.<br>> <br>> Oh… here’s a thought. Why don’t we compile it? According to <br>> https://davidgarcia.dev/posts/how-to-split-open-api-spec-into-multiple<br>> -files/<br>> <https://davidgarcia.dev/posts/how-to-split-open-api-spec-into-multipl<br>> e-files/>, you can maintain many different files, and then use <br>> something like swagger-cli to create a single built/compiled OpenAPI <br>> file.<br>> <br>> That way JSON::Validator wouldn’t need to resolve any references for <br>> the core API. I don’t know if the plugins have any $ref in them but <br>> I’m guessing not (just based on Coverflow). So that could be a big win.<br>> <br>> I’m working on other things at the moment, but I’m going to put that <br>> on my eternal list.<br>> <br>> David Cook<br>> <br>> Software Engineer<br>> <br>> Prosentient Systems<br>> <br>> Suite 7.03<br>> <br>> 6a Glen St<br>> <br>> Milsons Point NSW 2061<br>> <br>> Australia<br>> <br>> Office: 02 9212 0899<br>> <br>> Online: 02 8005 0595<br>> <br>> *From:*Renvoize, Martin <martin.renvoize@ptfs-europe.com <br>> <mailto:martin.renvoize@ptfs-europe.com>><br>> *Sent:* Friday, 23 April 2021 5:24 PM<br>> *To:* David Cook <dcook@prosentient.com.au <br>> <mailto:dcook@prosentient.com.au>><br>> *Cc:* Koha Devel <koha-devel@lists.koha-community.org<br>> <mailto:koha-devel@lists.koha-community.org>><br>> *Subject:* Re: [Koha-devel] Optimizing Starman startup<br>> <br>> Jan's code is certainly challenging to read and understand at times I <br>> agree.. I used to contribute to the plugin a number of years ago now..<br>> but the project that gave me time to play with that has since been <br>> sold on so I'm not involved at the level I used to be.. he uses lots <br>> of Perl foo which often takes me a long time to wrap my head around.<br>> <br>> As for the refs, I think we split our spec up too much in all honesty.. <br>> even the swagger spec suggest we went too far.. I think I might have <br>> been unclear when I first pushed for a split from one massive file.<br>> We could/should certainly reduce that somewhat.. it'll be <br>> interesting to see if it makes much difference.. that could be a fairly quick win.<br>> <br>> On Thu, 22 Apr 2021, 12:32 am , <dcook@prosentient.com.au <br>> <mailto:dcook@prosentient.com.au>> wrote:<br>> <br>> Hi Ere,<br>> <br>> I think you're right about the refs. While they get resolved by the<br>> OpenAPI plugin, you probably have to resolve them before trying to<br>> dynamically inject the routes from plugins.<br>> <br>> Jan Thorsen (the author of Mojolicious::Plugin::OpenAPI and<br>> JSON::Validator) thinks that the ref resolution is actually what's<br>> taking so long. I looked it up and I think we have over 400<br>> different references in the main OpenAPI spec alone. I haven't<br>> profiled it but something to think about.<br>> <br>> At some point, I'm going to have a play with newer versions of the<br>> modules. I'm gong to look at Ubuntu 20.04 and newer Debian versions<br>> to see what I can get away with in terms of newness. Needs more<br>> investigation, but I am really hoping that this is an issue that can<br>> be solved by just upgrading the OS.<br>> <br>> I find Jan's code to be unnecessarily opaque (could use more<br>> descriptive comments and function naming) but... I'll investigate.<br>> Probably not right away as I have a bunch of other priorities that I<br>> have to address but... this is on my mind.<br>> <br>> Starman startup time is probably the thing about Koha annoying me<br>> the most right now and probably the most practical thing I can<br>> improve at the moment...<br>> <br>> David Cook<br>> Software Engineer<br>> Prosentient Systems<br>> Suite 7.03<br>> 6a Glen St<br>> Milsons Point NSW 2061<br>> Australia<br>> <br>> Office: 02 9212 0899<br>> Online: 02 8005 0595<br>> <br>> -----Original Message-----<br>> From: Ere Maijala <ere.maijala@helsinki.fi<br>> <mailto:ere.maijala@helsinki.fi>><br>> Sent: Wednesday, 21 April 2021 6:31 PM<br>> To: dcook@prosentient.com.au <mailto:dcook@prosentient.com.au>;<br>> koha-devel@lists.koha-community.org<br>> <mailto:koha-devel@lists.koha-community.org><br>> Subject: Re: [Koha-devel] Optimizing Starman startup<br>> <br>> Hi David,<br>> <br>> I wish I'd remember all the details, but my memory fails me. I think<br>> not using JSON had something to do with how the refs are resolved.<br>> That may or may not have been the reason, but if everything works<br>> with JSON module, I can't think of a reason not to use it.<br>> <br>> Thanks for taking a look!<br>> <br>> --Ere<br>> <br>> dcook@prosentient.com.au <mailto:dcook@prosentient.com.au> kirjoitti<br>> 21.4.2021 klo 3.28:<br>> > Hi Ere,<br>> ><br>> > Thanks for your reply. 24700 looks much better. I'll look at<br>> backporting it locally.<br>> ><br>> > Although I'm looking at JSON::Validator::OpenAPI::Mojolicious at<br>> https://metacpan.org/pod/release/JHTHORSEN/Mojolicious-Plugin-OpenAPI-2.19/lib/JSON/Validator/OpenAPI/Mojolicious.pm<br>> <https://metacpan.org/pod/release/JHTHORSEN/Mojolicious-Plugin-OpenAPI-2.19/lib/JSON/Validator/OpenAPI/Mojolicious.pm><br>> and it says "Do not use this module directly. Use<br>> Mojolicious::Plugin::OpenAPI instead." I notice that you're using<br>> the "bundle" method. Do we really need that there? Why don't we just<br>> load the JSON using the JSON module, merge with the plugin spec<br>> files, and then pass it to the OpenAPI plugin? Shouldn't the plugin<br>> take care of the $ref replacement?<br>> ><br>> > Hmm... I didn't realize until now that the OpenAPI plugin was<br>> doing a validate behind the scenes. That's tricky.<br>> ><br>> > At a glance, we might be able to pre-load the app into the Starman<br>> > master process pre-fork. There are warnings about doing that with<br>> open<br>> > database connections, so we'd need to review plack.psgi, but a quick<br>> > glance suggests it might be OK. (Alternatively, I have wondered<br>> about<br>> > running the REST API as a separate process apart from Starman using<br>> > hypnotoad. According to<br>> > https://docs.mojolicious.org/Mojolicious/Guides/Cookbook<br>> <https://docs.mojolicious.org/Mojolicious/Guides/Cookbook>,<br>> > Mojo::Server::Prefork preloads the application in the manager/master<br>> > process, and Hypnotoad is based off that, so that would help.)<br>> ><br>> > It does seem like changes to the OpenAPI plugin would be needed<br>> for caching.<br>> ><br>> > I'm going to try backporting your change and try pre-loading and<br>> see how far that gets me.<br>> ><br>> > David Cook<br>> > Software Engineer<br>> > Prosentient Systems<br>> > Suite 7.03<br>> > 6a Glen St<br>> > Milsons Point NSW 2061<br>> > Australia<br>> ><br>> > Office: 02 9212 0899<br>> > Online: 02 8005 0595<br>> ><br>> > -----Original Message-----<br>> > From: Koha-devel <koha-devel-bounces@lists.koha-community.org<br>> <mailto:koha-devel-bounces@lists.koha-community.org>> On<br>> > Behalf Of Ere Maijala<br>> > Sent: Tuesday, 20 April 2021 4:48 PM<br>> > To: koha-devel@lists.koha-community.org<br>> <mailto:koha-devel@lists.koha-community.org><br>> > Subject: Re: [Koha-devel] Optimizing Starman startup<br>> ><br>> > Hi,<br>> ><br>> > I did some work on improving it here:<br>> ><br>> > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700<br>> <https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700><br>> ><br>> > That shaved a good bit of time from it, but it's still a heavy<br>> > operation, and it would make sense to<br>> ><br>> > 1.) avoid doing it too often<br>> ><br>> > 2.) cache the results and avoid doing it if results are cached<br>> ><br>> > If you could address the first one, that'd go a long way. I'm<br>> afraid the second one would require changes to the OpenAPI plugin to<br>> support caching.<br>> ><br>> > --Ere<br>> ><br>> > dcook@prosentient.com.au <mailto:dcook@prosentient.com.au><br>> kirjoitti 20.4.2021 klo 6.15:<br>> >> Hi all,<br>> >><br>> >> Do you despair when you see the following periodically in “top”<br>> when<br>> >> a starman worker is recreated ?<br>> >><br>> >> PID USER PR NI VIRT RES SHR S %CPU %MEM <br>> TIME+<br>> >> COMMAND<br>> >><br>> >> 9529 my-koha 20 0 460108 197212 17172 R 100.0 0.4 0:03.41<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> Or the following in top when you install koha-common package or<br>> >> restart the koha-common service?<br>> >><br>> >> 11101 1-koha 20 0 447232 193320 16076 R 10.6 0.4 0:09.09<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11168 1-koha 20 0 447240 193264 16056 R 10.6 0.4 0:08.72<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11306 2-koha 20 0 447220 193148 16000 R 10.6 0.4 0:08.07<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11543 2-koha 20 0 447232 193036 15828 R 10.6 0.4 0:07.07<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11784 3-koha 20 0 441536 189664 16172 R 10.6 0.4 0:06.04<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11830 3-koha 20 0 439548 187212 15748 R 10.6 0.4 0:05.82<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11831 4-koha 20 0 438620 186344 15748 R 10.6 0.4 0:05.81<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> 11853 4-koha 20 0 437680 185672 16000 R 10.6 0.4 0:05.79<br>> >> /usr/share/koha/api/v1/app.pl <http://app.pl><br>> >><br>> >> Well, I still have a lot of investigation left to do, but I<br>> notice 1<br>> >> place that a lot of time taken is here (per worker):<br>> >><br>> >> my $validator = JSON::Validator::OpenAPI::Mojolicious->new;<br>> >><br>> >> $validator->load_and_validate_schema(<br>> >><br>> >> $self->home->rel_file("api/v1/swagger/swagger.json"),<br>> >><br>> >> {<br>> >><br>> >> allow_invalid_ref => 1,<br>> >><br>> >> }<br>> >><br>> >> );<br>> >><br>> >> push @{$self->routes->namespaces}, 'Koha::Plugin';<br>> >><br>> >> my $spec = $validator->schema->data;<br>> >><br>> >> $self->plugin(<br>> >><br>> >> 'Koha::REST::Plugin::PluginRoutes' => {<br>> >><br>> >> spec => $spec,<br>> >><br>> >> validator => $validator<br>> >><br>> >> }<br>> >><br>> >> );<br>> >><br>> >> $self->plugin(<br>> >><br>> >> OpenAPI => {<br>> >><br>> >> spec => $spec,<br>> >><br>> >> route =><br>> >> $self->routes->under('/api/v1')->to('Auth#under'),<br>> >><br>> >> allow_invalid_ref =><br>> >><br>> >> 1, # required by our spec because $ref directly<br>> >> under<br>> >><br>> >> # Paths-, Parameters-, Definitions- &<br>> >> Info-object<br>> >><br>> >> # is not allowed by the OpenAPI specification.<br>> >><br>> >> }<br>> >><br>> >> );<br>> >><br>> >> Anyone have ideas for improving this? Do we have to validate the<br>> >> schema every time? Can we move the schema validation into a<br>> different<br>> >> module and preload it into Starman using the -M flag so that it’s<br>> >> done<br>> >> 1 time per Starman master instance rather than 1 time per<br>> Starman worker instance?<br>> >><br>> >> I find “/usr/share/koha/api/v1/app.pl <http://app.pl>” to be the<br>> bane of deployments,<br>> >> as it puts a massive load on a server, when you have multiple Koha<br>> >> instances on the server.<br>> >><br>> >> David Cook<br>> >><br>> >> Software Engineer<br>> >><br>> >> Prosentient Systems<br>> >><br>> >> Suite 7.03<br>> >><br>> >> 6a Glen St<br>> >><br>> >> Milsons Point NSW 2061<br>> >><br>> >> Australia<br>> >><br>> >> Office: 02 9212 0899<br>> >><br>> >> Online: 02 8005 0595<br>> >><br>> >><br>> >> _______________________________________________<br>> >> Koha-devel mailing list<br>> >> Koha-devel@lists.koha-community.org<br>> <mailto:Koha-devel@lists.koha-community.org><br>> >><br>> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>> <https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel><br>> >> website : https://www.koha-community.org/<br>> <https://www.koha-community.org/> git :<br>> >> https://git.koha-community.org/<br>> <https://git.koha-community.org/> bugs :<br>> >> https://bugs.koha-community.org/ <https://bugs.koha-community.org/><br>> >><br>> ><br>> > --<br>> > Ere Maijala<br>> > Kansalliskirjasto / The National Library of Finland<br>> > _______________________________________________<br>> > Koha-devel mailing list<br>> > Koha-devel@lists.koha-community.org<br>> <mailto:Koha-devel@lists.koha-community.org><br>> ><br>> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>> <https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel><br>> > website : https://www.koha-community.org/<br>> <https://www.koha-community.org/> git :<br>> > https://git.koha-community.org/ <https://git.koha-community.org/><br>> bugs :<br>> > https://bugs.koha-community.org/ <https://bugs.koha-community.org/><br>> ><br>> ><br>> <br>> --<br>> Ere Maijala<br>> Kansalliskirjasto / The National Library of Finland<br>> <br>> <br>> _______________________________________________<br>> Koha-devel mailing list<br>> Koha-devel@lists.koha-community.org<br>> <mailto:Koha-devel@lists.koha-community.org><br>> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>> <https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel><br>> website : https://www.koha-community.org/<br>> <https://www.koha-community.org/><br>> git : https://git.koha-community.org/ <https://git.koha-community.org/><br>> bugs : https://bugs.koha-community.org/<br>> <https://bugs.koha-community.org/><br>> <br><br>--<br>Ere Maijala<br>Kansalliskirjasto / The National Library of Finland<br><br><br><br><br>------------------------------<br><br>Subject: Digest Footer<br><br>_______________________________________________<br>Koha-devel mailing list<br>Koha-devel@lists.koha-community.org<br>https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel<br>website : https://www.koha-community.org/<br>git : https://git.koha-community.org/<br>bugs : https://bugs.koha-community.org/<br><br><br>------------------------------<br><br>End of Koha-devel Digest, Vol 185, Issue 31<br>*******************************************<br></div><br><br>NOTE: You are receiving this mail because, the Requester/Technician wanted you to get notified on this request creation.<br></body></html>