[Koha-devel] Optimizing Starman startup

[Ere Maijala]

Hi,

I did some work on improving it here:

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=24700

That shaved a good bit of time from it, but it's still a heavy 
operation, and it would make sense to

1.) avoid doing it too often

2.) cache the results and avoid doing it if results are cached

If you could address the first one, that'd go a long way. I'm afraid the 
second one would require changes to the OpenAPI plugin to support caching.

--Ere

dcook at prosentient.com.au kirjoitti 20.4.2021 klo 6.15:
> Hi all,
> 
> Do you despair when you see the following periodically in “top” when a 
> starman worker is recreated ?
> 
>    PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
> 
> 9529 my-koha  20   0  460108 197212  17172 R 100.0  0.4   0:03.41 
> /usr/share/koha/api/v1/app.pl
> 
> Or the following in top when you install koha-common package or restart 
> the koha-common service?
> 
> 11101 1-koha  20   0  447232 193320  16076 R   10.6  0.4   0:09.09 
> /usr/share/koha/api/v1/app.pl
> 
> 11168 1-koha  20   0  447240 193264  16056 R   10.6  0.4   0:08.72 
> /usr/share/koha/api/v1/app.pl
> 
> 11306 2-koha  20   0  447220 193148  16000 R   10.6  0.4   0:08.07 
> /usr/share/koha/api/v1/app.pl
> 
> 11543 2-koha  20   0  447232 193036  15828 R   10.6  0.4   0:07.07 
> /usr/share/koha/api/v1/app.pl
> 
> 11784 3-koha  20   0  441536 189664  16172 R   10.6  0.4   0:06.04 
> /usr/share/koha/api/v1/app.pl
> 
> 11830 3-koha  20   0  439548 187212  15748 R   10.6  0.4   0:05.82 
> /usr/share/koha/api/v1/app.pl
> 
> 11831 4-koha  20   0  438620 186344  15748 R   10.6  0.4   0:05.81 
> /usr/share/koha/api/v1/app.pl
> 
> 11853 4-koha  20   0  437680 185672  16000 R   10.6  0.4   0:05.79 
> /usr/share/koha/api/v1/app.pl
> 
> Well, I still have a lot of investigation left to do, but I notice 1 
> place that a lot of time taken is here (per worker):
> 
>      my $validator = JSON::Validator::OpenAPI::Mojolicious->new;
> 
>      $validator->load_and_validate_schema(
> 
>          $self->home->rel_file("api/v1/swagger/swagger.json"),
> 
>          {
> 
>            allow_invalid_ref  => 1,
> 
>          }
> 
>        );
> 
>      push @{$self->routes->namespaces}, 'Koha::Plugin';
> 
>      my $spec = $validator->schema->data;
> 
>      $self->plugin(
> 
>          'Koha::REST::Plugin::PluginRoutes' => {
> 
>              spec      => $spec,
> 
>              validator => $validator
> 
>          }
> 
>      );
> 
>      $self->plugin(
> 
>          OpenAPI => {
> 
>              spec  => $spec,
> 
>              route => $self->routes->under('/api/v1')->to('Auth#under'),
> 
>              allow_invalid_ref =>
> 
>                1,    # required by our spec because $ref directly under
> 
>                      # Paths-, Parameters-, Definitions- & Info-object
> 
>                      # is not allowed by the OpenAPI specification.
> 
>          }
> 
> );
> 
> Anyone have ideas for improving this? Do we have to validate the schema 
> every time? Can we move the schema validation into a different module 
> and preload it into Starman using the -M flag so that it’s done 1 time 
> per Starman master instance rather than 1 time per Starman worker instance?
> 
> I find “/usr/share/koha/api/v1/app.pl” to be the bane of deployments, as 
> it puts a massive load on a server, when you have multiple Koha 
> instances on the server.
> 
> David Cook
> 
> Software Engineer
> 
> Prosentient Systems
> 
> Suite 7.03
> 
> 6a Glen St
> 
> Milsons Point NSW 2061
> 
> Australia
> 
> Office: 02 9212 0899
> 
> Online: 02 8005 0595
> 
> 
> _______________________________________________
> Koha-devel mailing list
> Koha-devel at lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
> 

-- 
Ere Maijala
Kansalliskirjasto / The National Library of Finland