[Koha-devel] using CHI directly to use all of it's advantages

[Jared Camins-Esakov]

Dobrica,

> As some of you allready know, Jonathan Swartz had presentation about CHI:
Universal caching for Perl
> at YAPC::NA. Video recording is available at:
>
>
http://ics.webcast.uwex.edu/mediasite/Viewer/?peid=b503ecb42c18452a9f4a95b7372648711d
>
> It got me thinking about our new Koha::Cache layer. For a start, I would
> love to use CHI directly and make it required depenency for Koha since
> it has following features which we already use:
> - memcached back-end - making our own memcache implementation obsolete
> - fastmmap back-end - ditto

We don't actually have a fastmmap implementation yet. That is on my to-do
list, though not very high.

> - DBI cache - making mysql query cache problem solvable
>  way than DBD::Goper suggestion from
>  http://wiki.koha-community.org/wiki/Performance#MySQL_tuning
> - memoize
> - very nice syntax:
>
> my $customer2 = $cache->compute($name2, "10 minutes", sub {
>        get_customer_from_db($name2)
> });
>
> Which I would prefer to our existing Koha::Cache API. It would also save
> us quite a bit of code.

If we were to make CHI a core dependency, I see no reason not to have a
Koha::Cache wrapper subclassing CHI so that if we had to globally adjust
cache settings we could do so. We would use the exact same interface as
that exposed by CHI but would be able to transparently handle Koha-specific
creation options (e.g. any sysprefs for turning the caching on or off, or
selecting a non-default backend).

> CHI has powerfull statistics which basically obsolete my idea of writing
> statistics gathering code through Koha.
>
> I would love to have this in next release, since this is more-or-less my
> last major gripe with current Koha code (short-term goal[1]), so if I
> would love to hear feedback.
>
> 1: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7177
>
> My more long-term goal is to know more about Koha's data-access
> patterns (you are again advised to examine performance wiki page if you
> haven't up to this point).
>
> In most simple sense, only think which is big unknown is wheater we can
> find sane key namespace schema which will allow us to good-enough cache
> invalidation scheme. I strongly think that time-outs are useful just for
> OPAC, not for intranet use!

This is why I do not think memoize is a good idea. Useful cache
invalidation with memoize -- at least as I read the code implementing the
memoize function -- is impossible. Using compute around a method call would
be a much better option. If you want to make it appear seamless without
modifying the underlying function, do the computation in a private method
and surround a call to that with commute in a routine that does nothing
other than run the compute (or invalidate the cache if that is what is
desired).

> It seems to me that CHI redis back-end (which can introspect keyspace
> with keys *) and my "weekly push master in production because of plack"
> approach might bring us to better understanding how our Model layer
> might look like, allmost like reverse engeeniring existing code access
> patterns which has proven to bring a lot of low-hanging fruits so far.
>
> Only disadvantage I can think of is that CHI isn't packaged in stable
> Debian but it is in wheezy so Robin would not need to package it
> forever, just for little while :-)

CHI is very easy to package. I already created packages for myself.

+1 from me on making CHI a core dependency.
-1 on decentralizing cache access with memoize and direct 'use CHI'
throughout the Koha/C4 namespace
+1 on making Koha::Cache a transparent wrapper around CHI and using compute
instead of the existing get/set whenever possible

In summary, I almost entirely agree with you, and would sign off on a patch
changing our caching to CHI only provided it kept the code that interfaces
directly with CHI neatly cordoned off from the rest of the system.

Regards,
Jared

P.S. If we need to replace the SQL query cache with a local cache, doesn't
this mean we're doing it wrong?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/koha-devel/attachments/20120614/f93e38cb/attachment-0001.htm>