DBIx::Class vs current Koha's DBI performance
I just figured out today that kohacon10 is in progress, and one of topics seems to be DBIx::Class migration. Let me start by saying that I'm all for it, but probably not for same resons as everybody else. I was trying to profile koha search page. On first run under Devel::NYTProf it showed 2220 invocations of DBI::st::execute. That's more than 2000 calls which wait for database to return answer. This might explain extremly high usage of mysql query cache which I'm seeing in production: http://mjesec.ffzg.hr/munin/koha/koha-mysql_queries.html So, I decided to experiment a bit. and few patches later I was able to reduce number of queries issued on each page load down to 876 If you are interested in what I changed, my code is at: http://git.rot13.org/?p=koha.git;a=shortlog;h=refs/heads/nytprof-cache It's basically using either global hash to cache values per request of simple memoize around function. Which brings me back to DBIx::Class. If we can cache values using dbic, this will be huge WIN for Koha performance as opposed to loss :-) I never used dbic in that contex, but google seems to think it's possible, and I'm willing to take a try at it. -- Dobrica Pavlinusic 2share!2flame dpavlin@rot13.org Unix addict. Internet consultant. http://www.rot13.org/~dpavlin
Hello, Dobrica. Memoize and local caching is handy, and Koha has functions that benefit from its use. However, because Koha's functions' return values usually depend on database contents rather than just the parameters, you'd have to be very clear about unmemoizing functions if the data on which they depends gets changed. This is not always easy to determine since there's not a clear or enforceable domain defined for which parts of the code can read and write directly to various parts of the database contents. Using a cache in such a way as to avoid these staleness problems typically demands consistent use of a limited set of accessor and mutator functions for the database contents, isolating the cache seeding and checking code within those. That way, by defining those functions for, for example, the 'branches' table you can avoid caching the whole branch detail in one cache and then the branch name in another. From within GetBranchName you'd simply call GetBranchDetail (which in turn would call the private _get_branch_row or something), then return only the name field. I imagine you're already aware of much of this, and that this is a proof of concept testbed you're publishing. I only bring it up because it's important that others understand the subtlety of some of the issues that caching can introduce, particularly when the code doesn't naturally support its use. Caching DB contents has an even more pronounced impact when latency is introduced by hosting the database on a different machine than the CGI server. When you have 2k+ queries per script, even a few milliseconds RTT adds up to a rather large number. Consequently, for many of Koha's smaller tables, it often ends up being faster to seed the cache by fetching the entire table on the first call rather than retrieving the rows as-needed. In most Koha setups I've seen, bandwidth and memory are cheaper than latency + MySQL query overhead. Over Unix sockets the 65kB datagram size allows the contents of many tables to be delivered in a single packet. Ethernet is smaller, of course, but the 1500 byte standard MTU is still large enough for several tables to fit in a single packet, plus TCP windowing greatly reduces the impact of subsequent packet latencies. My experience is that the startup overhead introduced by DBIx::Class is not sufficiently offset by its caching features in a CGI environment. Running under Plack or something similar would easily recoup that overhead, of course. Cheers, Clay On Tue, Oct 26, 2010 at 10:17 AM, Dobrica Pavlinusic <dpavlin@rot13.org>wrote:
I just figured out today that kohacon10 is in progress, and one of topics seems to be DBIx::Class migration.
Let me start by saying that I'm all for it, but probably not for same resons as everybody else.
I was trying to profile koha search page. On first run under Devel::NYTProf it showed 2220 invocations of DBI::st::execute.
That's more than 2000 calls which wait for database to return answer. This might explain extremly high usage of mysql query cache which I'm seeing in production:
http://mjesec.ffzg.hr/munin/koha/koha-mysql_queries.html
So, I decided to experiment a bit. and few patches later I was able to reduce number of queries issued on each page load down to 876
If you are interested in what I changed, my code is at:
http://git.rot13.org/?p=koha.git;a=shortlog;h=refs/heads/nytprof-cache
It's basically using either global hash to cache values per request of simple memoize around function.
Which brings me back to DBIx::Class. If we can cache values using dbic, this will be huge WIN for Koha performance as opposed to loss :-)
I never used dbic in that contex, but google seems to think it's possible, and I'm willing to take a try at it.
-- Dobrica Pavlinusic 2share!2flame dpavlin@rot13.org Unix addict. Internet consultant. http://www.rot13.org/~dpavlin _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
My experience is that the startup overhead introduced by DBIx::Class is not sufficiently offset by its caching features in a CGI environment. Running under Plack or something similar would easily recoup that overhead, of course.
Yes, this is the solution. We need in 2010 a persistent environment to execute Koha in: an execution environment, in which they are application-level, session and page objects Memcaching everything isn't the solution. -- Frédéric
How much work would it be to move Koha to the Catalyst framework? Kyle http://www.kylehall.info Mill Run Technology Solutions ( http://millruntech.com ) Crawford County Federated Library System ( http://www.ccfls.org ) Meadville Public Library ( http://www.meadvillelibrary.org ) On Wed, Oct 27, 2010 at 2:27 AM, Frederic Demians <frederic@tamil.fr> wrote:
My experience is that the startup overhead introduced by DBIx::Class is
not sufficiently offset by its caching features in a CGI environment. Running under Plack or something similar would easily recoup that overhead, of course.
Yes, this is the solution. We need in 2010 a persistent environment to execute Koha in: an execution environment, in which they are application-level, session and page objects Memcaching everything isn't the solution. -- Frédéric
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
* Kyle Hall (kyle.m.hall@gmail.com) wrote:
How much work would it be to move Koha to the Catalyst framework? Kyle
Only a complete rewrite. The work we are doing with Plack and Starman will give us a persistent environment, that will, in all likelihood outperform any framework unless it was also running under Plack. Back to DBIx::Class, I have no plans to use it for anything other than schema abstraction for 3.4. April 22 isn't far away. However if someone sends in patches for a module that uses DBIx::Class and we don't take a big performance hit, (and it passes QA and the rest of the tests) it would go in. Small steps, we have a 6 month release, getting Template::Toolkit, Schema abstraction, C4/Search fixes, all the RFC in is going to be enough. Lets work on small manageable improvements. Anyone who wanted to work on finding the circular object references we have, and breaking them would move us a lot closer to being able to run the whole of Koha under a persistent tool. Currently people are testing with the Opac, and circulation, with good results. Chris
http://www.kylehall.info Mill Run Technology Solutions ( http://millruntech.com ) Crawford County Federated Library System ( http://www.ccfls.org ) Meadville Public Library ( http://www.meadvillelibrary.org )
On Wed, Oct 27, 2010 at 2:27 AM, Frederic Demians <frederic@tamil.fr> wrote:
My experience is that the startup overhead introduced by DBIx::Class is not sufficiently offset by its caching features in a CGI environment. Running under Plack or something similar would easily recoup that overhead, of course.
Yes, this is the solution. We need in 2010 a persistent environment to execute Koha in: an execution environment, in which they are application-level, session and page objects Memcaching everything isn't the solution. -- Frederic
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand
Do you have a roadmap/plan for the transition to TT? This process also seems like a good time to start separating out the monolithic "get_template_and_user" function. It would be very handy and allow for more elegant design if we were able to specify the template just prior to rendering it rather than at the very beginning. One method of breaking up the circularity that appears in many areas of Koha is by splitting functions out in a MVC-style arrangement. If you're finding yourself tempted to introduce a circularity in your data schema, it's often the case that that relationship can be expressed through a View or Controller class instead. Having worked with Catalyst, it's a fine framework, but it's cumbersome and not worth porting existing code to. If writing an app from the ground up it would be a consideration. Clay 2010/11/4 Chris Cormack <chrisc@catalyst.net.nz>
* Kyle Hall (kyle.m.hall@gmail.com) wrote:
How much work would it be to move Koha to the Catalyst framework? Kyle
Only a complete rewrite.
The work we are doing with Plack and Starman will give us a persistent environment, that will, in all likelihood outperform any framework unless it was also running under Plack.
Back to DBIx::Class, I have no plans to use it for anything other than schema abstraction for 3.4. April 22 isn't far away. However if someone sends in patches for a module that uses DBIx::Class and we don't take a big performance hit, (and it passes QA and the rest of the tests) it would go in.
Small steps, we have a 6 month release, getting Template::Toolkit, Schema abstraction, C4/Search fixes, all the RFC in is going to be enough. Lets work on small manageable improvements. Anyone who wanted to work on finding the circular object references we have, and breaking them would move us a lot closer to being able to run the whole of Koha under a persistent tool.
Currently people are testing with the Opac, and circulation, with good results.
Chris
http://www.kylehall.info Mill Run Technology Solutions ( http://millruntech.com ) Crawford County Federated Library System ( http://www.ccfls.org ) Meadville Public Library ( http://www.meadvillelibrary.org )
On Wed, Oct 27, 2010 at 2:27 AM, Frederic Demians <frederic@tamil.fr> wrote:
My experience is that the startup overhead introduced by DBIx::Class is not sufficiently offset by its caching features in a CGI environment. Running under Plack or something similar would easily recoup that overhead, of course.
Yes, this is the solution. We need in 2010 a persistent environment to execute Koha in: an execution environment, in which they are application-level, session and page objects Memcaching everything isn't the solution. -- Frederic
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux)
iEYEARECAAYFAkzTCNkACgkQZgbcHEvgMLMERACfaf1UurKAXRc+yZ+t4ie5n2dg FPwAnRfenGuZCDlW8j2SK4Z0Ydy04v2v =odUu -----END PGP SIGNATURE-----
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
* Clay Fouts (cfouts@liblime.com) wrote:
Do you have a roadmap/plan for the transition to TT? This process also seems like a good time to start separating out the monolithic "get_template_and_user" function. It would be very handy and allow for more elegant design if we were able to specify the template just prior to rendering it rather than at the very beginning.
Hi Clay We have a summer of tech student who has lots of experience with machine translation. The roadmap for 3.4 TT work looks like this 1/ Exact translation of the templates from HTML::Template::Pro to TT, scripted and repeatable. 2/ Lots and lots of testing 3/ Call a template freeze 4/ Run the conversion a final time. 5/ Start accepting template patches again, but only TT ones. 6/ Maybe start changing things. But the main aim is to get TT in there, so that we can start changing things like get_template_and_user, caching rendered fragments etc.
One method of breaking up the circularity that appears in many areas of Koha is by splitting functions out in a MVC-style arrangement. If you're finding yourself tempted to introduce a circularity in your data schema, it's often the case that that relationship can be expressed through a View or Controller class instead.
Yep, and if you come across one, patches to fix them would be gratefully received, thats the kind of refactoring we should encourage.
Having worked with Catalyst, it's a fine framework, but it's cumbersome and not worth porting existing code to. If writing an app from the ground up it would be a consideration.
I agree. Chris
Clay
2010/11/4 Chris Cormack <chrisc@catalyst.net.nz>
* Kyle Hall (kyle.m.hall@gmail.com) wrote: > How much work would it be to move Koha to the Catalyst framework? > Kyle
Only a complete rewrite.
The work we are doing with Plack and Starman will give us a persistent environment, that will, in all likelihood outperform any framework unless it was also running under Plack.
Back to DBIx::Class, I have no plans to use it for anything other than schema abstraction for 3.4. April 22 isn't far away. However if someone sends in patches for a module that uses DBIx::Class and we don't take a big performance hit, (and it passes QA and the rest of the tests) it would go in.
Small steps, we have a 6 month release, getting Template::Toolkit, Schema abstraction, C4/Search fixes, all the RFC in is going to be enough. Lets work on small manageable improvements. Anyone who wanted to work on finding the circular object references we have, and breaking them would move us a lot closer to being able to run the whole of Koha under a persistent tool.
Currently people are testing with the Opac, and circulation, with good results.
Chris > http://www.kylehall.info > Mill Run Technology Solutions ( http://millruntech.com ) > Crawford County Federated Library System ( http://www.ccfls.org ) > Meadville Public Library ( http://www.meadvillelibrary.org ) > > On Wed, Oct 27, 2010 at 2:27 AM, Frederic Demians <frederic@tamil.fr> > wrote: > > My experience is that the startup overhead introduced by DBIx::Class > is not sufficiently offset by its caching features in a CGI > environment. Running under Plack or something similar would easily > recoup that overhead, of course. > > Yes, this is the solution. We need in 2010 a persistent environment to > execute Koha in: an execution environment, in which they are > application-level, session and page objects Memcaching everything isn't > the solution. > -- > Frederic > > _______________________________________________ > Koha-devel mailing list > Koha-devel@lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> _______________________________________________ > Koha-devel mailing list > Koha-devel@lists.koha-community.org > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/
-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux)
iEYEARECAAYFAkzTCNkACgkQZgbcHEvgMLMERACfaf1UurKAXRc+yZ+t4ie5n2dg FPwAnRfenGuZCDlW8j2SK4Z0Ydy04v2v =odUu -----END PGP SIGNATURE-----
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand
Small steps, we have a 6 month release, getting Template::Toolkit, Schema abstraction, C4/Search fixes, all the RFC in is going to be enough. Lets work on small manageable improvements. Anyone who wanted to work on finding the circular object references we have, and breaking them would move us a lot closer to being able to run the whole of Koha under a persistent tool.
How do you identify exactly this circular object references? Other question: do you plan to make Moose a Koha requirement and recommendation? -- Frédéric
2010/11/8 Frédéric Demians <frederic@tamil.fr>:
Small steps, we have a 6 month release, getting Template::Toolkit, Schema abstraction, C4/Search fixes, all the RFC in is going to be enough. Lets work on small manageable improvements. Anyone who wanted to work on finding the circular object references we have, and breaking them would move us a lot closer to being able to run the whole of Koha under a persistent tool.
How do you identify exactly this circular object references?
They are quite hard to find, Test::Memory::Cycle and Devel::Cycle are two ways to find them, but you have to have some suspicion where to look.
From the incredibly smart Andy Lester
" Perl's garbage collection has one big problem: Circular references can't get cleaned up. A circular reference can be as simple as two reference that refer to each other: my $mom = { name => "Marilyn Lester", }; my $me = { name => "Andy Lester", mother => $mom, }; $mom->{son} = $me; "
Other question: do you plan to make Moose a Koha requirement and recommendation?
Not for 3.4. I think the above list plus all the rfcs to get done in 6 months is plenty ambitious enough. And until we can run Koha in a persistent manner, adding Moose would just kill us. Chris
They are quite hard to find, Test::Memory::Cycle and Devel::Cycle are two ways to find them, but you have to have some suspicion where to look.
Thanks. I will explore those modules. Could you point in HEAD an existing and known circular reference I could test with this technique?
Other question: do you plan to make Moose a Koha requirement and recommendation?
Not for 3.4. I think the above list plus all the rfcs to get done in 6 months is plenty ambitious enough.
Yes, ambitious and awesome.
And until we can run Koha in a manner, adding Moose would just kill us.
And using Moose outside Koha in /misc/* maintenance scripts? It'll be a Moose integration first step.
2010/11/8 Frédéric Demians <frederic@tamil.fr>:
They are quite hard to find, Test::Memory::Cycle and Devel::Cycle are two ways to find them, but you have to have some suspicion where to look.
Thanks. I will explore those modules. Could you point in HEAD an existing and known circular reference I could test with this technique?
Not off the top of my head, I suspect we might have issues in the Search results, that seems to leak the most. Thats the problem with them, its hard to track them down. I'm wondering if there is a biblio that references an item which references the biblio, or something similair to that.
Other question: do you plan to make Moose a Koha requirement and recommendation?
Not for 3.4. I think the above list plus all the rfcs to get done in 6 months is plenty ambitious enough.
Yes, ambitious and awesome.
And until we can run Koha in a manner, adding Moose would just kill us.
And using Moose outside Koha in /misc/* maintenance scripts? It'll be a Moose integration first step.
I wouldn't reject that out of hand :) It would add another dependency, but it might be worth it. Do you have a sample script? Chris
On Tue, Oct 26, 2010 at 02:24:24PM -0700, Fouts, Clay wrote:
I imagine you're already aware of much of this, and that this is a proof of concept testbed you're publishing. I only bring it up because it's important that others understand the subtlety of some of the issues that caching can introduce, particularly when the code doesn't naturally support its use.
I am aware of this, so I implementd only caching of values which aren't dependent on other data or user info. However, data dependent on user info could be cached in user session (like lists and similar values). You are correct that invalidation of caches is hard problem, but I assume that caching over DBIx::Class would take care of invalidation, but I might be wrong.
My experience is that the startup overhead introduced by DBIx::Class is not sufficiently offset by its caching features in a CGI environment. Running under Plack or something similar would easily recoup that overhead, of course.
I did try to run it under Plack using following simple plack app which supports just opac-search: http://git.rot13.org/?p=koha.git;a=blob;f=app.psgi;h=7ff4c251236fb37a9d88bf8... (sorry about long URL). However, it's not would problems: CGI::Compile doesn't like some parts of Koha code and can't compile it (but that would be rather easy to fix), however to make it real solution we should first move Koha to use strict everywhere. Another intersting bit is change at: http://git.rot13.org/?p=koha.git;a=commit;h=8b27e87c24b8efa5f4ffc9b21b8d1a97... which takes marc records directly from marc instead of XML. This bypasses XML parsing and transformation and reduces number of statements significantly: old new stmts 1112898 786000 subs 295048 228382 (and it does show in real-life performance also, not just in profiling ;-)
From my expirience so far, there are quite a few more low hanging fruits which we can pick in Koha code using caching. However, I would love to know if this moment is the right one to tackle this, or should I wait for DBIx::Class changes to land into code.
-- Dobrica Pavlinusic 2share!2flame dpavlin@rot13.org Unix addict. Internet consultant. http://www.rot13.org/~dpavlin
On Wed, Oct 27, 2010 at 4:15 AM, Dobrica Pavlinusic <dpavlin@rot13.org>wrote:
You are correct that invalidation of caches is hard problem, but I assume that caching over DBIx::Class would take care of invalidation, but I might be wrong.
DBIx::Class handles cache invalidation very well as long as it is used uniformly as the only means of accessing the database. My concern is more in regards to the challenges with getting all of Koha's use of a given set of tables to go through the dbic interface. Another intersting bit is change at:
http://git.rot13.org/?p=koha.git;a=commit;h=8b27e87c24b8efa5f4ffc9b21b8d1a97...
which takes marc records directly from marc instead of XML. This bypasses XML parsing and transformation and reduces number of statements significantly:
While this is much faster, the ISO format has hard-coded limitations that Koha needs to surpass.
From my expirience so far, there are quite a few more low hanging fruits which we can pick in Koha code using caching. However, I would love to know if this moment is the right one to tackle this, or should I wait for DBIx::Class changes to land into code.
Getting Koha to run persistently (and to do so safely!) seems like a higher priority in my estimation than the conversion to DBIC, if for no other reason than pervasive use of DBIC is going to seriously impede CGI performance. In mean time, I find that applying small, judicious use of local caching often gives a high return on very little time and additional code while not interfering with the API. Clay
On Wed, Oct 27, 2010 at 07:51:46AM -0700, Fouts, Clay wrote:
Getting Koha to run persistently (and to do so safely!) seems like a higher priority in my estimation than the conversion to DBIC, if for no other reason than pervasive use of DBIC is going to seriously impede CGI performance. In mean time, I find that applying small, judicious use of local caching often gives a high return on very little time and additional code while not interfering with the API.
I have been playing with Plack recently. It makes use of PSGI, an interface such as CGI or FastCGI or mod_perl that sits between an application and the webserver. The benefit of PSGI is that once an application targets it, it opens up a lot of possibilities with regard to webservers and persistence. http://plackperl.org One of the most informative talks listed is: Plack: Superglue for Perl web frameworks and servers http://www.slideshare.net/miyagawa/plack-at-oscon-2010 As a quick test, there are some Plack modules that allow you to test out CGI scripts in a Plack environment: http://search.cpan.org/dist/Plack/lib/Plack/App/WrapCGI.pm http://search.cpan.org/dist/Plack/lib/Plack/App/CGIBin.pm -kolibrie
Well there is no aim for widespread DBIx::Class usage in 3.4 we do only have 6 months. All we are proposing to do is make use of DBIx::Class::Schema On 28 Oct 2010 03:52, "Fouts, Clay" <cfouts@liblime.com> wrote: On Wed, Oct 27, 2010 at 4:15 AM, Dobrica Pavlinusic <dpavlin@rot13.org> wrote:
You are correct t...
DBIx::Class handles cache invalidation very well as long as it is used uniformly as the only means of accessing the database. My concern is more in regards to the challenges with getting all of Koha's use of a given set of tables to go through the dbic interface.
Another intersting bit is change at:
http://git.rot13.org/?p=koha.git;a=commit;h=8b27e87c2... While this is much faster, the ISO format has hard-coded limitations that Koha needs to surpass.
From my expirience so far, there are quite a few more low hanging fruits which we can pick in... Getting Koha to run persistently (and to do so safely!) seems like a higher priority in my estimation than the conversion to DBIC, if for no other reason than pervasive use of DBIC is going to seriously impede CGI performance. In mean time, I find that applying small, judicious use of local caching often gives a high return on very little time and additional code while not interfering with the API.
Clay _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
Gah bumped by people on the bus, what I was trying to say was that for 3.4 we plan to use the schema tools to move us one step away from mysql dependency. We are talking about plack at the dev conf tomorrow, please try and join us on irc. Chris On 28 Oct 2010 03:52, "Fouts, Clay" <cfouts@liblime.com> wrote: On Wed, Oct 27, 2010 at 4:15 AM, Dobrica Pavlinusic <dpavlin@rot13.org> wrote:
You are correct t...
DBIx::Class handles cache invalidation very well as long as it is used uniformly as the only means of accessing the database. My concern is more in regards to the challenges with getting all of Koha's use of a given set of tables to go through the dbic interface.
Another intersting bit is change at:
http://git.rot13.org/?p=koha.git;a=commit;h=8b27e87c2... While this is much faster, the ISO format has hard-coded limitations that Koha needs to surpass.
From my expirience so far, there are quite a few more low hanging fruits which we can pick in... Getting Koha to run persistently (and to do so safely!) seems like a higher priority in my estimation than the conversion to DBIC, if for no other reason than pervasive use of DBIC is going to seriously impede CGI performance. In mean time, I find that applying small, judicious use of local caching often gives a high return on very little time and additional code while not interfering with the API.
Clay _______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
Speaking of DBIC, has anyone looked into other ORM layers as potential candidates for integration? The only other one I've looked at and experimented with is Rose::DB::Object. It's considerably faster than DBIC, both during startup and ongoing execution. The API makes more sense to me, too, but that's a personal preference. However, it lacks some of DBIC's schema management utilities and the ResultSet-style of subquerying (which also has the advantage of not having deal with a ResultSet in the first place). Thoughts? Clay On Wed, Oct 27, 2010 at 12:06 PM, Chris Cormack <chris@bigballofwax.co.nz>wrote:
Gah bumped by people on the bus, what I was trying to say was that for 3.4 we plan to use the schema tools to move us one step away from mysql dependency.
We are talking about plack at the dev conf tomorrow, please try and join us on irc.
Chris
On 28 Oct 2010 03:52, "Fouts, Clay" <cfouts@liblime.com> wrote:
On Wed, Oct 27, 2010 at 4:15 AM, Dobrica Pavlinusic <dpavlin@rot13.org> wrote:
You are correct t...
DBIx::Class handles cache invalidation very well as long as it is used uniformly as the only means of accessing the database. My concern is more in regards to the challenges with getting all of Koha's use of a given set of tables to go through the dbic interface.
Another intersting bit is change at:
While this is much faster, the ISO format has hard-coded limitations that Koha needs to surpass.
From my expirience so far, there are quite a few more low hanging fruits which we can pick in...
Getting Koha to run persistently (and to do so safely!) seems like a higher priority in my estimation than the conversion to DBIC, if for no other reason than pervasive use of DBIC is going to seriously impede CGI performance. In mean time, I find that applying small, judicious use of local caching often gives a high return on very little time and additional code while not interfering with the API.
Clay
_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
participants (9)
-
Chris Cormack -
Chris Cormack -
Clay Fouts -
Dobrica Pavlinusic -
Fouts, Clay
-
Frederic Demians -
Frédéric Demians
-
Kyle Hall -
Nathan Gray