[Koha-devel] Restructuring C4

[Ian Walls]

If anyone wants to try their hand at breaking this, here's the big cycle I
mentioned (in .dot format)

digraph cluster_9 {
    graph [overlap=false];
    node [shape=doublecircle];
    "C4::Members" -> "C4::Members";
    "C4::Members" -> "C4::Reserves";
    "C4::Items" -> "C4::Items";
    "C4::Items" -> "C4::Acquisition";
    "C4::Letters" -> "C4::Members";
    "C4::Letters" -> "C4::Letters";
    "C4::XSLT" -> "C4::Items";
    "C4::Heading" -> "C4::Search";
    "C4::Acquisition" -> "C4::Suggestions";
    "C4::Suggestions" -> "C4::Letters";
    "C4::Reserves" -> "C4::Heading";
    "C4::Reserves" -> "C4::Reserves";
    "C4::Search" -> "C4::XSLT";
}

I commented out the connection from Items to Acquistions, but the overall
reduction was quite minimal.  Perhaps the connection from Reserves to
Heading could be snipped... not sure why Holds would need to factor in
Authorities...

Big project, for sure.


-Ian

On Wed, Mar 30, 2011 at 1:17 PM, Clay Fouts <cfouts at liblime.com> wrote:

> Hello, Ian.
>
> Disentangling these dependency cycles is essential for Koha's development
> to continue on in a meaningful way. As is, changing one small section of
> code often requires changing a very similar chunk of code in five other
> areas (if you know about all five of those other areas, which you probably
> don't, which means you've introduced action-at-a-distance bugs). The
> approach of separating the DB schema layer out from a model layer is a solid
> first step. Plus, the deep entangling makes reliable testing extremely
> difficult, meaning that the AAD discrepancies are difficult to detect in an
> automated fashion. Separating out the view and controller pieces is also
> essential for testing.
>
> However, this step by itself is only going to marginally optimize for
> execution time. It will help inasmuch as it may reduce the number of modules
> that get loaded for any given operation, and that's a boon. But each module
> only ever gets loaded and its BEGIN block processed once per script
> execution cycle, regardless of how many times it is "used." At the same
> time, many of the efficiency issues are the product of processing the same
> data a thousand times instead of once, because each model-layer subroutine
> makes direct calls to the database instead of normalizing those accesses
> into a schema management layer that can uniformly cache (and un-cache!)
> data.
>
> Ultimately, migrating from CGI to persistence is the big step that needs to
> be taken for improving speed, but that step cannot be taken until the API is
> disentangled, making execution more predictable.
>
> Regards,
> Clay
>
>
> 2011/3/30 Ian Walls <ian.walls at bywatersolutions.com>
>
>> Fellow Developers,
>>
>>
>> Last night, I stayed up late running circ/circulation.pl through NYTProf,
>> to get an idea where we may be able to optimize circulation for speed.
>> After much frustration (darn session IDs...), I was able to get a report.
>> The results were... not exactly what I expected.
>>
>> It seems that much of the time spent running circ/circulation.pl is spent
>> BEGINing the various C4 modules, multiple times.  C4::Items, for example, is
>> BEGuN 14 times in the execution of circ/circulation.pl.  More stats:
>>
>> C4::Accounts - 8 BEGINs
>> C4::Acquisition - 14 BEGINs
>> C4::Auth        - 14 BEGINs
>> C4::Biblio       - 14 BEGINs
>> C4::Branch     - 5 BEGINs
>> C4::Circulation - 22 BEGINs
>> C4::Context     - 18 BEGINs
>> C4::Dates       - 12 BEGINs
>>
>> ... and so on.  Even modules like C4::XSLT, C4::SMS, C4::Suggestions, and
>> C4::Search::PazPar2 are BEGuN several times, without any calls to any of
>> their subroutines.  Some modules, it seems, don't have an END at the end...
>>
>> I was also able to get a Graphviz file out, showing all the package calls
>> (folks may remember my keen interest in such things from KohaCon).  Running
>> it through sccmap to detect cycles, I found a very long one between several
>> of the modules.  About 8 packages long.
>>
>> All this indicate to me that we REALLY need to start thinking about
>> imposing some kind of structure on C4.  I'm thinking something with
>> two-levels:
>>
>> Level 1 :  Calls to the database, other direct interaction with stored
>> data
>> Level 2:   Calls to Level 1 functions to manipulate data
>>
>> all our scripts would then be modified to call only Level 2, which would
>> take care of talking to Level 1.  At the worst, a particular Level 1 package
>> would be BEGuN as many times as there were Level 2 packages BEGuN in the
>> script (and we could certainly do better).  It would eliminate all the
>> cycles, as well.
>>
>> I can provide access to my report for anyone who needs it, or more details
>> about how I obtained it.  Whatever any of you need to keep this conversation
>> going.  This is a big deal, in my opinion, and we need to start detangling
>> our code before too much long, or we're going to get seriously bogged down.
>>
>> Cheers,
>>
>>
>> -Ian
>>
>> --
>> Ian Walls
>> Lead Development Specialist
>> ByWater Solutions
>> Phone # (888) 900-8944
>> http://bywatersolutions.com
>> ian.walls at bywatersolutions.com
>> Twitter: @sekjal
>>
>> _______________________________________________
>> Koha-devel mailing list
>> Koha-devel at lists.koha-community.org
>> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>> website : http://www.koha-community.org/
>> git : http://git.koha-community.org/
>> bugs : http://bugs.koha-community.org/
>>
>
>


-- 
Ian Walls
Lead Development Specialist
ByWater Solutions
Phone # (888) 900-8944
http://bywatersolutions.com
ian.walls at bywatersolutions.com
Twitter: @sekjal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/koha-devel/attachments/20110330/999244e3/attachment.htm>