full text search
Hi, I'm playing with full text searches, that enables you do search very fast in a db, it replaces the where field like "%xx%" that is VERY slow on big tables (non indexed search). It's really a nice feature : immediate result for a search on a 400 000 lines table ! The same search with like %xx% is... more than 30s long (full parsing of the table) ! The problem is that i think this feature is not ANSI-92, so is mysql only compatible. Do you think we should ignore this feature, or become definetly mysql based product ? (unless it's ansi92, in this case, please say it !) documentation : http://www.mysql.com/doc/en/Fulltext_Search.html -- Paul POULAIN Consultant indépendant en logiciels libres responsable francophone de koha (SIBGB libre http://fr.koha.org)
On Tue, 3 Dec 2002, paul POULAIN wrote:
Hi,
I'm playing with full text searches, that enables you do search very fast in a db, it replaces the where field like "%xx%" that is VERY slow on big tables (non indexed search). It's really a nice feature : immediate result for a search on a 400 000 lines table ! The same search with like %xx% is... more than 30s long (full parsing of the table) ! The problem is that i think this feature is not ANSI-92, so is mysql only compatible.
Do you think we should ignore this feature, or become definetly mysql based product ? (unless it's ansi92, in this case, please say it !)
There are three avenues I think we could explore here (maybe more, I've been known to overlook things): 1) ignore this feature and stick to ansi92 so DBI buys us DB independence. In practice, this is not really where we are right now anyway, and we'd lose a nice optimization 2) buy into this feature and become MySQL dependant. Most ILSs are DB dependant (I think), so this is probably not too big a cost. 3) buy into this feature, but start to split our architecture along layers. If we have something like - presentation ------------ business logic -------------- business objects ---------------- data store as our target, then it will become easier to plug in optimizations like this (or maybe using Inline::C to write even faster DB access routines where needed) without mucking up the entire system. Similarily, we could more readily swap out presentation bits to fit on non web/cgi frontends more readily (e.g., resurrecting Steve's PerlTK frontend, or creating koha.el for the truly tech-savvy librarian). Of the above, I'd really like to see us move toward item three. I think the longer term benefits (perhaps in the post 2.0 world) buy us too much to overlook. -pate
documentation : http://www.mysql.com/doc/en/Fulltext_Search.html
-- Paul POULAIN Consultant ind�pendant en logiciels libres responsable francophone de koha (SIBGB libre http://fr.koha.org)
participants (2)
-
Pat Eyler -
paul POULAIN