[Bug 28448] New: Record with too many items is not shown in results
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 Bug ID: 28448 Summary: Record with too many items is not shown in results Change sponsored?: --- Product: Koha Version: 20.11 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 - low Component: Searching - Zebra Assignee: koha-bugs@lists.koha-community.org Reporter: b.pastern4k@gmail.com In our library there are some records (newspapers) that have lots of items, some have more than a thousand items. It seems that if there's more than a thousand items per one location then this record is not shown in any searches, only way to access it is to know biblionumber and insert it into URL. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #1 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Just a wild guess, but could you try setting a lower value for maxItemsInSearchResults to see if this changes things? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- Severity|enhancement |normal -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #2 from Bohdan <b.pastern4k@gmail.com> --- Changed it to 15, nothing happened. If there is more than 1 result found, it skips missing record. Instead of 10 or 20 records per page I see 9 or 19 and the enumeration goes from 1 to 3 for example and continues as usual. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 David Cook <dcook@prosentient.com.au> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dcook@prosentient.com.au --- Comment #3 from David Cook <dcook@prosentient.com.au> --- Are you sure you're not accidentally using the system preference OpacHiddenItems? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #4 from Bohdan <b.pastern4k@gmail.com> --- This happens both in staff interface and in OPAC. OpacHiddenItems preference is not used in that library. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #5 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- Bohdan, you said you were using Elasticsearch in another thread, is that correct? It does sound like some sort of size limit, but I am not an Elastic expert myself. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #6 from Bohdan <b.pastern4k@gmail.com> --- That is a different library. This one uses Zebra. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 Katrin Fischer <katrin.fischer@bsz-bw.de> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |fridolin.somers@biblibre.co | |m, | |kyle@bywatersolutions.com --- Comment #7 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- In that case I'll add some of our Zebra experts that have probably dealt with big records like that :) -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 George Williams (NEKLS) <george@nekls.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |george@nekls.org --- Comment #8 from George Williams (NEKLS) <george@nekls.org> --- My experience is that this happens when the Marc record exceeds 9999k in size. Our solution was to separate the record into multiple records and put all of the items from one year on one record, all of the items from the next year on the second record and so on . . . It wasn't a problem with Koha. The problem was that we were exceeding the maximum size of a Marc record. George -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #9 from David Cook <dcook@prosentient.com.au> --- (In reply to George Williams (NEKLS) from comment #8)
My experience is that this happens when the Marc record exceeds 9999k in size. Our solution was to separate the record into multiple records and put all of the items from one year on one record, all of the items from the next year on the second record and so on . . .
It wasn't a problem with Koha. The problem was that we were exceeding the maximum size of a Marc record.
George
That's interesting. I thought that we'd overcome that limitation by switching to MARCXML with Zebra but perhaps we didn't... -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #10 from David Cook <dcook@prosentient.com.au> --- Bohdan: With your 20.11 installation, is it a new installation or an upgraded installation that you've had for a long time? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #11 from Bohdan <b.pastern4k@gmail.com> --- This Koha is an upgraded one. I installed version 19.11. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #12 from George Williams (NEKLS) <george@nekls.org> --- (In reply to David Cook from comment #9)
(In reply to George Williams (NEKLS) from comment #8)
My experience is that this happens when the Marc record exceeds 9999k in size. Our solution was to separate the record into multiple records and put all of the items from one year on one record, all of the items from the next year on the second record and so on . . .
It wasn't a problem with Koha. The problem was that we were exceeding the maximum size of a Marc record.
George
That's interesting. I thought that we'd overcome that limitation by switching to MARCXML with Zebra but perhaps we didn't...
It's possible that the 9999k limit has been fixed and I just never realized it because we trained everyone to limit the size of the records that might hit that limit. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #13 from Bohdan <b.pastern4k@gmail.com> --- Created attachment 127084 --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=127084&action=edit Search results image -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #14 from Bohdan <b.pastern4k@gmail.com> --- In the attachment you can see that for some reason the result with number 1 is missing. This still persists on 21.05. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #15 from Fridolin Somers <fridolin.somers@biblibre.com> --- See Bug 10482 We have this code for some libraries with records having more than 1000 items I don't know which limit is in Zebra against this. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #16 from Katrin Fischer <katrin.fischer@bsz-bw.de> --- There is a parameter that limits the size of record Zebra will index. As he problem is that the record is not searchable, could this be the issue here? -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #17 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #16)
There is a parameter that limits the size of record Zebra will index. As he problem is that the record is not searchable, could this be the issue here?
Do you mean the "memMax" parameter? You're probably right. According to https://software.indexdata.com/zebra/doc/idzebra.pdf, "The indexed documents are parsed into a standard XML DOM tree, which restricts record size according to availability of memory." Last year, I discovered a size issue with the memMax parameter and Indexdata fixed it: https://github.com/indexdata/idzebra/issues/34
From Zebra 2.2.4 onwards you should be able to specify as much memory as you want for "memMax": https://github.com/indexdata/idzebra/blob/master/NEWS
Prior to Zebra 2.2.4, the max was 2047M. I have a library with a large collection of large records and so far 2047M has been fine for them. -- You are receiving this mail because: You are the assignee for the bug. You are watching all bug changes.
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28448 --- Comment #18 from David Cook <dcook@prosentient.com.au> --- (In reply to Katrin Fischer from comment #16)
There is a parameter that limits the size of record Zebra will index. As he problem is that the record is not searchable, could this be the issue here?
Or maybe you meant zebra_max_record_size from bug 18909... Interestingly I've never had to modify that parameter. -- You are receiving this mail because: You are watching all bug changes. You are the assignee for the bug.
participants (1)
-
bugzilla-daemon＠bugs.koha-community.org