On Mon, Feb 4, 2013 at 12:55 PM, Frédéric Demians <frederic@tamil.fr> wrote:

A better approach would be to do as for GoogleBooks covers: after HTML page loading, sending asynchronously requests to Amazon Web Service (AWS) in order to get ASIN from ISBN, then book covers, and then complete HTML DOM with appropriate <img> tags.

Would this approach not also solve the problem of missing covers for movies, many of which seem to lack ISBNs on Amazon? Cab Vinton Sanbornton PL

A better approach would be to do as for GoogleBooks covers: after HTML page loading, sending asynchronously requests to Amazon Web Service (AWS) in order to get ASIN from ISBN, then book covers, and then complete HTML DOM with appropriate <img> tags.

This discussion assumes that AWS is still usable by libraries. My understanding of the latest terms of AWS is that your application must have as its primary purpose the sale of material from Amazon. That's one of the reasons other Amazon API features were removed from Koha. -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org

On Feb 5, 2013 7:14 AM, "Frédéric Demians" <frederic@tamil.fr> wrote:

...

This discussion assumes that AWS is still usable by libraries. My understanding of the latest terms of AWS is that your application must have as its primary purpose the sale of material from Amazon. That's one of the reasons other Amazon API features were removed from Koha.

I agree. This is a question for a lawyer. If your assertion is true, Amazon book cover should be wiped out from Koha. With a decreasing number of match, Amazon book covers, as they are currently implemented, are more and more irrelevant.

Currently implemented we are within the terms of use, and for some collections Amazon provides ok coverage. It's a syspref, people have to turn them on, ripping it out when people are using them sounds like a bad idea to me.

Isn't Open Library the solution?

It's a solution, yes, and people can turn that on, or syndetics, or ltfl, or google or local covers, etc. The enhancement I want to see is allowing a mix of sources. It's on bugzilla but I can't remember the number. Chris

_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/

The enhancement I want to see is allowing a mix of sources. It's on bugzilla but I can't remember the number. http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7187

My thoughts on designing such a system: - On a Koha community server (or cluster of servers, or Koha library server), a repository of cached book covers URLs is maintained. It associates a book/resource cover per covers provider with resource ID (ISBN/EAN). Covers providers could be: Amazon (aws), Google Books (gb), Open Library (ol), Koha (kws), etc. - On Koha pages, a JavaScript function grabs resource ID and send a request to the central server (kws) asking for ID cover URLS: http://kws.koha-community.org/cover/src=aws,gb,kws,ol&\ isbn=a,b,c,d&icallback=myjscallback src: ordered list of targeted cover providers. isbn: list of ISBN (EAD?) callback: JS function to call back with ID URL - kws server receive the request. Find directly URLs in its cache, send them immediately to JS call back function. Not already cached URL are searched in parallel, and send back if found. kws returns a JSON that looks like that: { "0596000278":{ "id":"0596000278", "cover":"http://books.google.com/books?id=ezqe....", }, "ISBN0765304368":{ "id":"ISBN0765304368", "cover":"http://amazon.fr/thumnail/xx.jpg", }, } A mechanism marks as failed, URLs not found from a provider at a time, in order not to fetch them again and again. But periodically, a batch task can retry fetching those URLs. - On Koha pages, the JS call back function receive the result, get cover for each ID, and modify HTML DOM accordingly.

I think caching is something that Amazon (and probably others) rules out explicitly.

This is correct. -- Owen -- Web Developer Athens County Public Libraries http://www.myacpl.org

I think caching is something that Amazon (and probably others) rules out explicitly. The point of that service is to gain hits, sales and user info, not providing a nice source to harvest for your own server.

It's not about caching thumbnails themselves, but URLs to them. There is no interest in caching images when Amazon or Google have CDN designed to deliver their content in a breeze. So the number of hits to the images stay the same for the providers, originating from browsers, whereas the number of requests to AWS or Google Book services decrease, since their result is cached. And this cache may be cleared periodically, or checked since providers can decide to move their thumbnails URLs. I described a solution for bug 7187: http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=7187 And indeed, I just reinvented the wheel: http://wiki.koha-community.org/wiki/Covers_from_multiple_sources_RFC

* Frédéric Demians (frederic@tamil.fr) wrote:

I've played with this idea of a cover images centralized cache of URLs. I have a mock-up. Anybody interested in testing, commenting?

Client side (Koha), two syspref enable the service. URL are fetched from the cache, via JSONP, just Google Books:

http://git.tamil.fr/?p=koha;a=commitdiff;h=e8e223437a5f61c8320cba1b9563a2eb8...

Server side, it's a node.js + redis small application:

http://git.tamil.fr/?p=coce;a=commitdiff;h=5e8021a95df07c7a5b14692955ac4c23c...

It works now with Amazon and Google Books, but can be extended easily to support other providers. It could work with other ILS than Koha. I can show how it works on a Koha devel instance.

I was thinking we could do it all client side with js and html5 (and localstorage or indexdb) You'd define the order in system preferences, local, google, amazon, ltfl etc. Then the js would try them in that order and cache the resulting url (or the fact it couldn't find one) in localstorage or indexdb, which persists even when the browser is closed. That way it wouldnt need to be another server/service, since you have the js already, it shouldn't be too hard to convert it to an html5 client side app. Thoughts? Chris

_______________________________________________ Koha-devel mailing list Koha-devel@lists.koha-community.org http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/

-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand

I was thinking we could do it all client side with js and html5 (and localstorage or indexdb) You'd define the order in system preferences, local, google, amazon, ltfl etc. Then the js would try them in that order and cache the resulting url (or the fact it couldn't find one) in localstorage or indexdb, which persists even when the browser is closed. That way it wouldnt need to be another server/service, since you have the js already, it shouldn't be too hard to convert it to an html5 client side app. Thoughts?

I understand and agree that having cover images URL as a service may have disadvantages. That's arguable. Thoughts: (1) Client-side storage may be insufficient. localstorage (and others) have size limits (~5MB): http://dev-test.nemikor.com/web-storage/support-test/ If URL have an average of 60 characters (120 bytes in UTF-16), for a 100,000 biblio records catalog (with ID), it means data URLs alone, without ID, may require as much as 12,000,000 bytes. Then, I'm not sure whereas localstorage has been designed to fetch quickly a value (URL) from a key (ISBN/EAN). On the contrary, Redis as a key/value store is very efficient. I don't know for IndexedDb. (2) There may be a security issue with AWS request done client-side. Request must be send to AWS with accessKeyId, secretAccessKey, and associateTag. It a security risk to send them to the client browser. (3) Technically, the AWS queries are sent by the node.js script using this library: https://github.com/livelycode/aws-lib As far as I know, there isn't an equivalent client-side. This library does a lot of stuff. AWS request and reply are complex: request has to be signed/encrypted, XML parsing is required, etc. So converting the node.js app to a client side app shouldn't be that easy. (4) A centralized cover URLs cache has several advantages: On a hosting plateform, as any Koha vendor has, a unique cover URL cache is easy to administer, and tweak. Centralized cached will grow quickly to contain 'all' useful URL. It could also be used outside Koha.

On Mon, Feb 4, 2013 at 1:14 PM, Frédéric Demians <frederic@tamil.fr> wrote:

Isn't Open Library the solution?

Yes, but most libraries aren't picking Open Library cause the images are so tini tiny :) So hopefully that is resolved in 3.12 or in a later 3.10 release so that I can start getting everyone to switch!! Nicole

On 2013-02-5, at 2:39 PM, Nicole Engard wrote:

On Mon, Feb 4, 2013 at 8:35 PM, Nicole Engard <nengard@gmail.com> wrote:

On Mon, Feb 4, 2013 at 1:14 PM, Frédéric Demians <frederic@tamil.fr> wrote:

Isn't Open Library the solution?

Yes, but most libraries aren't picking Open Library cause the images are so tini tiny :)

hey Nicole did you know its possible to swap the 'S' in the cover image url, for an 'M' or 'L', to make the images not so tiny :p like so… http://covers.openlibrary.org/b/id/6121771-S.jpg http://covers.openlibrary.org/b/id/6121771-M.jpg http://covers.openlibrary.org/b/id/6121771-L.jpg

So hopefully that is resolved in 3.12 or in a later 3.10 release so that I can start getting everyone to switch!!

Nicole

On 2013-02-17, at 2:52 AM, Nicole Engard wrote:

On Fri, Feb 15, 2013 at 6:50 AM, Mason James <mtj@kohaaloha.com> wrote:

On 2013-02-5, at 2:39 PM, Nicole Engard wrote:

...

On Mon, Feb 4, 2013 at 8:35 PM, Nicole Engard <nengard@gmail.com> wrote:

On Mon, Feb 4, 2013 at 1:14 PM, Frédéric Demians <frederic@tamil.fr> wrote:

Isn't Open Library the solution?

Yes, but most libraries aren't picking Open Library cause the images are so tini tiny :)

hey Nicole

did you know its possible to swap the 'S' in the cover image url, for an 'M' or 'L', to make the images not so tiny :p

like so…

http://covers.openlibrary.org/b/id/6121771-S.jpg http://covers.openlibrary.org/b/id/6121771-M.jpg http://covers.openlibrary.org/b/id/6121771-L.jpg

Yes, but you can't do that as a user - that needs to be done in the code.

patch sent :p http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=9660

On 2013-02-21, at 9:49 AM, Joe Atzberger wrote:

On Mon, Feb 4, 2013 at 8:35 PM, Nicole Engard <nengard@gmail.com> wrote:

On Mon, Feb 4, 2013 at 1:14 PM, Frédéric Demians <frederic@tamil.fr> wrote:

Isn't Open Library the solution?

Yes, but most libraries aren't picking Open Library cause the images are so tini tiny :) So hopefully that is resolved in 3.12 or in a later 3.10 release so that I can start getting everyone to switch!!

I regret to report that as of last week at #code4lib conference in Chicago, the status of Open Library as a jacket image source is rather questionable. It has not been loading updates since last summer and current development is in unrelated areas areas of interest (e.g. ebooks). It is unclear when this production/maintenance will be picked up and by whom.

--Joe

hey Joe, long time no chat :) wow, thats a bummer - OpenLibrary was looking like the best option for everyone :/ Joe, did anyone at #code4lib agree on a better option for the distant future, library-thing perhaps?

On 2013-02-21, at 3:50 PM, Mason James wrote:

On 2013-02-21, at 9:49 AM, Joe Atzberger wrote:

...

On Mon, Feb 4, 2013 at 8:35 PM, Nicole Engard <nengard@gmail.com> wrote:

On Mon, Feb 4, 2013 at 1:14 PM, Frédéric Demians <frederic@tamil.fr> wrote:

Isn't Open Library the solution?

Yes, but most libraries aren't picking Open Library cause the images are so tini tiny :) So hopefully that is resolved in 3.12 or in a later 3.10 release so that I can start getting everyone to switch!!

I regret to report that as of last week at #code4lib conference in Chicago, the status of Open Library as a jacket image source is rather questionable. It has not been loading updates since last summer and current development is in unrelated areas areas of interest (e.g. ebooks). It is unclear when this production/maintenance will be picked up and by whom.

--Joe

hey Joe, long time no chat :)

wow, thats a bummer - OpenLibrary was looking like the best option for everyone :/

Joe, did anyone at #code4lib agree on a better option for the distant future, library-thing perhaps?

although, i should also mention that while OpenLibrary may not be actively adding new cover images ...there are still 13 million? *existing* cover-images at OL, that will hopefully remain available for us to use - until we sort a work-around :)

On Wed, Feb 20, 2013 at 10:03 PM, Mason James <mason.loves.sushi@gmail.com>wrote:

although, i should also mention that while OpenLibrary may not be actively adding new cover images

...there are still 13 million? *existing* cover-images at OL, that will hopefully remain available for us to use - until we sort a work-around :)

Also, users can add cover images, so we don't need them to maintain it, I know that a few of our libraries are adding images all the time. Nicole