Thursday, August 07, 2008

A million free covers from LibraryThing

A few days ago, just before hitting thirty million books, we hit one million user-uploaded covers. So, we've decided to give them away—to libraries, to bookstores, to everyone.

The basics. The process, patterned after the Amazon.com cover service, is simplicity itself:
  1. Take an ISBN, like 0545010225
  2. Put your Developer Key and the ISBN into a URL, like so:
    http://covers.librarything.com/devkey/KEY
    /medium/isbn/0545010225
  3. Put that in an image tag, like so:
    <img src="http://covers.librarything.com/devkey/KEY/medium/isbn/0545010225">
  4. And your website, library catalog or bookstore has a cover.

Easy details. Each cover comes in three sizes. Just replace "medium" with "small" or "large."

As with Amazon, if we don't have a cover for the book, we return a transparent 1x1 pixel GIF image. So you can put the cover-image on OPAC pages without knowing if we have the image. If we have it, it shows; if we don't, it doesn't.

The Catch? To get covers, you'll need a LibraryThing Developer Key—any member can get one. This puts a top limit on the number of covers you can retrieve per day—currently 1,000 covers. In fact, we only count it when a cover is made from the original, o our actual limit will be much higher. We encourage you to cache the files locally.

You also agree to some very limited terms:
  • You do not make LibraryThing cover images available to others in bulk. But you may cache bulk quantities of covers.
  • Use does not involve or promote a LibraryThing competitor.
  • If covers are fetched through an automatic process (eg., not by people hitting a web page), you may not fetch more than one cover per second.
You will note that unlike the new API to our Common Knowledge data, you are not required to link back to LibraryThing. But we would certainly appreciate it.

Caveats. Some caveats:
  • At present only about 913,000 covers are accessible, the others being non-ISBN covers.
  • Accuracy isn't guaranteed--this is user data--and coverage varies.
  • Some covers are blurrier than we'd like, particularly at the "large" size. This is sometimes about original files and sometimes about our resizing routines. We're working on the latter.
Why are you doing this? The goal is half promotional and half humanitarian.

First, some background. This service "competes" with Amazons cover service, now part of Amazon Web Services. Amazon's service is, quite simply, better. They have far more covers, and no limit on the number of requests. By changing the URL you can do amazing things to Amazon covers.

The catch is that Amazon's Terms of Service require a link-back. If you're trying to make money from Amazon Affiliates, this is a good thing. But libraries and small bookstores have been understandably wary about linking to Amazon. Recent changes in Amazon's Terms of Service have deepened this worry.

Meanwhile, there are a number of commercial cover providers. They too are probably, on average, better. But they cost money. Not surprisingly many libraries and bookstores skip covers, or paste them in manually from publisher sites.

That's too bad. Publishers and authors want libraries and bookstores to show their covers. Under U.S. law showing covers to show off books for sale, rental or commentary falls under Fair Use in most circumstances. (We are not lawyers and make no warrant that your use will be legal.) We've felt for years that selling covers was a fading business. Serving the files is cheap and getting cheaper. It was time for someone to step up.*

So we're stepping up. We're hoping that by encouraging caching and limiting requests, we can keep our bandwidth charges under control. (If it really spikes, we'll limit new developer keys for a while; if you submit this to Slashdot, we will be Slashdotted for sure!) And it will be good for LibraryThing—another example of our open approach to data. Although none of our competitors do anything like this—indeed our Facebook competitors don't even allow export although, of course, they import LibraryThing files!—we think LibraryThing has always grown, in part, because we were the good guys—more "Do occasional good" than "Do no evil."

If we build it, they will come. If the service really pick up, we're going to add a way for publishers, bookstores and authors to get in on it. We'd be happy to trade some bandwidth out for what publishers know—high-quality covers, author photos, release dates and so forth. We've already worked with some publisher data, but we'd love to do more with it.


*In the past, we had been talking to the Open Libary project about a joint effort. We even sent them all our covers and a key to the identifiers that linked them. But nothing came of it. To some extent that was our fault, and to some extent not. (I think them and us would differ on the blame here.) In any case, I was tired of the time and transactional friction, and wanted to try a different approach.

Labels: , , ,

52 Comments:

Anonymous Anonymous said...

Nicely done, Tim! Libraries and their patrons will be very appreciative.

8/07/2008 5:24 PM  
Anonymous Anonymous said...

I agree that charging for covers belongs to yesterday. Hope you don't mind me mentioning that Open Library covers are in fact available to WordPress bloggers via my OpenBook plugin. However, yours has much wider usefulness, i.e., those who don't use the WP plugins. Regards.

8/07/2008 9:04 PM  
Blogger Chris Dahl said...

Thank you. The more I read here, the more I learn. This is lovely. I look forward to using it soon.
Chris

8/07/2008 10:17 PM  
Blogger John. said...

Are there any copyright considerations for libraries using book cover artwork sourced from librarything?

8/08/2008 12:21 AM  
Blogger lucy tartan said...

Covers. Yes. This is something which makes me highly exasperated with LT.

When I look at all the user-uploaded covers of a work, all I can apparently do is choose a different cover for my own specimen - I can't find out any of the bibliographic info for each book behind the cover image. (or can I? If so it's very hard to see how, and I've looked.)

This isn't just idle curiosity, I actually want to know for research purposes connected to charting the publishing history of certain novels.

So frustrating to be so close to the information I'm looking for and yet unable to access it.

8/08/2008 8:10 AM  
Anonymous Anonymous said...

Huh huh huh.

Tim said "libary."

Huh huh huh.

8/08/2008 11:56 AM  
Anonymous Anonymous said...

I don't understand anything of this.

8/08/2008 7:49 PM  
Anonymous Anonymous said...

what is open book - i briefly looked - could see how to bulk upload

Is Library thing dating openbook??

8/10/2008 9:33 AM  
Anonymous Anonymous said...

my first time here. what a great site. keep up the good work!

8/11/2008 12:43 AM  
Anonymous Anonymous said...

"Nor are cover images copyrightable under most circumstances. "

Are you really sure about this ? I didn't know they had Fair-Use clause in other parts of the world.

8/11/2008 4:23 AM  
Anonymous Anonymous said...

How much local disk space will be required I wonder?

8/11/2008 10:09 AM  
Blogger Tim said...

How much local disk space will be required I wonder?

The big problem is disk block size. A 1 byte file doesn't actually take up 1k. It takes up one "block," whatever the size that is. (Imagine a rule that vehicles could only be packed one to a parking space, whether they were SUVs or roller skates.) On very large disks, block sizes can be quite significant.

8/11/2008 1:04 PM  
Blogger Jano said...

Hypothetically speaking, if a library would like to donate covers digitized in-house to LT, how would that go about? =)

8/12/2008 12:03 PM  
Blogger Tim said...

Send me an email (tim at librarything dot com) and let's chat. I'd love to do that. It's exactly the sort of thing that should happen and I'd be willing to do what I can to make sure it's how you want it done.

Tim

8/12/2008 12:25 PM  
Blogger Unknown said...

Any way to figure out if LT has the book cover image so that I can pull images from elsewhere if it does not?

8/13/2008 4:33 PM  
Blogger Tim said...

Any way to figure out if LT has the book cover image so that I can pull images from elsewhere if it does not?

Server-side: The standard technique with Amazon is to look at the header. If a GIF, it's the 1x1.

Client-side: After an image loads you can test the size of it. If 1x1, try elsewhere.

8/13/2008 4:42 PM  
Blogger ODULibrary said...

Sorry for the ignorance. Can anyone point me to some help on how to cache the covers?

Thanks in advance.

8/14/2008 12:19 PM  
Blogger Rob Szarka said...

I've been too busy to keep up with all the improvements to LT lately, but once again it looks like you're doing the Right Thing with your data. Kudos!

One suggestion: while some developers might prefer to cache images locally for further manipulation, it would be good to have an option to reference them directly without a devkey, i.e. with a URI that would be the same for everyone. An important part of this approach would be serving those images with an expires header set in the future, which would both save your bandwidth and improve performance for end-users. With this approach, your site's visitors would arrive with some cover images already cached by their browser from viewing them on other sites, and vice versa.

8/14/2008 4:02 PM  
Anonymous Anonymous said...

Please consider posting a list of the top 100 not found ISBN's after you have some stats to work with. That would provide a way for contributors to scratch the largest itches first.

8/14/2008 5:57 PM  
Blogger Lisa said...

I tried using the links you provided for info on Amazon's cover services, but I have been unable to find anything on the Web Services pages that seems even remotely related. I most often want to use covers for the ARCs I'm blogging about, and they seem to be the books most likely to not have a cover on LT.

8/14/2008 7:48 PM  
Anonymous Anonymous said...

Hi Tim! This is a great news!

As a librarian, I wonder how we could link to your covers from our OPAC?

How does the download process work for the images and metadata?

Would we have to wait 1000 days x 1000 images before we're able to create a local cache for all the images?

Cheers

8/15/2008 2:32 AM  
Anonymous Anonymous said...

The Developer Key link doesn't work? Can you help? Thanks!

8/15/2008 10:04 AM  
Anonymous Anonymous said...

This is an awesome development, don't get me wrong -- but it would be even better if your example encouraged people to add "alt" and "title" attributes to their image tags when they add them to their pages. It's hard to read the title & author of some books in those little images!

8/15/2008 1:16 PM  
Anonymous Anonymous said...

I'm having trouble with the developer key as well.

8/16/2008 2:39 PM  
Blogger Muskrat said...

I think I am misunderstanding something: If one "may not fetch more than one cover per second" for automatically fetched covers, won't that slow our OPACs down considerably?

We have thumbnail covers appearing on our OPAC hitlists (10 titles/page). Won't that guarantee a 10 sec MINIMUM load time for our hitlists? As someone pointed out earlier, even if we do local caching, it would be almost 3 years before every LT image was cached.

The idea is great, and sounds like it might work well for small libraries. But, unless I'm missing something, I can't see implementing it in a large, busy library.

8/18/2008 4:18 PM  
Blogger Jano said...

@muskrat: I think those are the "ground rules" and not real technical limits. And I think the limits only apply for automated fetching (meaning "someone wrote a program to systematically get covers from LT") instead of "normal" web browser image fetching.

In my interpretation, you can indeed use them in your OPAC, and yes, a patron viewing, say, search results, will launch 5, 10 or more quick cover requests to LT... but that (IMO) is allowed.

8/19/2008 9:40 AM  
Blogger Gert Goris said...

I want to use the LT-covers in the OPAC of an (academic) library.

I could use some advise from anyone with some experience on it... Anyone ?

How does LT relate to other services like Amazon or Google btw ?

Thanks
Gert (Belgium)

8/20/2008 4:13 PM  
Blogger Gert Goris said...

I would like more information about the collection of book covers (or books) on LT.

Mostly fiction / non-fiction, year of publication, language, subject etc.

PS thanks for the service LT and all of the contributors !

8/20/2008 4:19 PM  
Anonymous Anonymous said...

The big problem is disk block size. A 1 byte file doesn't actually take up 1k.

:)

But there's nothing stopping you packing thousands of 1 byte files into a single file, using a single disk block, if that's a problem for you.

8/24/2008 11:30 PM  
Anonymous Anonymous said...

Wow: "Use does not involve or promote a LibraryThing competitor."

"Involve", "promote", and "competitor" are pretty broad words here, and there's not a lot of detail about what they mean. Would you consider my project or site a competitor? Or any of the components "involved" in my project? How would I even find this out?

Even though you're clearly offering this great data in good faith, I don't know that you can consider something truly open if you build in a loophole like this that lets you exclude someone just because you don't like the content of what they do.

9/01/2008 2:16 PM  
Anonymous Anonymous said...

8/11/2008 10:09 AM Anonymous said...

How much local disk space will be required I wonder?

8/11/2008 1:04 PM Tim said...

How much local disk space will be required I wonder?

The big problem is disk block size. A 1 byte file doesn't actually take up 1k. It takes up one "block," whatever the size that is. (Imagine a rule that vehicles could only be packed one to a parking space, whether they were SUVs or roller skates.) On very large disks, block sizes can be quite significant.


AFAIK, the disk size is irrelevant. Rather, one is to take into account the partition size.

Actually, these are some real life examples when using e.g. Linux OS :

The limits are as follows :
-----
Attribute : ext2
Maximum partition size** : 4TB
Maximum file size* : 2GB-2TB
Block size : 1KB-4KB
-----

If you want really big partitions, try and use a different attribute :

-----
Attribute : ReiserFS
Maximum partition size** : 16TB
Maximum file size* : 8TB
Block size : 4KB only
-----

*The maximum file size for ext2 is actually dependent on the choice of blocksize and hardware architecture.
**The maximum partition size for ext2 is actually dependent on the choice of blocksize and hardware architecture.
-----

For LT is accepting book cover images up to 600K file size now, it seems that block size (1-4K for partition sizes up to 4TB) -- and storage space loss because of not filling up entire blocks -- turns out to be rather irrelevant.

Real life example :

As it turns out, in order to store and keep 1,000,000 original size user provided book cover image originals, each of them up to 600kB max, even when maxing out all of those (1,000,000 * 600kB = 600 GB), a lousy 750 GB HDD is sufficient and has got quite some spare room remaining indeed.

At your home system, you may be using a 750 GB HDD (street price ~$150) or even a bigger one already (today, spending ~$200 can get you a 1 TB HDD), and your typical block size will be in the 512 bytes - 4 kB range.

FYI : Even when using "big" block size (4 kB), a book cover image "small", will need and use no more than 3 blocks (12 kB) max -- though some exceptions may need 4 blocks (16 kB) indeed, but many more will suffice with 2 blocks (8 kB) only.

Fair enough, when using 4 kB block size (big blocks, that is), and your image size is 8,193 bytes (8 kB + 1 byte), the third byte's space will be most entirely wasted.

Therefore, if you have got lots and lots of (very) small files, you may want and keep them in a disk partition with minimal block sizes indeed.

As it turns out, keeping with the SUVs and roller skates example, even when differentiating more (e.g. trucks and roller skates), would you rather have them parked and occupying bricks or stones.

If someone thinks that somehow my explanation, numbers, examples, either mathematics or conclusions are wrong, instead of calling me names, better try and tell me what and where and why I 've got it wrong. Thank you.

9/09/2008 8:16 PM  
Blogger Tim said...

I don't know the details, but I suspect that's true. Um, so?

9/09/2008 9:06 PM  
Blogger Tallmanicus said...

This has a lot of potential and is much appreciated! Now, the question: does the "download" of a cover count in the daily quota of 1000 if only the 1x1 gif is returned? Our spider is finding very few of our academic titles. We'd like to try to get 1,000 actual images/day instead of 1,000 errors.

9/15/2008 11:13 AM  
Blogger Tim said...

It depends if the image has been requested before in the last week or so. If yes, it doesn't count. If no, it does.

Changes are, if you're spidering, you're requesting mostly new stuff.

9/15/2008 11:29 AM  
Blogger Tim said...

It depends if the image has been requested before in the last week or so. If yes, it doesn't count. If no, it does.

Changes are, if you're spidering, you're requesting mostly new stuff.

9/15/2008 11:30 AM  
Anonymous Anonymous said...

I just started testing out the free covers. I put the code into the bibimage wwwoption. Mostly what I get is broken graphic links. Any idea on what I am doing wrong? Example: http://loveland.lib.co.us:2082/search/X?the%20juror&Da=&Db=&p=&SORT=D

9/19/2008 4:22 PM  
Blogger Mack said...

I'm confused. I got a developer key. I pasted the example into a blog post with my developer key and the isbn for a book the cover of which I uploaded and selected as the cover I wanted to display. It doesn't display in the blog post. What am I missing.

9/19/2008 11:37 PM  
Blogger Tim said...

The cover for an ISBN is not dynamically updated. It takes a process, during which LT figures out what the *best* cover for a given ISBN should be.

9/19/2008 11:59 PM  
Blogger Mack said...

Hh, thanks Tim. I suspected that it wasn't dynamic but wanted to be sure. I did inspire me to scan some covers that were not represented in LT and that was fun.

9/20/2008 9:02 AM  
Blogger Mary said...

Is there a way to use the images on a secured catalog server? We don't want our users to have the "some of the items on this page aren't secure" warning popping up on every page.

9/23/2008 10:56 AM  
Blogger Mack said...

Tim,

I'm curious about the process LT goes through to select a cover for display. I uploaded 4 covers and they are the only user uploaded covers available for those books. I put the URL for the covers into a blog post on September 20. I made sure they have the same ISBN as the books I added. I'm still seeing the little square instead of the cover. Is this common?

9/28/2008 2:17 PM  
Blogger JVK said...

I want to retrieve some of the images I have scanned and uploaded at full size, because I had a hard drive crash before I could upload them to Flickr.

Is this possible?

I've put my devkey and isbn in the suggested html but nothing is coming back.

10/20/2008 1:18 PM  
Anonymous Anonymous said...

Please I uploaded 4 images associated each one to an isbn but when using my developer key nothing shows up? How many days does it take to LT to decide if my cover books shows or not?

12/15/2008 1:45 PM  
Blogger Unknown said...

I am in process of building a website, and it allows users to add books via isbn. Currently I retrieve data via isbndb (title/summary). I would also like to grab cover art (perhaps from LibraryThing). My question is can we actually download/scale/store the images, or must the images only be referenced from the within the web-pages? (we have no problem adding 'powered by LibraryThing') In turn our users also can upload images for their books, I would be happy to make these images available for download into LibraryThing.

2/11/2009 8:23 PM  
Blogger Kewl Librarian 1 said...

I have just successfully gotten your book covers to work on an III system. However, when you return the 1X1 gif files, I get a broken link in IE. (It's happy in Firefox of course.)
Any thoughts?

3/25/2009 5:43 PM  
Anonymous Anonymous said...

There is an error when I was trying to get bookcover. It keeps redirecting to an coverthing404.php page.

Is there something wrong with LT book cover?

6/22/2009 9:27 PM  
Anonymous Anonymous said...

The URL starting with http://covers.librarything.com ... doesn't work. Changing it to www.librarything.com ... fixes the problem.

6/23/2009 11:05 PM  
Blogger Susie said...

I still can't get the covers to show up even using the address change and all the covers that I had, have disappeared!

7/01/2009 12:05 PM  
Blogger Suja PV said...

I have the same problem..

"There is an error when I was trying to get bookcover. It keeps redirecting to an coverthing404.php page"

Someone please help...

7/10/2009 2:45 AM  
Anonymous Anonymous said...

I am a church librarian and want to pirnt book covers so that I can put them on a bulletin board outside the libary. What is a devkey? How do I obtain one?

8/06/2009 3:09 PM  
Anonymous abhi said...

while trying to fetch coverpages i dont know if the coverpage exists for a given ISBN. is there a way to handle that.. ex: url returns 404

1/16/2010 3:26 PM  
Blogger Unknown said...

Hai,

I need to get the number of images i have downloded today and what is limit remaining today using API ,

How can i get that , Please help me !!

1/18/2010 9:44 AM  

Post a Comment

<< Home