Neil Gaiman explains the Unsuggester

Neil Gaiman (American Gods, Neverwhere, etc.) has mentioned LibraryThing on his blog before, but today's entry really caught our eye. It wasn't just that he might join up, but he wrote a dead-on explanation of Unsuggester and what statistics mean and what they don't. Here's his reader's email and his response:

Neil, I think the unsuggester might be broken. I've tried multiple titles, including yours, and there is always at least one or two items from my library on the list...

It's not broken, it's simply pointing out statistical anomalies—it's not even talking about whether or not you'd like something (as some people have written to me, complaining they like books from both lists). It's simply saying that it ought to be able to find a certain number of copies of book Y for people who own Book X, and it can't. Statistically, people who have a copy of Mein Kampf on their shelves, for whatever reason, have fewer copies of Terry Pratchett books than might be expected. It may be that all the people with both Mein Kampf and Guards! Guards! just aren't on LibraryThing yet, and once they join the anomaly will vanish. Or it may be that there's something to be learned from that.

Would that he had been on Slashdot when that cry arose! Actually, part of the problem may be that people put their books in and try the Unsuggester the same day. But the calculations are pretty hairy, so I have to cache them for a while.