Monday, March 01, 2010

Beta: "Read Alike" recommendations

I've pushed through a beta version of a new recommendation engine. The "Read Alikes" recommendations supplement our existing automatic and member recommendations. "Read Alikes" are based directly on the members who have your books—the people who "read alike" you, or whatever.

So far, opinion is divided. Some members love it, and are getting great recommendations. Others report a parade of things they already know about. Is it quite consciously, however, a beta feature. It may be improved, or it may go away. Most likely, it will go away and be replaced by a better overall algorithm, with better tools for managing your recommendations.

Labels: ,

Monday, February 15, 2010

Dead horses to ponies...

Borrowing a joke from Brightcopy, I've turned some dead-horses into ponies, bringing some long-requested features to life, and even improving on them.

Books You Share Preferences. Some members have long campaigned for sorting the profile-page "Books You Share" box by author, not title. But I held off—that's not the right choice for everyone. Instead I've added a preference for it, with a number of different sorting options.

Critically, I set the default to sorting by popularity from low to high, something nobody had ever requested. I thought members might pounce on me for it, but quite a few have said it was an unexpectedly good choice. It brings out the unusual books you share. And those are often the most interesting.

I also added a preference to change how many shared books are displayed.

See this topic for more about the feature.

Tag Combination. After a 16-month hiatus, new tag combinations and separations are back!

The idea is simple. LibraryThing allows members to combine tags that are highly similar in meaning and application. Classic examples are tags like "World War II" and "wwii" or "ww2." We discourage combining terms that don't entirely overlap, either in meaning or in usage. (If you're interested in the ideas behind tagging, check out my What's the Big Deal About Tagging? talk on YouTube.)

Tag combination only affects "global pages"; user tags are never changed.

So far as I know, we're the only website to experiment with this idea, something noted in Gene Smith's Tagging: People-Powered Metadata for the Social Web. Tag-combination combines a new idea—tagging—with an older idea—what librarians call "authority control."

This time, however, we've given it a twist—democratic authority control. Any member can propose a combination or separation, but the matter is put up to a vote—with a supermajority needed for any action. We hope it will slow down the process and make it more deliberate.

It'll also save our servers from having to recalculate tags. With more than 60 million tags, and "science fiction" now at three million uses, instant, any-user combinations were really putting a strain on our system.

See more about it, and some examples here.

Labels: ,

Thursday, February 11, 2010

Favorite messages and marking last-read

I've added a simple feature to "favorite" message. You can also mark a last-read, for those times when Thingamabrarian eloquence prevents a topic from being read in a single sitting.



More here.

Labels:

Wednesday, February 10, 2010

Pictures get a lot better

I've just released phase one of the new picture system—much better profile pictures:

The new system allows members to post multiple profile pictures, see pictures at large sizes, describe pictures, leave comments on them, and share them with other members. There's also a tagging feature, so members can organize their pictures and swarm around common tags, like my library.

The new system was designed to be used across the site. I am particularly anxious to get it working on books—so members can show multiple images, and separate out covers, title pages, spine images and so forth.

Read and talk more about it here.

Labels: ,

Tuesday, February 09, 2010

Connections lists and other profile-page upgrades

Mike has revamped LibraryThing's "connections" feature, changing the UI somewhat and allowing members to specify new types of contacts, like "best friends," "employees," "librarian peeps," etc. These categories show up everywhere connections do, such as work pages.



There were also significant changes under the surface, preparing us for better contact handling generally—both inside-LibraryThing contacts and reaching out to other social networks.

We've also added an area for "Groups you share."



Members are divided over this addition. There's a poll being held about it right now.

There were a host of smaller changes. Read more about it on Talk.

Labels:

Monday, January 11, 2010

Colored check marks

I've change the green check marks to a palette of four colors, so you can distinguish at a glance books in your library, wish list, read-but-unowned list and other collections.

The check marks pop up on recommendations and statistics pages. So far, members have been particularly happy to see them on your Series Statistics page, which lists all the series in your library, and which books in the series you have in the various collections.

As should become immediately obvious, colors can't do justice to all the complexity of collections. Not every collection gets a color, and we aren't mixing up paint buckets when a book belongs to two collections. We may revisit this, allowing members to set colors for their collections, but it will depend on other priorities, and we'll never be able to squeeze all the information in collections into colors alone. For what they are, however, we hope they're useful.

See more information and discussion.

Labels: ,

Sunday, November 22, 2009

The Books of Wikipedia

UPDATE: I'm going to take another swipe at the data here. I've found too many places where it looks like citations aren't registering. I'll update when I can. Meanwhile, I'm moving the Secret Santa post to the top again. (Tim)

I've added a new "References" section to work pages, and within that a list of the Wikipedia articles that cite the work.

This feature comes from a complete parse of the English Wikipedia data dump, looking for citations and other references to books.

All told, the data covers 251,911 pages and includes 540,000 citations. They cover some 227,852 works. This is a marked improvement over 2007, when a similar effort found citations to LibraryThing works on only 90,136 pages. It's not perfect. I don't try to capture non-ISBN references in running text.* But it's interesting.** Come talk about it here.

Here are the top 100 books:

Top 100 Most-Frequenty Cited Books in the English Wikipedia
1.2,122Guinness World Records: British Hit Singles and Albums
2.1,313Andrews' Diseases of the Skin: Clinical Dermatology (James, Andrew's Disease of the Skin)
3.1,231Air Force Combat Units of World War II
4.1,184Jane's encyclopedia of aviation
5.839British parliamentary election results, 1918-1949
6.764The Ship of the Line: The Development of the Battlefleet, 1650-1850 (The Ship of the line)
7.603Handbook of British Chronology (Royal Historical Society Guides and Handbooks, Volume 2)
8.603Conway's All the World's Fighting Ships, 1860-1905
9.591The science-fantasy publishers: A critical and bibliographic history
10.560Civil War High Commands
11.539Wrestling Title Histories
12.514A Plain Introduction to the Criticism of the New Testament (2 volumes)
13.504The Concise Oxford Chronology of English Literature
14.464The Dinosauria
15.463The DC Comics Encyclopedia, Updated and Expanded Edition
16.460The Birth of the Palestinian Refugee Problem
17.452The Canadian directory of Parliament, 1867-1967
18.442All That Remains: The Palestinian Villages Occupied and Depopulated by Israel in 1948
19.419Air Force combat wings : lineage and honors histories, 1947-1977
20.415The Encyclopedia of Science Fiction and Fantasy Through 1968; Volume 1: Who's Who A-L
21.414The New Grove Dictionary of Music and Musicians
22.406Fields of Praise: Official History of the Welsh Rugby Union, 1881-1981
23.403Birmingham City
24.398The New Grove Dictionary of Opera : A-D
25.377Kurzgefasste Liste der griechischen Handschriften des Neuen Testaments
26.371Fitzpatrick's Dermatology In General Medicine (Two Vol. Set)
27.362NFL 2001 Record and Fact Book
28.356The Marshall Illustrated Encyclopedia of Dinosaurs & Prehistoric Animals: A Comprehensive Color Guide to Over 500 Specie
29.348The Pimlico Chronology of British History: From 250, 000 BC to the Present Day
30.347Michigan Place Names (Great Lakes Books)
31.345The Directory of Railway Stations: Details Every Public and Private Passenger Station, Halt, Platform and Stopping Place
32.342The Oxford Companion to Wine, 3rd Edition
33.341Collins Guide to the Sea Fishes of New Zealand
34.334The Book of Sydney Suburbs
35.331The Men Who Made Gillingham Football Club
36.331Register of Ships of the U.S. Navy, 1775-1990: Major Combatants
37.315Oregon Geographic Names
38.305日本写真家事典―東京都写真美術館所蔵作家 (東京都写真美術館叢書)
39.296U.S. Submarines Through 1945: An Illustrated Design History
40.294Chronicle of Gods and Sovereigns
41.289The Encyclopedia of Science Fiction and Fantasy Through 1968: A Bibliographic Survey of the Fields of Science Fiction, F
42.281Reed New Zealand Atlas
43.277Music in the Renaissance
44.277Ohio Atlas and Gazetteer (Atlas and Gazetteer)
45.273Enzyklopädie des deutschen Ligafußballs 7. Vereinslexikon
46.271The Text of the New Testament an Introduction to the Critical Editions and to the Theory and Practice of Modern Textual
47.268Blackpool (Complete Record)
48.266The PFA Premier & Football League players' records, 1946-2005
49.261The Oxford Companion to Chess
50.259Dictionary of Minor Planet Names
51.256A Guide to the Birds of Costa Rica (Comstock Book)
52.251Saints!: Complete Record of Southampton Football Club, 1885-1987
53.247Cassell's Chronology of World History: Dates, Events and Ideas That Made History
54.247The Book of Golden Discs
55.243Squadrons of the Royal Air Force and Commonwealth, 1918-88
56.236Retreat to the Reich: The German Defeat in France, 1944
57.236RAF Squadrons: A Comprehensive Record of the Movement and Equipment of All RAF Squadrons and Their Antecedents Since 191
58.231Australian Chart Book 1970-1992
59.231The Phoenix Book of International Rugby Records
60.228Warships of the Imperial Japanese Navy, 1869-1945
61.224US Air Force Air Power Directory
62.222Handbook of Inorganic Chemicals
63.221Rough Guide to World Music Volume Two: Latin and North America, the Caribbean, Asia & the Pacific (Rough Guide Music Gui
64.219In the Nick of Time: Motion Picture Sound Serials
65.215A Guide to the Birds of Trinidad and Tobago
66.212Encyclopedia of Fishes, Second Edition (Natural World)
67.208The Clements Checklist of Birds of the World
68.205The Kentucky Encyclopedia
69.205Arsenal Who's Who
70.205Guia de Catalunya. Tots els pobles i totes les comarques
71.204Armor Battles of the Waffen SS, 1943-45 (Stackpole Military History Series)
72.203Sixty Years of Arkham House: A History and Bibliography
73.203Indie Hits: The Complete UK Independent Charts 1980-1989
74.201The Complete Book of the Summer Olympics
75.201The geographic atlas of New Zealand
76.201The International Rugby Championship 1883-1983
77.200The Virgin Encyclopedia of Reggae (Virgin Encyclopedias of Popular Music)
78.199Arkham House Books: A Collector's Guide
79.198Complete Encyclopedia of World Aircraft
80.196Conway's All the World's Fighting Ships: 1906-1921 (Conway's All the World's Fighting Ships, Vol 2)
81.195The fighting ships of the Rising Sun: The drama of the Imperial Japanese Navy, 1895-1945
82.192Economics: Principles in Action
83.191New RHS Dictionary of Gardening
84.190Fungal Families of the World (Cabi Publishing)
85.189The Vertigo Encyclopedia
86.188Dermatology: 2-Volume Set
87.187The National Register of Historic Places in Minnesota: A Guide (Minnesota)
88.187Minding the House: A Biographical Guide to Prince Edward Island MLAs, 1873-1993
89.186Domesday Book: A Complete Translation (Alecto Historical Editions)
90.186The Beaulieu Encyclopedia of the Automobile (3 Vol Set)
91.185Mit Eichenlaub und Schwertern : die höchstdekorierten Soldaten des Zweiten Weltkrieges
92.184The Mountains of England and Wales, Volume 1: Wales
93.183The Gangs of New York: An Informal History of the Underworld
94.183The Oxford Dictionary of Opera
95.179Generals in Blue Lives of the Union Commanders: Lives of the Union Commanders
96.178The Arkham House Companion: Fifty Years of Arkham House : A Bibliographical History and Collector's Price Guide to Arkha
97.177David and Charles Book of Castles
98.175Encyclopedia of Contemporary Christian Music (Recent Releases)
99.174The Simpsons: A Complete Guide to Our Favorite Family
100.171Hot Dance/Disco 1974-2003
Source: LibraryThing.com


Here's the table above, in selectable form, in case you want to put it on your blog:




*The parse used ISBNs, OCLC and LCCN numbers and title/author combinations, in both running text (for ISBNs) and citations. In 2007, I used titles and authors in running text. But this produces some false positives and, basically, would have tied up a server for a week.
**I don't want to start a fight, but I think the winners suggest alarming unevenness. They look like special cases—a bunch of devotees going citation-mad. While everyone knows Wikipedia is driven by passion, the theory is that large numbers and diverse viewpoints tamp down some of the excesses there. That said, I am going to take another crack at the data, hoping some of these effects diminish.

Labels: , ,

Wednesday, July 22, 2009

HelpThing: Member-driven help

We've added a "Help" button to every page of LibraryThing. The button goes to "HelpThing," a member-driven help system taking shape as we speak:

The idea is simple:
  • Every page on LibraryThing gets a HelpThing page
  • HelpThing is wiki-editable by any LibraryThing member
  • Members and staff collaborate to create a detailed, but accessible guide to LibraryThing
HelpThing started as a "stealth project" by LibraryThing programmer Chris (ConceptDawg). It took a while before I was convinced of the idea.

While I was ignoring the idea, however, members were busy realizing it, official sanction or no. Most of the content was written by LibraryThing member fyrefly98, with contributions from mvrdrk. A somewhat separate—but integrateable—guide to collections was produced by PortiaLong and Lquilter. These members, and the others who helped them, are simply awesome.

Well, now it's your turn. From being a non-feature, then a Beta feature, it's now available for everyone to edit. To bring some structure to it, and because, well, I'm still a little afraid of it, I started a HelpThing Style Guide, and fixed up a few pages.

Come and discuss the feature on the New Features post. Ongoing conversation can be had in the Common Knowledge and WikiThing group.

Three cheers for Chris and everyone who's worked on it so far. Now let's make it as helpful and compelling as we can!

Labels: ,

Friday, July 03, 2009

Twitter your reviews

We've added a feature to make it easy to Twitter (or Tweet) your reviews.



You'll see the option—a tiny Twitter logo—on your reviews. When you click it, it takes you to Twitter and fills in the message box. You can, of course, edit it however you like.



You can spot most such tweets with this Twitter search.

This is our second Twitter-based feature. The other is an easy way to Twitter your books to LibraryThing, handy for making a note of a book when you're in a bookstore or library. Like that, the Twitter your review feature is all about restraint and options. We've rejected the idea—popular among book and non-book sites—of automating that process, of making it easy to machine-gun all your friends and followers with trivial updates.

Are you on Twitter? Follow us. Most LibraryThing-related news comes from my account, LibraryThingTim. The LThing account is for incoming messages mostly. John, Chris and Luke are also on, discussing LibraryThing's irrationally vague vacation policy.

Labels: , , , ,

Monday, June 29, 2009

More servers, less sleep

We just finished moving about a dozen of our original servers from our "colo" in Maine to one in Somerville, where another dozen were waiting. In the process, we've basically doubled our server power.

We're still waiting to get all our metrics back up, and we have a few weeks of retasking servers (a lot will be wearing different hats), but, so far, the results have been very encouraging. We are faster today, and will be getting faster tomorrow. We have a lot more memory, disk space and system redundancy too—so keep adding books.



We've collected all our pictures on a Flickr tag Great LibraryThing Server Schlepp. Here are some of the better ones.

The move was a group affair. Abby, Sonya, Mike, Dan and I did all the physical work. Our Australian systems administrator directed us by video chat—and burned through his monthly bandwidth doing it. Abby and I did a trial run with one server on Wednesday. The rest pulled an all-nighter, except Sonya who arrived like a well-rested and showered cavalry at 7am. When we were done, Mike and I went back to my parents' in Cambridge and slept like logs. Abby, who was taking care of a toddler, stayed up the whole day. Ouch—and kudos to her.

Labels: ,

Tuesday, June 09, 2009

Collections, at last

It's arrived. Members can organize their books into "collections."

The Motive. From the beginning, LibraryThing members have used the site for different things. Some used it to list only the books they own, others what they've read and a few even just the books they wanted. Meanwhile, people like me used it for everything—owned, read, lost, destroyed, wanted—using tagging as our sole way of keeping everything straight. But even tag-zealots like me had to admit there were times you wanted sharper distinctions—"buckets" or "sub-libraries"—and ways to tie those to how you connected with other members and with book recommendations. New members, whether familiar with tags or not, were regularly asking for some way to do wishlists and currently-reading lists.

The Feature. The feature, literally years in the making, gives members the ability to separate out categories of books, like "Wishlist" and "Currently reading" more definitely than could be accomplished with tags. Each collections works like a mini library and can be separately viewed, sorted and searched. Other members can see your collections, on your profile and elsewhere. Features like member-to-member connection and book recommendations react to the new system as well. (See below on integration progress.)

As we offer users new flexibility, we avoid forcing members into "our" way of thinking about books. We've provided a number of default collections—Your library, Wishlist, Currently reading, To read, Read but unowned and Favorites. Data from these collections can be aggregated across all users, and their names are even translated on LibraryThing's non-English sites. But you can also create your own collections, and remove ours. And you can ignore collections entirely, keeping everything in "Your library."

A Work in Progress. As members know, we play things pretty fast and lose here. Our motto is "beta, forevah!" But collections had to be different. Before public release we subjected it to a month of testing in our large (and non-exclusive) BETA Group. We cannot thank that group enough for all the work they did, and the passion they showed.

We hope we got most of the major bugs, but the feature is not "finished"—and this is hardly the last blog post you'll see about the feature! Most significantly, collections is now mostly a "cataloging" feature, with only limited reach to other areas of the site. Although you can specify how collections affects member connections and recommendations—so you can stop having your Wishlist or for that matter your husband's books running the social and recommendation parts of the site—implementation is basic and, in light of extraordinary collections-related load, there's a lot of caching in place. We left a few features out in order to get it the main features out now.*

We also think "unfinished" (we prefer not prematurely specified) features are the best way to engage users, and get the best for everyone. Come and contribute on Recommended Site Improvements and Bug Collectors. We also have a Announcement post in New Features.


*We had spec'ed out a complex interaction between reading-dates and "Currently reading." But the system was probably more than most members wanted. And it certainly was taking a long time to finish, so, for now at least "Currently reading" is just a collection.

Credits: Chris (conceptDawg) headed up the project, doing most of the user interface and a majority of the back-end code. Chris and I (timspalding) designed the feature together, and I did some core back-end code. Abby (ablachly) didn't code, but she dogged us about it for years. (I'm not sure what she's going to do with herself now.) But the most important factor was the members. Members, particularly the BETA group, contributed to the effort as I've never seen it—not in any website or project, ever. Chris and I owe members an enormous amount. (I'll be blogging about this specifically soon. It needs telling.)

Top photo by radiant_guy" (Flickr, CC-SA).

Labels: , , , ,

Monday, May 25, 2009

Better statistics, other improvements

I spent the weekend cooking up code, not sausages:

1. Series statistics. By popular demand, the member Series Statistics page can now show your series books you have in context of the complete series. (See talk post.)

2. Awards, characters and places. I've added similar statistics pages for three other "Common Knowledge" categories—Awards, Characters, Places. (See talk post.)

I also added series, awards, characters and places stats in your profile* and the "Your Zeitgeist" box on your home page (see talk post.)

3. More Green Checkmarks. Green check-marks, the mark that shows when you have a work, have spread further. They are now appearing on work-page recommendations, recommendation pages and in other members' catalogs. (See talk post.)

4. Power Edit gets better Previously, you could only Power Edit a page at a time (ie., no more than 100 books at a time). I added a feature to allow you to power-edit all the books in a given result set. So, you can do all your books, all the books that match a particular search, etc.



See the talk post.

5. Message Flagging. I've improved message-flagging in Talk, so that members can reverse their flagging, as well as counter-flag a message, if they think it was wrongly flagged. (See talk post.)

I also proposed making the Wikipedia policy "Assume good faith?" an official LibraryThing policy, triggering a lively debate about community norms, just what spam is and so forth. See the talk post.


*Originally high, but I moved it down when members hollered.

Labels: , ,

Friday, May 22, 2009

Author interviews posted

Our first two author interviews, first seen in the revamped State of the Thing newsletter, are live on the site itself. The interviews are:Abby and I enjoyed reading the books—we both read from Bad Mother (thumbs up, but it will tweak you), and Abby read The Song is You (thumbs up)*. Philips' interview convinced me I should check him out. "A child actor, a jazz musician, a speechwriter, a dismally failed entrepreneur, and a five-time Jeopardy champion" and a huge fan of Pale Fire? Will he come to our next party?

Want to do an author interview? Know someone who might? We're looking for authors, and we'd rather get great ideas from members than declare open-season on our inboxes from publisher PR types. Email abby@librarything.com about it.

We have a number of other things authors and publishers can "do" with LibraryThing on the about page. We'll be sprucing it up in time for Book Expo America in New York.


*I think we have to stop saying if we liked a book, as we'll eventually read one we positively hate and "Check out this interview with Mr-Can't-Write" probably won't win us any friends.

Labels: , ,

Thursday, May 21, 2009

State of the Thing

Our monthly "State of the Thing" newsletter just went out.

We "did it up," with a full HTML version and some special features. Notably, the May newsletter includes our first two author interviews:

Ayelet Waldman, author of the Mommy Track Mysteries, and now Bad Mother
Arthur Phillips, author of The Egyptologist and The Song is You

We ask penetrating questions like "Describe your library" and "Is your husband really that perfect?" We'll be doing more of them as time goes by, and making them available on the site generally.

For now, however, you can only get them in the newsletter. So if you want to get a copy, be sure to edit your account preferences. We'll send you out a copy soon after.

Labels:

Wednesday, May 13, 2009

Compare your library with others'

I've added a new "Compare: Connections" feature, similar to the Legacy Library feature. But instead of munging statistics about Jefferson and Yeats, the new feature works on your friends, interesting libraries and other connections lists.

Members: find yours here.
Others: here are mine.

As with the Legacy Libraries page, you get a couple options. First, you choose which list you want to look at—friends and so forth. Then try:
  • Shared books, books. Shows the most popular books you also share.
  • Shared books, people. Shows all the connections, with how much they share.
  • Top books. Shows the most popular books, whether or not you share them. It's a "Most Popular Books" for your friends and other connections.
I'm anticipating some time-outs on larger libraries. The calculations here are brutal—100MB of RAM is not atypical. I'll mitigate them tomorrow.

A feature for LibraryThing Authors. I've also added a small, but cute feature for LibraryThing Authors. In certain circumstances, LibraryThing Authors now get a "Your Readers" list, alongside their friends and so forth. Right now, these lists are appearing on all author and work pages—again, only if you are a LibraryThing author. They aren't showing up in the "Compare" feature because many authors have hundreds or thousands of readers, and the system can't handle all that calculation right now.

Talk about both features on this blog or here.

Labels:

Monday, May 11, 2009

Who reads an author?

I've "brought back" top member lists on author pages, significantly enhancing them with lists that show the author's readers among your friends and connections, and among Legacy Libraries (eg., C. S. Lewis had a lot of Twain).

Also new, LibraryThing Authors now get a new "Your Readers" connections category, so they can find out what your readers think of a given author or work.



Discuss here

Labels: , , ,

Tuesday, April 28, 2009

Commune with the dead

Can you guess what they are?
I've made some major changes to members' Legacy Library pages, bringing this wonderful member project—the private libraries of over 100 readers from the past—closer than ever before.

It has never been easier to compare the reading of Jefferson and Adams (427 books!), Hemmingway and Fitzgerald. And is has never before been possible to compare that of Tupac Shakur and LibraryThing's Australian systems administrator John Dalton!

The core, default feature is a list of Legacy Libraries and the books they share with you. New features include:
  1. You can get it book-by-book, instead of person-by-person.
  2. From that, you can now see the top shared books across the Legacy Libraries, with you or any subset.
    The top books list is somewhat surprising. I've pasted it on the right, with the titles blacked out. See if you can guess number one. For combination reasons it's not the Bible, but it's probably not any of the others that leap immediately to mind. The top books between signers of the Declaration of Independence is also quite surprising. And why on earth did three American presidents bother to acquire General view of the agriculture in the county of Somerset?!
  3. The libraries are broken down into groups, so you can see what you share with actors, musicians, politicians, etc.
    Among these are the splinter project, the Libraries of Early America, which Jeremy, the Legacy Library project leader, is working on in collaboration with archives, libraries and museums across the country.
  4. You can filter everything in all sorts of clever ways.
  5. Although the page is a dynamic explorer, it provides a permalink to send to friends and a nifty "Share on Twitter" button. (Did you know you can enter your books through Twitter?)
Later today I'll push out a podcast I did with Jeremy, a long but enjoyable romp through the legacy libraries, cataloging, the meaning of books through history and book-love generally.

Discussion going on here.

Labels: ,