Wednesday, October 10, 2007

Common Knowledge: Social cataloging arrives

Chris has just released Common Knowledge, the innovative, open-data and insanely addictive "fielded wiki" we've been talking about for a month.

Common Knowledge adds fields to every author and work, like:
  • Author: Places of residence, Awards and honors, Agent
  • Work: Important places, Character names, Publisher's editor, Description
All-told there are fourteen fields. But Common Knowledge is less a set of fields than a structure for adding fields to LibraryThing. Adding more fields is almost trivial, and they can be added to anything existing or planned—from tags and subjects, to bookstores and publishers. They can even be added to other Common Knowledge fields, so that, for example, agents and editors can, in the future, sport photos and contact information.* This can lead to, as Chris puts it, "nearly infinite cross-linking of data."

Common Knowledge works like a wiki. Any member can add information, and any member can edit or revert edits. All fields are global, not personal. Common Knowledge diverges from a standard wiki insofar as each field works like its own independent wiki page, with a separate edit history.

Some example:
  • Jonathan Strange and Mr. Norrell. I've been conservative with characters and places. (See Longitude, worked on by Chris for the opposite approach.) But I wish I had her editor!
  • The history page for "important places" in Jonathan Strange and Mr. Norrell, showing improvement over time.
  • David Weinberger. Half-filled. He mentions his agent, but I can't tree his major at Bucknell and the honors section is empty.
  • Hugo Award Winners. This is going to get very cool.
  • The global history page. Mesmerizing.

Right now we're basically slapping fields on pages, but this structure is built for reuse. The license is also built for reuse. We're not asking members to help us create a repository of saleable, private data. Whatever you add to Common Knowledge falls under a Creative Commons Attribution license. So long as you include a short notice (eg., "Powered by the LibraryThing community"), you can do almost anything you want with the data—take it, change it, remix it, give it to others. You can even sell it, if someone will buy it. Regular people, bookstores, libraries--even our competitors--are free to use it. We'll be adding APIs to get it out there all the more. Go crazy, people.**

Common Knowledge isn't the answer to everything. Some data, like web links, requires a more structured approach; some, like our "work" titles, works best when it "bubbles up" from user data; and some, like page counts, have yet to be extracted from the MARC and ONIX information we have. But the possibilities are great. Series information? Blurbers? Cover designers? Books about an author? Tag notes? Other classification schemes?*** Bookstore locations? Publicists? Venues? Book fairs? Pets? Pets' vacination dates?

Anyway, we've done our thinking, but this is the ultimate member-input feature. We're going to have to figure it out together. Fields will need to be added (and removed?). Rules will be debated, formatting discussed. Although the base is solid, the feature set is still skeletal.****

Go ahead and play. Chris, John and I spent the evening playing with it, and we guarantee it's addictive. Or talk about. Leave a note here. I've also changed the WikiThing group into a Common Knowledge and WikiThing group. I've started a first-reactions topic and another for bug reports.

Why I'm excited. LibraryThing means a lot of things to a lot of people. Some come for the cataloging, some for the social aspect. A lot come for what happens between those two poles. As I see it, Common Knowledge is the perfect LibraryThing feature. I don't mean it's good; I mean it's in tune with what makes LibraryThing work. It's social, sure, but it's based in data. It's not private cataloging and it's not MySpace-like "friending."

LibraryThing is sometimes called a "social cataloging" site. When I used this term at the American Library Association, it became an unintentional laugh line. Social cataloging sounded impossible and funny, like feline water-skiing. This more than anything else got me fired up about doing this. True "social cataloging"; it was an idea that had to be tried!*****

Details, acknowledgements and caveats. Common Knowledge is deeply unstructured. This is going to give some members hives! Names aren't in first-middle-last format, but free text. You can enter places however you want. We've arranged some careful "hint" text, and fields have a terrific "autocomplete" feature, but we're not validating data and returning hostile error messages. We're aiming for accessibility and reach, not perfection. This is Wikipedia, not the Library of Congress. It scares us too, but we're also excited.

Abby, Casey, Chris and I planned this feature during the Week of Code. We worked through the issues together, and Casey, Chris and I all wrote the initial code. When we broke up, the rest of the coding and the interface design all fell to Chris. Although it was a team effort, this is really his feature. I'm very pleased with what he did with it.

We decided to work on this (and on our standard wiki, WikiThing, which grew out of it) because it was an ideal project for the entire group to tackle. This jumped it past collections. I still think this was a good idea, but there has certainly been some grumbling. We heard you. Collections is next on our list, with nothing new in between.


*So far we have only three data types—radio buttons (gender), long fields (book descriptions and author disambiguations) and short fields (everything else).
**Competitors who use it might want to stop asserting copyright over everything posted to their site. This was legally bogus already, but it certainly would conflict with a Creative Commons license... Incidentally, we haven't decided whether to go with CC-Attribution Share-and-Share-Alike or straight CC-Attribution (discussion here), but it's going to be one or the other.
***This particular one may happen very soon.
****And yes, we can discuss the whole radio-buttons-for-gender topic. See here, here. I'm of the opinion that two genders plus maybe "unknown" and "n/a" (for Nyarlathotep?) are the best you can get without consensus-splitting disagreement. You'll note we aren't including other potentially-contentious fields, like sexual orientation or religion.
*****In conception, Common Knowledge most closely resembles the Open Library Project, the Internet Archive's incipent effort to "wikify" the library catalog. Open Library is also a "fielded wiki," based on Aaron Schwartz's superior Infogami platform. You'll notice that we've mostly steered clear of the "traditional" cataloging fields that Open Library is starting from. We do cataloging differently, and we don't want to duplicate effort. Anyway, we're hoping they and others mash up the two data sets, and others.

Labels: , , , , ,

37 Comments:

Anonymous Anonymous said...

The worklinks are to the "forbidden" Athena server...

10/10/2007 5:17 AM  
Blogger Tim said...

Fixed.

10/10/2007 5:31 AM  
Anonymous Anonymous said...

Thanks. This is an awesome feature.

10/10/2007 6:14 AM  
Anonymous Anonymous said...

Small bug with the description field. When you write a 2 line book description and save it, and then reload the page the first line sticks to the top of the description box, followed by lots of white space in between and the second line hugs the bottom of the description box.

In two rows like so:

XXXXXXXXXXX



XXXXXXXXXXX

10/10/2007 6:25 AM  
Anonymous Anonymous said...

Also, recently updated book descriptions do not appear to be displaying for non-logged-in users.

10/10/2007 6:37 AM  
Anonymous Anonymous said...

"Go crazy, people.**"

At first I read this as "Go, crazy people". I think it might have been something you said in another post about your "insane" users. ;-)

Er, I need something even *more* addictive on LT?! Yikes!! Thanks for the fun playtime, though.

I'm always amazed by what the group "Tim and company" can come up with. Great work!

10/10/2007 8:35 AM  
Anonymous Anonymous said...

Ooh, I like this feature :)

The one problem I've noticed: the example you get when you click the Edit button is a bit confusing. For example, if you go to edit character names, the example makes it look like you should enter them like so: "[name]", "[name]", "[name]"
when as far as I can tell, you're actually, supposed to do it without the quotes, without the commas, and adding a new data line for each one.

Or maybe this is just me.

Anyway, it's just a minor problem; I figured it out quick enough.

10/10/2007 8:51 AM  
Blogger r.orrison said...

You mentioned in the blog post that Series information is a possible use... true, but I hope that doesn't preclude adding a proper sortable series field. (Unless there's some way to sort on a Common Knowledge field?)

10/10/2007 9:13 AM  
Anonymous Anonymous said...

First reaction, "Oh, BOY!!"

Second reaction: "Why does he do this when I'm leaving town and will likely be without internet access for a few days?"

Third reaction: "Well, at least I won't get caught in another time sink!"

Seriously, this is going to be almost as much fun as paste papers.

Hmm, or not. I can't seem to get the little pencil thingy to allow me to edit the fields on the author page.

10/10/2007 9:23 AM  
Anonymous Anonymous said...

I think that in some respects this is asking for trouble. To have people add serious book information like "abridged text" or "contains the following three works previously published separately" would be quite useful for us bibliography geeks. But where on earth do you stop with "character names" for Lord of the Rings? And what relevance does any of this literati stuff have to the oceans of non-fiction books out there, for which tags do most of the work already?

10/10/2007 10:26 AM  
Anonymous Anonymous said...

Dropping some acronyms: Any chance that this is available in RDF? Or that you have a SPARQL endpoint?

10/10/2007 12:06 PM  
Anonymous Anonymous said...

Okay, let's discuss the radio-buttons thing for gender.

"Unknown" is okay when the gender of an author really is unknown, and "n/a" works for a collaboration or for a corporate 'author'. But neither of these apply to the case of, say, Leslie Feinberg, who rather firmly identifies as "none of the above". Zir gender is well-known, and zie certainly *has* a gender -- it's just neither male nor female. Can you really construct an articulate rationale for being so exclusionary?

It may affect only a small number of authors -- but it's a *very* important issue for those it affects. Please reconsider.

10/10/2007 12:35 PM  
Anonymous Anonymous said...

It's not clear whether to list an award on a work page, or on the author's page. It would be optimal if awards listed on work pages could bubble up to the author page as well, to prevent the likelihood of (or need for) double-entry.

For an example of this confusion, take a look at the Booker Prize award page, where someone listed author John Banville the award winner instead of Banville's award-winning work -- http://www.librarything.com/commonknowledge/awards/Booker+Prize

10/10/2007 1:34 PM  
Blogger Soji Slade said...

Neat, I just added some award honors (Edgar, Shamus) and some characters.

10/10/2007 1:53 PM  
Blogger Tim said...

It would be optimal if awards listed on work pages could bubble up to the author page as well, to prevent the likelihood of (or need for) double-entry.

Yes, absolutely. In fact, that's the plan. So I think we should start discouraging author-level awards that were given for a book.

But neither of these apply to the case of, say, Leslie Feinberg, who rather firmly identifies as "none of the above". ... Can you really construct an articulate rationale for being so exclusionary?

No, I shall have to be inarticulate. The problem is that LT CK is designed for facts to which all can agree or reach an easy consensus. Opening the gender field up for free text is very handy when there's a single editor, particularly when it's the person in question. So you get sites which present a dozen or more genders, like Pownce.com, where members can choose to identify as "dude," "lady," "guy," "bloke," etc. That works less well when anyone can have their say. (Is Shakespeare a dude or a gentleman?) What I don't want is identity fights over dead authors and endless terminological fights, whether bisexual is to give way to pansexual to avoid binary connotations, and so forth.

As I discuss at some length in my talks, tags like GLBT and LGBT may seem like synonyms in a cataloging context, but the actual book-sets are rather different. It's fascinating to watch users choose their terms and create spaces of meaning for themselves. Capturing that dance of identity and meaning is what tags do best. Common Knowledge takes these many and varied worlds of meaning and pushes them through a cake nozzle. There's one meaning and someone gets to decide it. If you think it's you, you're probably statistically wrong.

Much the same is why LibraryThing has no space for religion or sexual identity. These are things we get to decide for ourselves. Letting others do it is a recipe for disaster. I myself have spent four years watching the Wikipedia community tackle the sexuality of Alexander the Great. Can they do it? The results are not encouraging. Most Wikipedians can't understand that ancient concepts are not modern ones, and everything is clouded by thick partisanship, so Wikipedia lurches violently from "gay" to "not gay," with all manner of nastiness generated in the lurching.

10/10/2007 2:14 PM  
Blogger Soji Slade said...

Education: I've put Colleges and majors in there. What exactly should be put in there? For example: George R.R. Martin attended Northwestern (added), and left with a B.S. and M.S. in Journalism (summa cum laude). Is the Northwestern part the only thing of interest?

10/10/2007 2:30 PM  
Anonymous Anonymous said...

There's some ambiguity in date entry formats.

Right now, the date entry help suggests dates like "12/30/2007", which uses American-style Month-Day-Year ordering, while much of the rest of the world uses Day-Month-Year. I'd suggest using Year-Month-Day, as it's much more unambiguous -- or alternatively, asking users if they meant "June 2, 1920" or "February 6, 1920"

Leading zeroes could be an issue. For example, the Kurt Vonnegut author page has the date of death listed as "04/11/2007", which links to the "04/11/2007" date, rather than the "4/11/2007" date. Normalizing these dates would be great.

10/10/2007 2:40 PM  
Blogger Languagehat said...

What anirvan said about dates.

". Zir gender is well-known, and zie certainly *has* a gender -- it's just neither male nor female. Can you really construct an articulate rationale for being so exclusionary? It may affect only a small number of authors -- but it's a *very* important issue for those it affects. Please reconsider.

I strongly disagree. I could announce that I wanted all references to me to be accompanied by hopping up and down on one foot, and this was very important to me, but too bad for me, nobody would do it. A tiny group of people do not get to throw monkey wrenches into social software to satisfy their ideological cravings.

And of course Leslie Feinberg "has a gender"; the fact that she doesn't like to be so categorized has no bearing on the issue. From the Wikipedia entry:

Leslie Feinberg is Jewish and was born female and today prefers gender-neutral pronouns "hir" and "ze". Feinberg writes: "I have shaped myself surgically and hormonally twice in my life, and I reserve the right to do it again.

That's all well and good, and she has the right to do whatever she likes with her body and to use whatever pronouns she chooses; she has no right to try to control the usage of the rest of humanity.

Also, words like "zir" are terminally dumb. English has perfectly good ways of being gender-neutral; both "they" and "he or she" work fine.

10/10/2007 3:16 PM  
Anonymous Anonymous said...

tim:

I agree that free-form text for the gender field would be decidedly suboptimal -- the ideal situation in my opinion would be a third radio button, for "Neither/Other".

languagehat:

While Feinberg was born female, there are certainly cases where even birth gender doesn't fall neatly into one of the two radio-button boxes, so you don't have that to fall back on. (Also, I didn't want to use "they" to refer to an individual of known gender, vs. one of unknown gender, so I chose the pronoun Feinberg prefers.)

Maybe everyone will have the good sense to just leave the gender field unchecked in these cases, but I'm not terribly optimistic.

10/10/2007 4:10 PM  
Blogger Barbara said...

Details aside ... this is fabulous. As a librarian, I'm chuffed as little mint balls to see all the extra kinds of information that could be searchable. (That "mint balls" phrase is one Stephen Booth used once, and it comes to me when I'm particularly pleased.) People in the publishing/book business would find agent and editor especially helpful information.

10/10/2007 4:29 PM  
Anonymous Anonymous said...

This is wonderful. It will keep us obsessives busy, and make LT a lot more useful as the default-page-you-link-to-when-you-mention-a-book.

10/10/2007 5:53 PM  
Blogger K.G. Schneider said...

Tim: leave your wife and marry me now. Ok, I'm a lesbian... in a fifteen-year relationship... just forget those details. We *belong* together.

Seriously, this is wicked good. I agree you need to do something about gender (neither/other is a good choice) and I'm sure more hairballs will land on the carpet.

Nevertheless, bravo.

10/10/2007 8:32 PM  
Anonymous Anonymous said...

I see how this would be very useful, if it was searchable, for those of us who work in libraries and get patrons who say the character's name was "David" or the story took place in "Cuba".
"Wonderful, wonderful".

10/10/2007 9:27 PM  
Blogger Joanna said...

I was wondering, can a user add new fields? I think it would be neat to note down what books the characters read inside the book, like Herodotus and Gravity's Rainbow in "American Gods."

10/10/2007 10:23 PM  
Blogger Unknown said...

Fantastic! Great features and awesome potential for data collection. Your interaction has encouraged people to complete work here that has gone unused at Worldcat. Kudos.

10/10/2007 10:30 PM  
Anonymous Anonymous said...

Would it be possible to mark information as spoiler and non-spoiler and give the option to view the spoiler or non-spoiler version of the knowledge?

For example, a certain SF novel may be about the colonisation of a planet with no intelligent life forms, but it is only revealed at the end of the novel that the planet in question is Earth. If the novel is new to me, I really don't want to see "Earth" in the list of places.

10/10/2007 10:50 PM  
Anonymous Anonymous said...

I haven't tried actually using this feature just yet, but it looks like it's going to be great.

It is definitely a brilliant idea.

10/11/2007 4:13 AM  
Blogger symac said...

As the anonymous said, I also think giving access to theses informations as rdf will be a very intersting thing, when the common knowledge base will be important !
Keep up the good work LT guys !

10/11/2007 5:52 AM  
Anonymous Anonymous said...

wow - thank you Tim and team. I'm quite new to librarything and I'm already hooked as I've never before been hooked to any other web project. Looking forward to playing around with the new feature.

At the moment it seems that the common knowledge fields are primarily designed for fiction. Are there any plans to expand them to cover non-fiction books as well?

10/11/2007 6:12 AM  
Anonymous Anonymous said...

What is the intended purpose of the "Publisher's editor" field in a book's common knowledge? I couldn't find any books that had an entry for this field. Is this for the employee (i.e., editor) who worked on the book at the publisher? That seems like a surprisingly specialized bit of trivia compared to the other common knowledge fields. So perhaps it's for something else?

And if it is for the book's editor, I assume that would be the acquiring editor, as opposed to the other "editors" who worked on the book---copy editors, managing editors, etc.

10/11/2007 11:03 AM  
Anonymous Anonymous said...

Needs a non-free form way for entering birth and death dates -- drop-down lists? -- (perhaps in addition to free form for early dates that don't fit well into that straitjacket. Otherwise, those of us on opposite sides of the Atlantic will continue to confuse each other.

10/11/2007 6:07 PM  
Anonymous Anonymous said...

As far as dates go you could go for the ISO standard format (which has the advantage of being equally strange for most North Americans and Europeans). However its big advantages are you can sort as text and partial dates (year or year and month) fit nicely eg:

1961-10-06 (= 6 October 1961)
1912-05 (= May 1912)
1936 (= 1936)

10/12/2007 6:22 PM  
Blogger Yvette Hoitink said...

Great new feature! However, the common knowledge doesn't show up on the translated pages.

Compare Dutch page for Queen of Sorcery by David Eddings with the English page for Queen of Sorcery by David Eddings and you will see that the English version has Common Knowledge while the Dutch version shows none. This means Dutch users could accidentally add the same information again, and don't profit from this useful new feature.

Could someone fix this please?

Yvette

10/13/2007 10:30 AM  
Blogger Tim said...

Thanks for the note. This was actually intended to be a feature, not a bug. Many of the fields are inherently language-specific—certainly book descriptions, but even city-names vary a lot (eg., Leghorn vs. Livorno, London, Londres, etc.). We are now reconsidering this, and will allow "forked" data as an opion, but not the only one.

10/13/2007 10:57 AM  
Anonymous Anonymous said...

Tim,

Has anyone else suggested adding "Original publication date" -- that is, the date on which the book was first issued, not the date of my particular edition -- to the Common Knowledge fields? This is the one piece of data that I dearly wish could be tracked in LT, and since it will be common to all users, Common Knowledge seems the perfect place for it!

Please?

10/19/2007 4:15 PM  
Blogger RoseHelene said...

What about cover artist? :)

I'd love to be able to add info 'bout that!!! Shinyyy!!

12/03/2007 8:20 PM  
Anonymous Anonymous said...

Bit new, but loving the site! Question re: CK. I like tracking errata in published works, and this is one way antiquarians differentiate editions.

(Modern example: on the front cover of the 2001 paperback edition of Copyright Plain & Simple," there is a typo. A major, very ironic, truly hilarious typo.)

What's an appropriate place for noting such identifying data? Is CK the place?

8/09/2008 6:28 PM  

Post a Comment

<< Home