Tags and hiccups

This weekend we made a number of important structural changes to how LibraryThing's 34.8 million tags are stored in the database. (For database heads, tags are now "fully normalized.")

The immediate upside is that tags can now be up to 255 characters long. It will also allow us to improve some features, such tag editing.

The downside is that the changes have hurt performance. Certainly the site is running very "hot," forcing us to choose between running slow and pulling back service. Right now we're doing the latter, redirecting most non-member traffic to the home page to sign in or sign up. This cuts back the large proportion of our traffic that is bot-related, but we can't run this way for too long.

It's unclear if the change is itself to blame or the loss of various tag-related caches, which need to be built up afresh with the new structures. There also appear to be some places that are hurting more than others, which code can perhaps be re-written. We're going to be looking carefully at what we can do, and deciding whether we need to make additional changes, or pull back the ones we made.

Thanks for your patience. The next few days may see occasional slowness or downtimes. With the four of us on it, however, we hope to minimize problems and solve this to everyone's satisfaction.