Monday, May 07, 2007

Subjects get faster; the rest will follow

Everything on the web is better if it's faster. Slow pages are a silent killer.

So we're working to speed thing up. We've long done "situational" caching. But our growth is relentless—we'll hit 200,000 registered members today—and we've had no good, generalized solution. We've recently been working on two solutions, for database and page-level caching. Together they should speed up certain cacheable pages, like works, authors and tags. The more resources we can free, the faster the uncacheable pages, like Talk, will become as well.

So far, only subject pages are being cached, eg.,
Subject pages were a big problem. The worst took a minute to load. When Google's "spider" program went at them, with one request/second, the servers would sweat. Subject pages are now cached whenever someone hits a page, and stays so for at least week.

Subjects are a test. There are some kinks to work out. (For example, changing the non-English translations doesn't immediately clear all affected pages.) Once we get where we want, we'll roll it out page-caching wherever we can use it. Query caching will follow.

4 Comments:

Blogger Felius said...

Geek question time:

Have you written your own caching system for this, or are you using an "off the shelf" PHP caching layer, or are you using squid or similar in front of LT with modifications to "cheaply" return 304's for certain pages?

Just curious about the implementation :)

5/07/2007 2:39 AM  
Anonymous Anonymous said...

So it's possible that the subjects pages I see will be a week out of date?

5/07/2007 2:22 PM  
Blogger Tim said...

Well, actually they are already. Underneath, it's hitting time-slices of the subject->work data. Figuring out all the works for a subject and then how tags and subjects relate to each other can't be done on the fly. Its the same math as recommendations, and they scale linearly at best.

5/07/2007 2:32 PM  
Blogger jouni miettunen said...

Howabout other speed improvements? For example Multi Selection would help (me :) a lot to enter my books, manage tags etc!

Multiselection meaning that I could choose 10 books and add "my book" tag for all at the same time. Also possibility to add all books by same author etc.

--jouni

5/24/2007 6:27 AM  

Post a Comment

<< Home