Sunday, October 26, 2008

Database maintenance

I truncated the database and added a few more indexes to RSSmeme that will hopefully improve stability. Some people were using the API in ways that I didn't expect and adding the indexes to the database without truncating it first would have taken hours/days to finish; since RSSmeme isn't mission critical I decided to start fresh! RSSmeme should "auto discover" your Google Reader shared feed all over again (the stats page shows we've already found 1200 in the last hour).

Wednesday, July 30, 2008

Stop the dog-pile: freshen your cache

It's no question that RSSmeme has been gaining in popularity. With increased visitors comes increased load and with that comes increased wait times. RSSmeme caches the data that comes out of database for several minutes to keep things snappy but let's look at what happens when that cache gets old. Under the old, simplistic system:
  1. The home page cache gets old and stale.
  2. 50 visitors request the home page at the same exact time.
  3. The server gets clogged running the exact same query for all 50 visitors.
  4. Meanwhile more visitors hit other pages and nobody ever gets results because the server is overloaded; the dog-pile effect.
Under the new, system (using MintCache):
  1. The home page cache gets old.
  2. 50 visitors request the home page at the same exact time.
  3. The 1st visitor triggers a database query to get fresh data. The other 49 get the now slightly stale data. Only 1 call has been made to the database.
  4. The cache is updated with the fresh data that was sent to the 1st visitor and everyone is happy.
It's slightly more complicated but the results are already promising. Hopefully this keeps RSSmeme speedy.

Friday, July 11, 2008

Search RSS feeds and JSON disabled

RSSmeme has been slowing down over the last few days and my brief analysis looks like it's due to a lot of people using the search RSS feeds and API. Unfortunatly I've had to disable the search parameter for any format other than the standard HTML that RSSmeme displays.

I'll hopefully be able to restore this soon but in the meantime all of you using complicated Yahoo Pipes filters with RSSmeme might not be seeing results that you'd expect. Sorry.

Update: The personalize feature has also been disabled while I test things out.

Update: RSS and JSON re-enabled. Personalized feature still disabled.

Monday, June 30, 2008

10 days and 10000 new users later

Exactly 10 days ago I wrote about RSSmeme crossing the 20000 user barrier. Today RSSmeme has over 30000 users!

Digg on RSSmeme

Now that RSSmeme is being fed data from Digg it only makes sense to let users Digg right back. Next to the "share" link is a new "digg" link!

Friday, June 27, 2008

Video: New Customize Tab

I added a new tab to RSSmeme this morning; here's how to use it:

RSSmeme Customize Tab from Benjamin Golub on Vimeo.

The customize form will get more filters over time; this is just the beginning!

Thursday, June 26, 2008

Video: How To Use RSSmeme To Find Popular Stories

I thought the last video went well so let's keep it going:

RSSmeme Popular Stories from Benjamin Golub on Vimeo.

(Sorry that the detail is gone; I'm still learning this screencast business, it looked great before uploading to Vimeo)

Wednesday, June 25, 2008

How Top Sharers Works

I think it's best explained by talking it through so here goes nothing:

Monday, June 23, 2008

RSSmeme Expands Beyond Google Reader; 10x More Ways to Share

RSSmeme isn't a Google Reader only service anymore. Last week I wrote about leveraging the FriendFeed API to make RSSmeme gather data even faster. What I didn't tell you was the second part of that development would enable RSSmeme to support every service that FriendFeed supports.

For the last few days RSSmeme has been counting all of your Google Reader shares in addition to stories you share on Delicious, Digg, Diigo, FriendFeed, Google Shared Stuff, Magnolia, Mister Wong, Mixx, Netvibes (which our friends at ReadBurner support), and StumbleUpon. The only service missing from that list is reddit and I hope to be able to add them ASAP.

While I love Google Reader I know that it isn't perfect for everyone. RSSmeme is about "finding signal in the noise"; now that signal can come from anywhere! Looking at the leaderboard it looks like Digg Boss is a very prolific digger; digging many stories that make it to the top of RSSmeme. I'm happy to report that his diggs now count as a vote in RSSmeme!

And it requires absolutely zero registration to get started. If you have a public FriendFeed account then RSSmeme is already counting your shares! If you share the same story using multiple services it will only count the share once to keep things democratic.

You may have noticed stories reaching the RSSmeme front page without any author, source, or excerpt. This is because the story wasn't (or hasn't yet been) shared through Google Reader. Once someone does share it via Google Reader the extra meta data will appear.

So share the way you want to share; and you can bet that as FriendFeed adds more services so will RSSmeme!

Friday, June 20, 2008

Over 20000 Users!

RSSmeme is now tracking over 20000 users. A recent update (you'll hear more about it later) was responsible for this large jump in users and it will only continue to grow.

The more users in RSSmeme the better and quicker it is at "finding signal in the noise". Happy sharing!

Full Content Disabled

RSSmeme received an email yesterday afternoon threatening legal actions unless I remove a blogger's content from the site. I always believe in "better to be safe than sorry" so I removed the content and have disabled full content republishing on RSSmeme. Most people didn't even know that you could get full content on RSSmeme since the default was to show extracts but yes it was possible and yes there is a Google ad.

I am not saying that full content is gone for good. I need to do some research and discover if I can renable it; perhaps if I remove ads on pages that serve full content? If you know anything about this please reply in the comments or via email:

I wanted to thank the FriendFeed community for being so supportive. I was actually out at a jazz festival when my phone buzzed to tell me I had an email and so I posted to FriendFeed using FF To Go not thinking it would stir things up this much.

A huge thank you to Louis Gray who immediately made himself available via phone to get the full story and write up a very supportive post on the matter.

Thursday, June 19, 2008

Recent Comments

I've added a recent comments box to the sidebar courtesy of Disqus. This is something I should have done a long time ago; they make it so easy!

I've loved using Disqus for RSSmeme (they also power comments on this blog). They even featured RSSmeme on their blog when I whipped together the dynamic slide down Disqus comments you see in RSSmeme.

Wednesday, June 18, 2008

Thank You Chris

Chris Wetherell, the creator of Google Reader is graduating from Google. I wanted to take a moment to thank Chris and the rest of the Reader team (especially Mihai who's been very helpful) for all of their hard work.

Without Google Reader there would be obviously be no RSSmeme but more personally I'd be completely lost on the web. I practically live inside Google Reader (it and Gmail are permanently open in Firefox).

Good luck Chris; although if anyone doesn't need luck to succeed it looks like it's you!

Tuesday, June 17, 2008

An Official Blog

RSSmeme now has an official blog. It's about time I stop using my personal blog for RSSmeme updates and give it something more permanent. Please subscribe to the RSS feed to receive all of the updates.