Simple Update Protocol: Fetch updates from feeds faster

When you add a web site like Flickr or Google Reader to FriendFeed, FriendFeed's servers constantly download your feed from the service to get your updates as quickly as possible. FriendFeed's user base has grown quite a bit since launch, and our servers now download millions of feeds from over 43 services every hour.

One of the limitations of this approach is that it is difficult to get updates from services quickly without FriendFeed's crawler overloading other sites' servers with update checks. Gary Burd and I have thought quite a bit about ways we could augment existing feed formats like Atom and RSS to make fetching updates faster and more efficient. Our proposal, which we have named Simple Update Protocol, or SUP, is below. You can read more details and check out sample code on Google Code. Discuss the proposal in the SUP FriendFeed room.

SUP is just a proposal at this stage. We are eager to get feedback and ideas, and we expect to update the protocol based on feedback over the next few months.

Simple Update Protocol

SUP (Simple Update Protocol) is a simple and compact "ping feed" that web services can produce in order to alert the consumers of their feeds when a feed has been updated. This reduces update latency and improves efficiency by eliminating the need for frequent polling.

Benefits include:

  • Simple to implement. Most sites can add support with only few lines of code if their database already stores timestamps.
  • Works over HTTP, so it's very easy to publish and consume.
  • Cacheable. A SUP feed can be generated by a cron job and served from a static text file or from memcached.
  • Compact. Updates can be about 21 bytes each. (8 bytes with gzip encoding)
  • Does not expose usernames or secret feed urls (such as Google Reader Shared Items feeds)

SUP is designed to be especially easy for feed publishers to create. It's not ideal for small feed consumers because they will only be interested in a tiny fraction of the updates. However, intermediate services such as Gnip or others could easily consume a SUP feed and convert it into a subscribe/push model using XMPP or HTTP callbacks.

Sites wishing to produce a SUP feed must do two things:

  • Add a special <link> tag to their SUP enabled Atom or RSS feeds. This <link> tag includes the feed's SUP-ID and the URL of the appropriate SUP feed.
  • Generate a SUP feed which lists the SUP-IDs of all recently updated feeds.

Feed consumers can add SUP support by:

  • Storing the SUP-IDs of the Atom/RSS feeds they consume.
  • Watching for those SUP-IDs in their associated SUP feeds.

By using SUP-IDs instead of feed urls, we avoid having to expose the feed url, avoid URL canonicalization issues, and produce a more compact update feed (because SUP-IDs can be a database id or some other short token assigned by the service).

Because it is still possible to miss updates due to server errors or other malfunctions, SUP does not completely eliminate the need for polling. However, when using SUP, feed consumers can reduce polling frequency while simultaneously reducing update latency. For example, if a site such as FriendFeed switched from polling feeds every 30 minutes to polling every 300 minutes (5 hours), and also monitored the appropriate SUP feed every 3 minutes, the total amount of feed polling would be reduced by about 90%, and new updates would typically appear 10 times as fast.

You can see SUP <link> elements in FriendFeed's Atom feeds (e.g.,, and you can see FriendFeed's SUP feed at

Update: Several people have asked how using SUP compares with using HTTP If-Modified-Since headers. The two features are complementary. With SUP, feed consumers can monitor thousands of feeds with a single HTTP request (to fetch the latest SUP document) instead of having to request each feed individually. For example, each user's feed on FriendFeed has a unique SUP-ID (mine is "53924729"), but all of the feeds point to a single SUP URL, Therefore, it's possible to watch for activity on thousands of separate FriendFeed URLs by polling just one URL, If my SUP-ID appears in that SUP document, then you know that my feed has updated and it's time to fetch a new copy. This is substantially more efficient than polling each of those thousands of URLs individually.

Embed FriendFeed in your blog

We just launched a new set of customizable FriendFeed widgets for spicing up your blog or web site. There are several ways to embed FriendFeed, and you can pick as many of them as you want.

FriendFeed Badge

Create a customized FriendFeed badge to show off the services you use and your recent FriendFeed activity.

FriendFeed Feed Widget

Embed your feed or a room's feed on your blog or website with the feed widget.

"Share on FriendFeed" link

Add links to to let your readers share content from your site on their FriendFeed. We have custom code for WordPress and Blogger to get you started.

FriendFeed Chiclet

The FriendFeed chiclet is the most compact way to link your site's visitors to your FriendFeed.

Check them all out at As always, let us know what you think in the FriendFeed Feedback room.

Preview the new FriendFeed design

We are happy to announce a new design for FriendFeed that makes it easier to read your feed and easier to get updates from the people you care about most.

We are launching the new design today in "beta" to get feedback from all of you before we flip the switch to make it the primary FriendFeed interface. You can check it out at, and share your thoughts in the FriendFeed Beta feedback room.

Some of our favorite new features are described below.

Friend lists


Friend lists enable you to organize your friends into groups. With friend lists, you can get updates from your family separately from your coworkers, or you can add an acquaintance to a list and remove them from your home feed.

You can also get best of day and best of week summaries for every friend list, just like you can for your home feed. A "best of" page for a friend list shows you the most popular entries among people in the list.

Create a friend list with the "New list" link on the right hand side of your home feed.

Photo posting


We have enhanced our share box quite a bit, and you can now upload photos with your posts. Our photo uploader lets you select and upload multiple files at once with no additional downloads required.

To upload a photo, type something in the share box at the top of your home feed. The photo upload link will appear when you start using the share box.

Quick navigation


The most visually prominent new feature is the sidebar on the right hand side of the page. The new sidebar is designed to give you one-click access to the feeds on FriendFeed you care about most.

Now your favorite FriendFeed rooms are one click away from any page, and your closest friends are one click away with friend lists.

See other people's home feeds


You can now see a feed of a person and all of their subscriptions. This new feature is a great way to show your uninitiated friends what your FriendFeed experience is like, and it is a great way to find interesting people you haven't subscribed to yet.

You can find other people's home feeds by clicking a tab on their profile page. For example, here's my FriendFeed.

You can use all these new features at Please send us feedback in the FriendFeed Beta feedback room, and let us know what you think!