Monday, January 29, 2007

HTTP Conditional Get for RSS Hackers

What is a conditional get?

A full-length RSS feed is above 24,000 bytes long. It probably gets updated on average twice a day, but given the current tools, people still download the whole thing every hour to see if it's changed yet. This is obviously a waste of bandwidth. What they really should do, is first ask whether it's changed or not, and only download it if it has.

The people who invented HTTP came up with something even better. HTTP allows you to say to a server in a single query: “If this document has changed since I last looked at it, give me the new version. If it hasn't just tell me it hasn't changed and give me nothing.” This mechanism is called “Conditional GET”, and it would reduce 90% of those significant 24,000 byte queries into really trivial 200 byte queries.
Client implementation

The mechanism for performing a conditional get has changed slightly between HTTP versions 1.0 and 1.1. Like many things that changed between 1.0 and 1.1, you really have to do both to make sure you're satisfying everybody.

When you receive the RSS file from the webserver, check the response header for two fields: Last-Modified and ETag. You don't have to care what is in these headers, you just have to store them somewhere with the RSS file.

Next time you request the RSS file, include two headers in your request.. Your If-Modified-Since header should contain the value you snagged from the Last-Modified header earlier. The If-None-Match header should contain the value you snagged from the ETag header.

Read Whole Tutorial @ fishBowl

0 Comments: