Current Word

Monday, February 13th, 2006

Throwing a Feed Reader Together in a Weekend

I’ve been interested in developing a client based feed reader for some time now. Current readers (both web and client based) are pretty much straight text readers. I think the whole River of News concept works, but it doesn’t go far enough. One of the beautiful things about RSS and Atom, is that they are both based on XML, and so they are capable of carrying within them a great deal of structured data. Most of the data flowing around the blogsphere today is simple text, or HTML markup – some images, and maybe a few links. But I think this is going to change over time. Some efforts toward this have already been made, good examples are the iTunes extension, Yahoo!’s media extensions, Microsoft’s Simple List Extension, and so on. I particularly like where the the guys at Structured Blogging are going – they are working on embedding micro content (basically XHTML microformats) into feeds. This opens up a whole new landscape for RSS. On top of basic text based communications additional meta data can be embedded that smart readers can extract out of feeds and and.. well, do something with it. For example there are microformats for contact information, calendar events, presence data, Geo-location data, and payment information just to name a few. Add to this media extensions which bring with them rich sets of meta data about the media they carry, like album, artist, and publishing data. The problem is, so far, very few readers make use of this data in constructive ways. Most readers that look for some of this data are highly targeted, like iTunes, they don’t manage structured data they aren’t familiar with. So my interest in developing a reader was to start looking into ways of pulling all of this information out, managing it, categorizing it, downloading it, and storing it in the most appropriate ways.

Here’s a good example: I recently updated my IM address, which I posted to my blog in clear text. My friends and family need to read the post and manually update their the various contact applications they make use of. Wouldn’t it be great if their reader simply pulled this information out and silently injected it into their address book without them having to do anything? Or maybe, if they didn’t know me that well, or maybe weren’t sure if they wanted my information handled this way, the default action would be something slightly different – the reader could extract this information and organize it so they could look it over (along with all the other contact information they had accumulated) at a later time. The process of injecting the information they wished to keep would be automated as well. If every reader out there supported such functionality, (or if address books supported RSS consumption!) we wouldn’t have a need for services like Plaxo. Plaxo, after all, is PUSH, and we all know how successful PUSH is. (It’s not) Historically PUSH services have been replaced with user controllable PULL services with great success.

Today a lot of this meta data is flowing through email in the form of email messages sent around to friends or coworkers. Email suffers from a number of drawbacks, but probably the biggest drawback is that email isn’t structured. RSS is, by design. Almost any small (or even fairly large) structured dataset can be encapsulated safely within an XML document. So although today, we spend most of our time in readers perusing a River of News, tomorrow, It may be quite different. Which brings me back to my interest in developing my own reader.

So what took me so long you ask? Well, when I first started looking at the problem, I was faced with one very daunting task – developing an RSS and Atom feed parser and feed store. Yuck. It’s kept me thinking about the project for over six months now. Thankfully, Microsoft recently released a beta of IE 7.0, and tagging along with this excellent browser, is the beta of Microsoft’s RSS platform – a full blown RSS and Atom parser and feed store. Yeah! So this weekend, I decided to dive into the project with two incredibly useful tools at my disposal – the RSS platform and Micrsoft Visual Studio 2005. I got a heck of a lot farther than I ever expected. I decided the best place to start was to develop a basic desktop reader, after which I can start working on some of the extended functionality I have in mind. At this point, after about a day and half, I’ve got a reader that sports the following features:

- display all feeds and folder in a hierarchical tree view
- options for updating individual feeds, feeds within a folder, or all feeds
- add, delete, rename and reorganize feeds and feed folders
- display feed properties like link, feed url, copyright, etc..
- display all entries within a single feed in a list
- indicate new vs. read posts, and be able to set the read / unread property on entries
- display feed entries in a nice browser view for reading
- a newspaper reading style where unread feeds are displayed at once in the reading pane
- import OPML functionality through a nice wizard, with extended options for setting update times, and removing duplicates

The result, looks like this (click for a bigger view):

It’s a little rough UI wise at this point, but not bad for a couple days of work. Next weekend I’m going to clean up the fonts and user interface, and fill out any missing functionality. I’m also going to do some custom drawing work on the upper toolbars area, in an effort to make it look more like some of the Vista apps coming out. (For example, I’d like a master set of forward / back navigation buttons like Internet Explorer.) The next step will be to implement a new right hand panel view, which will actually be three views, activated through a set of large custom tabs on the toolbar area. These three areas will be Feeds, Web Browsing, and Content. Here are some conceptual wire frames I drew up six months ago when I first started thinking about this project:

---

---

Having spent the weekend working with Microsoft’s Beta RSS Platform, I’ve found some of the good, and some of the bad. The good, I’m using, so no point in going into that. I’d like to make some suggestions in the hopes somebody from MS comes along and finds this. They might find it useful:

1) IFeedsManager has a method GetFeed where you pass the path of the IFeed. It might be nice to have a GetFeedItem where you pass in IFeedItem id and get back the item.

2) The IFeed interface seems to be missing a categories enum where I can look at the tags for the post. It seems odd I will have to parse the raw XML to find the categories.

3) I’d appreciate a Type property exposing the type of the feed. (e.g. RSS 2.0, Atom)

4) IFeedEnum would benefit from some more advanced filtering properties. Before walking across the enum with foreach, it would be helpful if I could set some Properties on how the list was returned. The Xml property has this to some degree, but I think they could take it further. I’d like to see fine grained control over sorting and filtering. Sort newest to oldest, oldest to newest, and by alphabetical based on title. Filter unread, read, and newer than a specific date (facilitates lists from last week, last two weeks, last month).

5) I’d like a better way to easily get access to extension elements within a feed item.

6) The purging of items should have more fine grained control. Period of time as well as count.

7) The GetFeed method on IFeedFolder accepts a name as it’s parameter. I’d prefer this to be a url to the feed since names of feed can be changed by the user outside of the application.

8) An event when a feed item’s read state changes would be helpful.

All in all though, the MS feed platform was a breeze to work with, and thanks to the rapid prototyping capabilities of Visual Studio 2005, I got a heck of a lot farther than I ever expected. Stay tuned!