Happy Little Monoliths is available for pre-order.

As you can probably guess, yes, in Python. Writing an aggregator like MXNA or Full as a Goog isn't parkwalkian at all. Once you're out of the nightmare of parsing the multiple RSS variants and Atom, next you have to order items by date of publication. This means you'll have to write code to parse the different date formats used in RSS and Atom, which includes dealing with timezones, so you can finally convert them to Unix Time and make them sortable.

Fortunately, someone has spent time to provide us with a fast and stable library that can parse every known syndication format on earth seamlessly. The Universal Feed Parser is an absolute must for any application that deals heavily with syndication formats.

If you were to build an aggregator, you would probably end up defining a class to represent entries and whatever additional information is associated with them. But first, there needs to be some sort of storage mechanism for the feed URLs. In this example, I'll use a simple text file. The code starts with the importing of the require modules, the parsing of the feeds' URLs and the definition of an array to hold the items once we parse them.

import time
import feedparser
sourceList = open('feeds.txt').readlines()
postList = []

Next, we define the Entry class. It will act as an wrapper for the entry object the Universal Feed Parser returns. The modified_parsed property contains the entry date in a tuple of nine elements, where the first six are the year, month, day, hour, minute and second. This tuple can be converted to Unix Epoch with the built-in method time.mktime():

class Entry:
    def __init__(self, data, blog):
        self.blog = blog
        self.title = data.title
        self.date = time.mktime(data.modified_parsed)
        self.link = data.link
    def __cmp__(self, other):
        return other.date - self.date

The __cmp__ method defines the standard comparision behavior of the class (you could also override specifically == behavior by defining a __eq__ method, but __cmp__ works just the same). Once we get an array with Entry instances and call sort(), the __cmp__ method will be used to define the order.

Now comes the part where the UFP saves us 200 lines of code. Since we want to show entries ordered by date, it's prudent to at least verify if the entry actually includes a date. Further measures would include checking if the date is within the current century. Or, you could just check for the bozo bit and refuse invalid feeds altogether.

for uri in sourceList:
    xml = feedparser.parse(uri.strip())
    blog = xml.feed.title
    for e in xml.entries[:10]:
        if not e.has_key('modified_parsed'):
        postList.append(Entry(e, blog))


Finally, we output the data:

print('Content-type: text/html\n')
print('<ul style="font-family: monospace;">')

for post in postList[:20]: # last 20 items
    date = time.gmtime(post.date)
    date = time.strftime('%Y-%m-%d %H:%M:%S', date)
    item = '\t<li>[%s] <a href=\"%s\">%s</a> (%s)</li>'
    print(item % (date, post.link, post.title, post.blog))

If you want something as complete as Full as a Goog, you might want to check PlanetPlanet, which also uses the Universal Feed Parser. But the simplicity of this example should invite you to improve and tweak it yourself. Let me know if you make something interesting out of it. Enjoy.

Follow me on X and GitHub.

Go to the index page.