How to create an aggregated feed

Hi. You don’t know me, but I’m here to try to help out with some of the technical aspects of science blog aggregation. I’m going to start by writing about how some bloggers might get together to set up a blog aggregation.

So: you are an independent blogger, and you want to aggregate your blog with some friends’ blogs, and then you want scienceblogging.org to aggregate that aggregation. What does that mean, and how do you go about it?

The people

The first step is to find a group of people who blog about similar topics at least some of the time.

You sometimes post about cognition, sometimes meta thoughts about science blogging, and sometimes personal ramblings, at ramblingscienceblogger.blogspot.com. Your friend Jane writes about neuroanatomy, addiction, and her young daughter at janesaddictionneuro.blogspot.com. Your friend Bob writes about behavior, neurotransmitters, and his dogs at bobsbehavior.wordpress.com. The three of you would like to create a “Brain and Behavior” aggregated feed. You agree that any posts in any of your three blogs tagged “neuroanatomy” or “behavior” should be included in the new aggregated feed. Posts that don’t have either of these tags won’t be included, although posts that have one or both of these tags and some other tags will be included.

For example, when Jane writes a post about the hippocampus, she tags it “hippocampus” and “neuroanatomy.” This post will be included in the aggregated feed. When she writes a post about her daughter, she tags it “kidblogging” only. It will not be included in the aggregated feed. Bob writes a post about a funny thing his dog did yesterday and tags it “ginger”; it is not included. Then Bob writes a post about how his dog's behavior during thunderstorms reminds him of a recent article he read about fear conditioning in rats, and he might tag that “ginger” and “behavior.” That post would be included.

My examples assume that all the bloggers in this group are independent bloggers, but of course they could just as easily be bloggers on a network of some sort.

The aggregation

You choose an aggregator service to manage your new aggregated feed. You tell this aggregator service to aggregate the following feeds:

In other words, you are telling the aggregator service to pull in RSS feeds for Jane’s, Bob’s, and your blogs, but only the posts with the tags that you care about. You must include a separate URL for each tag and each feed — so for two blogs and two tags, you include four URLs. For three blogs and two tags, you include six URLs, and so on. The patterns demonstrated above will work for all blogspot and wordpress blogs.

The aggregator service then provides you with a new RSS feed which contains all the posts from Jane’s, Bob’s, and your blogs tagged “neuroanatomy” or “behavior.” You publicize that RSS feed however you want -- you may just blog about it, or you may create a web page as a home site for it. You definitely let scienceblogging.org know to aggregate it (contact@scienceblogging.org).

If you later decide that you want to aggregate your meta science blogging thoughts with some other people who also like to write about science blogging in general, there's nothing to stop you from having a second aggregated feed on an entirely different topic, with entirely different people, using your same blog. Just use different tags. In fact, one post could show up in both aggregated feeds, if it used the right tags.

What is a tag?

Tagging is a way of noting the subject matter of a particular blog entry. Most blogging platforms will provide a way for you to tag each blog entry.

What tags should I use?

That’s for you and your co-bloggers to decide. It would be nice if bloggers started coming to a consensus on tag names for particular topics, so that aggregating different blogs by tag was easier. We’ll see if this happens.

What if my blog is not a Blogspot blog or a WordPress blog? How do I find out what the right format is for a feed for a particular tag?

Try doing a web search for “tag RSS” and the name of your blogging platform. Or comment here and I will try to help you figure it out.

What’s the point of this, anyway?

Subject aggregations are convenient for readers — a way to get an overview of blog posts by topic, rather than by author. They are also a way to build community, with several different authors working together to generate related content.

What aggregator services are out there and which are the best ones to use?

There seem to be several to choose from, but I don’t have experience with them to know which are better. If you have found particular ones that you like or don’t like, comment here and let people know. Yahoo Pipes seems to be widely used, but there are others.


  1. Argh, you didn't list any aggregator services other than Yahoo pipes??? This is always where I get hung up. It seems a lot of early aggregator services bailed because feed aggregation is a big bandwidth hog for them. Can someone please list at least 5 aggregator services to choose from? This post isn't very useful without mentioning specific aggregator services to choose from.

  2. Some very useful tips! Thanks for posting this. Good point about the tags to weed out irrelevancies and teh kittehs.

  3. Combining the label feeds is a nice idea.

    FWIW, I've done the grunt work with the Yahoo Pipes. You'll need a Yahoo account, but once you have one you can simply Clone this pipe: http://pipes.yahoo.com/fieldofscience/full

    With your Clone, go to Edit Source, then change the feeds in the Feed Fetch module to yours, and in the Simple Math module put the number of feeds you are combining.

    Then if you want to get totally redundant, burn it too: http://feeds.feedburner.com/FoSCombinedFeed

    Hope that's helpful.

  4. I'll start doing some research about what aggregators are out there and try to put together a separate post about that. I think it would have been a little too much information for this one. I would love to get feedback from others about what aggregators work and don't work for them so that I can include those tips.