Building a better network: Identifying trends/posts of interest

When you build a network of blogging networks, the problem quickly escalates from "how do I collect as much data as possible?" to "how do I manage all this data?"

Take a look at the Science Blogging Aggregated home page. There's lots of great stuff there -- too much for the typical reader to handle. Even if you visit several times a day, the information rushes by too quickly to discern any trends, and it's hard to know which posts are really well thought out and which are just one-off posts that hardly merit your attention at all.

We talked yesterday about one way of sorting through the data -- tags. However, this method alone probably won't satisfy all users. A person might be interested in all posts tagged "psychology," but they might just want to see the highlights of what's going on in other fields, and tagging won't help them identify the most interesting, thoughtful posts.

We see at least four possible ways of sifting through the posts to find the most interesting ones.

1. Crowd-sourced ranking. Users rate or recommend posts they like, so others can sort by rating or number of recommendations to find the posts they want to see. An advantage is that there is no central authority telling readers what to like. A disadvantage is that blogs that are already very popular are perhaps most likely to be recommended, so this system might not help users identify up-and-coming blogs that are very high quality.

2. Self-promotion. Bloggers could promote a small number of their posts, indicating these are their best work (one per week? one per month?). This overcomes the "up-and-comer" problem, but a blogger whose work is mediocre could exploit the system by promoting posts that aren't very interesting or useful to others.

3. Active curation. Editors could be chosen for each field (physics, biology, etc.) and actively promote one or two posts each day. That way readers would know that an expert has read all the posts on a topic and selected the most interesting or relevant. Advantages are that editors may be able to identify trends that more automated systems don't catch, and that editors may be less swayed by the most popular blogs. Disadvantages include possible bias of editors, and variable editor quality. It would also require coming up with a system for selecting editors. Would a central person be in charge of that, or would we need to create some sort of a system for nominating/voting for editors?

4. Social networking. We could create a truly social network where users are only shown the "likes" of their friends. However, this requires a significant programming effort, and people are reluctant to join new social networks when they already participate actively in one or more networks. I think we might be better off using the social features of other networks, rather than building our own. If we could make it really easy for people to post their "likes" to Twitter and Facebook, then we could leverage those networks to perform the social function.

There is, of course, no reason that we shouldn't do all of these things over the long run. But we have limited resources. Which of these approaches is most useful? Are there any other approaches that would work better? Do you have any specific suggestions for how to implement any of these ideas? Let us know in the comments.


Building a better aggregator: Goals, Tagging

The ScienceBlogging site you see now was always intended to be a temporary solution. What we really need is a site that not only aggregates blog posts, but also allows users to classify them, search them, highlight their favorites, point their friends to them, and do many other things we haven't even imagined yet.

Behind the scenes, Bora, Anton, Jessica, Mark, and I have been discussing how to do that, but we realized that limiting the discussion to just ourselves is depriving us of a valuable resource: The people who'll be using and contributing to the new site.

So, over the next few days, I'll be offering some thoughts about how to proceed and inviting your comments. Our plan is to have at least a partially functional, working prototype of the new site by the ScienceOnline conference in January 2011. Let's get that started right now by discussing the goals for the site.

Here are the goals we came up with for the site:

  • To be a central site where scientists, media, other experts, and laypeople see what scientific topics are being discussed on blogs, in real time

  • To be a resource for locating past discussions

  • To promote science blogging and other online discussion of science

  • To promote scientific accuracy and avoid pseudoscience and crackpottery

  • To be encyclopedic and inclusive

  • To be searchable and filterable

  • To have a system (or multiple systems) for highlighting discussions and posts that are especially topical / high quality

  • To have a means of removing or hiding posts that are not scientific (e.g. vacation photos, political rants unrelated to science, etc.)

  • To be multilingual

  • To be open source / open access

Should anything be added, changed, or removed?

One of the first considerations will be how to keep track of all this information, and a huge key to that will be classifying it. That's why we think it will be essential to have a unified tagging system in place. If bloggers don’t select their primary tags from a central list, then it will be difficult for users to find posts on the topics that interest them. On the other hand, if bloggers must visit our site to choose primary categories, then usage will suffer. We can allow bloggers to set default tags for their posts using their registration page, but there should be some way to override those settings for individual posts, still using our list of preferred tags.

Could we create a WordPress plugin for this? Or adapt an existing plugin? What about other blogging platforms? What about templates that don’t support tags? One possibility is using a bookmarklet, which would be platform neutral but not ideal. Any other ideas on how to implement a tagging system?

That's just the first bit -- there's a lot more to discuss, but we thought this would be a good way to get the conversation started. So please, let us know what you think in the comments.


Finding people to aggregate with

Aggregating blogs is not technically difficult. (See previous posts: How to create an aggregated feed and Feed aggregator choices.) Finding other people to aggregate with can be a challenge, however. Feel free to comment here in order to find other people to aggregate with. One way to start is to suggest a topic (neuroscience? medicine? new bloggers? meta-science blogging? students? faculty? physics? astronomy? anthropology?) that you blog about, and ask if others want to aggregate posts on that topic.

Remember, if you are an independent blogger who wants to be listed on, putting together an aggregated feed is currently the only way.

Also, remember that you can choose to aggregate only selected posts if you want, using tags as filters. Or you could aggregate your entire blog.