Ben Hughes

I promised myself that I wouldn’t be drawn into blogging or tweeting about Twitter, but some of the comments flying around about #SXSW “information overload” highlighting the failure of Twitter to scale have irked me somewhat.

This is a basic problem in any information management system: you need to adapt how informaton is managed based on the volume and other characteristics of the data. And it’s not necessarily something that can be automated, because information categorisation happens to be one of those things that humans are very good at.

If I ran a phpBB forum with a single forum for all posts, and saw a sudden increase in posts that made following the content difficult, I wouldn’t run around screaming about the inability of forum software to scale. I’d just split  the main forum into several sub-forums, based around the main topics.

If I had a single pile of papers on my desk and found it’d grown to a size where I could no longer find things quickly, rather than declaring that piles of paper were dead, I’d split the pile into several categorised smaller piles.

This is nothing new.

The issue seems to be that people have forgotten how the Twitter syntax has developed over time. @Replies (@<username>), retweets (RT<message>) and hashtags (#<topic>) were all conventions started by users, which virally became popular, and then started being supported by the various Twitter  clients. This is a demonstration of the power of twitter: it’s minialism means that people can use it in very different ways, and the community can extend and adapt it without needing to convince a central authority.

So calling for ‘new analytical infrastructure’, to me, is forgetting the essence of Twitter as a community driven medium.

So as a member of the wider Twitter community, what are my ideas for a solution? First you have to look at the problem a little closer. Hashtags are used in two ways: to allow your followers to easily access the wider context of your tweet, and to allow people following the wider context to find your contribution. The latter is a great information mining resource, and this is where the complaints around SXSW are stemming from.

I think the most obvious solution is similar to the examples above: either use multiple hastags to multi-categorise tweets, or use hierarchical hashtags to sub-categorise them. For a large event like SXSW, sub-tagging would allow much better filtering. For example, using #sxsw:social for social events. 

Why would someone be motivated to use up more valuable tweet space with extended tags? For the same reason they use hashtags now: to improve the quality and usefulness of their stream to followers, and to surface their tweets to a wider community. Category overload is as much a problem for contributors who want to be found as it is for consumers looking for data.

If you happen to be following a large number of people who are attending something like SXSW, some enhanced Twitter tools might be nice. For example, being able to filter out tweets with a particular hashtag, or in-built group support and the ability to temporarily ignore updates from a group. But it is possible to handle now, either by using a more powerful Twitter client like TweetDeck, or by actively managing the people you follow. 

Going forward, I think improved filtering is definitely going to be important when it comes to the usefulnesss of hashtagging, whether provied by Twitter or third parties. Right now though it seems silly to be saying that social media tools have failed to scale simply because people have dumped all of their content into a single category. There was a failure of users to anticipate and react to the volume of tweets, but nothing, I think, fundamentally flawed with the infrastructure itself.

The phrase “In Real Life”, once used regularly to articulate the gap between online and offline activity, is ceasing to have any relevance. Online communication is increasingly becoming a large part of ‘real life’ relationships, which has some interesting effects that we’re just beginning to see.

Social interaction on the Internet isn’t as new as seems to be implied by all the Web 2.0 talk. From pre-Internet dial-in Bulletin Boards, to Usenet, IRC, IM and online forums, the Net has always had a large interactive component. This interactivity played a large part in the rate of growth of Internet usage, despite some people seeming to think that the early adopters were spending all their time online reading static web pages.

The communities that evolved early on were often heavily topic focused which encouraged a somewhat isolationist mentality to evolve, where a person would bring a slightly or significantly different persona to each community. And most importantly, that persona was most likely quite different to who they were in the real world.

For someone used to this world, at first it’s quite hard to understand something like Facebook because the idea of using a web app to manage real life relationships is completely foreign. Although genuine friendships were developed online, it was rare (and more than a little strange) to find out that someone behind an avatar was someone you knew in real life.

The real revolution of new online social media is that it bridges the gap between online and offline personas, largely because it is focused on relationships rather than topics. In part as a consequence it has vaslty increased the number of people who use the Internet for social communication, which has some interesting consequences.

Read the rest of this entry »