thinking stiff

Cursebird Leaderboard

Posted in twitter by thinkingstiff on 2009/02/04

The programmer of Cursebird, @richardhenry, has released a preliminary leaderboard (That would have saved me a lot of time three weeks ago!). One interesting thing I see on the list is that @bollocks gets every one of his tweets counted because of his name. He doesn’t even have to bother cussing.

Leaderboard Problems

Most people have wanted to see a leaderboard, but they haven’t necessarily thought through what that means. As I was working with @ThinkgingStiff, I thought a lot about the problems of creating one. Most people want bots excluded, and that seems easy at first. Bots like @ThinkingStiff and @Fuckbot are obviously there just to get to the top. But since Twitter doesn’t forbid bots, it leaves the job of excluding them up to the programmer. A first pass cleaning of them would be easy enough, but they will keep coming, I imagine, and it will become a daily task.

You also have bots that are reposting articles, scanning websites for keywords, or aggregating data. These are useful outside of the Cursebird universe and many of them are interesting and have many followers. Should they be excluded? After that, would come real people that are cussing their hearts out just to make it to the top. They have real accounts, real friends, and are really cussing. As soon as you have a leaderboard, people are going to compete to get to the top. That is just human nature. So pretty soon, you can be sure, the leaderboard will be all people that are trying to be there. It’s not very easy to determine who is just trying to be on the list and who just cusses a lot.

And once Cusrsebird has a list, with rules about who can be on it, there will inevitably be “good citizens” who want to rid the world of wrongdoers who will be scanning everyone’s tweets looking for “fakes.” They will then email the programmer to complain. Trying to keep the leaderboard clean would not be a job I would want to manually tackle on a daily basis.

Another problem I thought of is the effects of a leaderboard on website performance. Once there is a leaderboard, everyone will be clicking on these people to see who they are. Currently, it displays every swear a tweeter has ever said. For someone like @ThinkingStiff, at over 7000 swears, it takes a long time for the page to load. Only displaying the previous 500 might help with that.

Possible Solutions

It seems like all the manual solutions suck. I thought about a few automated ways to make bots less likely, but all of them depend on processing time on the database, and I don’t know how much time Cursebird has free in a day.

1. Limit the total daily swears to something more human, say 50. Twitter’s limit is 1000, of which about 500 will get counted (explanation). Only counting the first 50 swears would make bots much less useful.

2. Require an account to have at least one tweet over a month old. When someone starts to write a bot, they usually use a new account. If they have to wait a month to see any results, they will probably lose interest.

3. Require an account to have at least 10 followers. Very few people want to follow a bot. They may start, but will quickly stop following them.

4. Exclude duplicate swears. This is a big one and wold eliminate most common bots. A bot is either random words, which people quickly discover, or a finite list of tweets that get repeated.

Suggestion #1 could be done real time or as a batch on everyone. Suggestion #2, #3 and #4 would only need to be done on maybe 100 of the top cussers when building the leaderboard. Of course there are ways around #3, and #4 as a bot writer, but #1 is a big roadblock.

Conclusion

I’ll admit that a bot-free list would be interesting. Tweeters like @mollena are amazing. She just really cusses (and tweets) all the time.

Share: Digg . del.icio.us . StumbleUpon . Facebook . Twitter

6 Responses

Subscribe to comments with RSS.

  1. Richard Henry said, on 2009/02/04 at 12:52 PM

    @bollocks is on the list of bugs to fix before I release the leaderboard. It’s caused by the Twitter API returning swears matching a username in addition to the text body, which isn’t the desired result. It was only when someone tweeted me about it that I realised things weren’t happening as expected.

    Possible Solutions:

    1, 2: These are both possible solutions; and it’s always better to automate rules if possible to keep a level playing field. I’m thinking about a couple things myself.

    3: Unfortunately this is pretty much impossible. Grabbing follower counts for each user would require far too much API interaction above and beyond that I’m already doing; and I would quickly hit the rate limit.

    4: It would be too easy to evade that by having an array of random dictionary words inserted into tweets to prevent duplication.

    Conclusion:

    Perhaps I could publish a filtered list alongside a raw list?

    Good article, you’ve said a lot of things I wanted to say myself in a post but never had the time.

    Richard

  2. thinkingstiff said, on 2009/02/04 at 1:22 PM

    I’ve also thought that maybe I’m overestimating the staying power of the bots. Perhaps manual cleaning won’t be that big of a deal and the bots will get tired of trying.

    Two lists might work.

  3. thinkingstiff said, on 2009/02/04 at 1:34 PM

    I also just noticed that on a cleaned list, with my original hand-typed 700 tweets, I still would have been on the top. An example of the “cussing their hearts out” tweeter being able to take the top.

  4. […] this is just from seeing people come up on the live feed. Many people have asked for a Top 10 list (thoughts on a leaderboard). I wouldn’t mind seeing one either, but after staring at it for hundreds of hours, […]

  5. Brian said, on 2009/12/20 at 9:55 AM

    All the best of luck to you!.Thanks for the information.

  6. JESSE DZIEDZIC said, on 2011/10/19 at 10:15 PM

    That was interesting piece of writing!


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s