FireStarter

SXSW: Finding Interesting User Generated Content

Got an allstar panel here with people from Dachis Group, Mashable, Facebook, AT&T Labs, and the former Reddit Scientist.

What do you keep track of?

Facebook says that they think in terms of the graph, and all their successes and failures end of coming from how the social graph (edge ranks) of interactions are forming, and adapting the interface to surface the information that people are gathering around as important.

What are your business metrics? Before you start harvesting data, understand what information will be important to the business and begin generating queries that will return data that counts. Otherwise, you’ll pull back insights that don’t have an effect on the business.

Facebook can do deeper analysis where they can see if a viewed status update leads to something they call, “Important:” linking all the way to a new friend. But their challenge is based around the mom factor in the news feed. She’s important to your social graph, but her content won’t generate a ton of views, clicks, likes, or comments.

Facebook just switch from a manual tool to understand what you want via the graph to more of a machine learning method. They did for flexibility, and what they could learn. Much like Lou Rosenfeld’s point in Site Search Analytics, they want to get to the point where they can have fun trying different things and uncover insights they weren’t expecting.

Try Everything

If you have a predictive problem — as they did on the way to winning the Netflix algorithm prize, just try everything. A huge advantage was that the Netflix data was incredibly clean. But in real life, much of your time is spent collecting and cleaning the data until it’s exactly what you want, which keeps people from having the time to play with the data and experiment with different algorithms.

Qualitative analysis — looking at the data — is important to help systems present more signal and less noise. If you just look at the data, the noisiest people will get the bulk of the systems attention. 90% of the user base were reading a lot, but not contributing meaningful data into the system.

The fix was randomly picking among them to surface, but now they’ve (Reddit) has gone more to a community problem where people can self sort their own groups to be visible, and this helps to drive more meaningful data back to the analysts for informing changes.

At Reddit, they also spent a lot of time thinking about anti-gaming patters. What kind of voting patterns look suspicious. Most people trying to game the system are actually lazy, so they’re easy to catch.

For the Netwflix prize, there were two users who had rated 15,000 out of the 18,000 movies in the database. They were real people who were fanatical about using the system.

At Facebook, they’ve limited the sharing options to the point that it’s tougher to spam. They have a team dedicated to Graph Integrity: looking for real people, looking for massive amounts of pages created quickly. You need the right tools, and the right people reviewing what the tools present.

Lots of FB apps have a ton of incented sharing methods, and they use sharing as a game mechanism to help them in the game. They need to constantly evaluate how to treat people who have friend groups exclusively for gaining rank in the games.

Facebook say the most interesting think is interactions with people who bring content into the system. Most of Reddit’s pageview activity comes from comments, not content.

Content and context are key. Understanding who you turn to on a specific topic within your social group is the big challenge and the next phase for companies trying to make a difference for people using the system.