FireStarter

SXSW: Beyond Wordclouds - Using APIs to Identify Trends

Chris Busse is going through 100,000 coffee tweets over the first 48 hours of SXSW.

He looked at the tool space, was frustrated by the offerings, and decided to begin creating his own tools. Here are the notes and memorable moments from the talk:

So what’s an API? If you use Twitter, and you use it on any other platform, like your phone, then you’re using the Twitter API to get that information.

The problem he faced was monitoring a large volume of SM data, identify trends that aren’t readily apparent, manage the communications activities of a team of digital advocates, and then link this data to other internal software and comms systems.

How he looked at automating process:

Search: Create a script to search tweets at scheduled intervals

Store: Capture the tweets to your database (which should match fields returned by the API)

Crunch! Parse words, mashup with other APIs (like bit.ly) and any correlating customer data (which you’re attaching at the customer ID and Twitter name record in the CRM.

Report: Create tools to filter and display the data

Tweets + Your CRM Data = Social CRM

Think about being able to open tickets through Facebook, so that the customer call center time is spent prepared to resolve the issue faster.

ROI: Now you can track sales directly back to the social media efforts.

100,000 Tweets About Coffee

These tweets were collected over the first 48 hours at SXSW.

About 1,400 entries included “I’m at” and “Starbucks” — a written check in

He then searched URLs that included Gowalla and Foursquare, as well as All or “Austin.” Gowalla had far more checkins.

Halcyon and Jo’s were by far the most mentioned term when searching “I’m at” + “Coffee” + “Austin”

Starbucks owns about 30% of all coffee check-ins over that period over all mentions, Austin and other.

He also looked at time of day: Interestingly, Thursday at 8 am was a big spike when people are flying in, as well as noon. Friday, people are spiking on coffee around 5pm.

The most retweeted tweet: @johnmoe: worried about the instability of the nation’s coffee shops when all the hipsters are at SXSW.

The data confirms this. Inside the term coffee, New York is 28%, Austin is 72%. But this is natural language. So this gets us most of the way, but tweets like Brooklyn would be left out.

What’s in a word?

140,000 words in the 100,000 tweets

Strip out ! ? and also :

Strip out - excluse common words like “the” and”

Put all the words left into a separate database for analysis.

The problem is that general word clouds lack relevance. It doesn’t help you get to real information or correlation.

He exported the words into a spreadsheet, scanned the first thousand, and starred the most relevant words manually.

Then you can export and scan the frequency of those words.

This method popped up decaf as the #5 term. Why? Looking back at the tweets, there was a bot tweet that got retweeted a ton and clouded the data.

The sixth most popular link was someone who posted a picture of his coffee cup on Tumblr. But this is a way to uncover someone who is clearly reaching a large audience with content.

What else can you explore in more detail?

Lat/Long location service: Tons of interesting data to sift through

Words over time: How do the phrases change between the last pull and this one?

Begin to find real influencers in the context of your strategic goals. It’s not as simple or easy as scoring them, but looking at these spikes quantitatively.

What does the future hold?

More closed APIs: Twitter is closing many of the data streams that people could create public-facing software with. Internal uses will still be okay.

Traditional software vendors working with platform APIs intelligently — not just buzzword compliant.

Smart people solving this problem for the enterprise and smaller orgs in a way that fits strategy, not forcing a strategy into a way the tool works.