Taking Big Data Down to The Personal Level

Aspirin was unveiled by Bayer in 1899 for the treatment of toothaches, headaches and other pains. But it wasn’t until the 1960s and 70s that it was recognized as a treatment for heart attack and high blood pressure. This gap in knowledge existed simply because pain reduction wasn’t an application they were measuring for. Tons of people were benefitting, but that data wasn’t being captured so no one was making the correlation.

I learned this information in a 2010 Wired article about Sergey Brin’s search for a Parkinsons cure, and it has stuck with me vividly since then and made me wonder what sort of valuable insights might be hidden in my own data. For years I’ve been using apps to comprehensively track my life. I use Miso for the movies and TV I watch, Foursquare for the places I go, Pose for the outfits I wear, and many other apps to follow everything from my period to the food I eat. But right now this is all just a lump of data, with no real application, outside of showcasing my lifestyle on a few social networks.

I know it’s not a revelatory observation to say that wrangling big data has immense value, but so far I see most of it is happening on the enterprise level rather than individual. One of the big opportunities I see now is to help people make sense of this personal data to identify unexpected correlations that might be affecting their lives. If Netflix can use big data to deduce that I might like a Morgan Spurlock documentary, then why shouldn’t I have the ability to parse out that I’m more likely to prefer strong male leads in movies on the 10th day of my ovulation cycle.

I’ll admit, some of the applications I have in mind are a little frivolous. I would love to see if the entertainment I watch affects the outfits I wear, or if the music I listen to affects the places I want to go out to. But more seriously, as a generalized anxiety sufferer, I’d really value a tool that helped me identify the more silent sources of some of my worry. Perhaps getting less than 300 steps a day drastically affects my mood, or perhaps wearing particular colors results in elevated mood.

The technology exists to do this, and big brands are already harnessing tons of big data to make correlations that help them sell. Gilt Groupe famously sends out 3,000 different version of their messaging for different types of customers. So perhaps if the technology is here, but the service isn’t, it’s a sign that the market for this level of self quantification and data analysis just isn’t there yet. That’s not surprising considering that self tracking can still be pretty tedious (today Foursquare still only has less than 20 million active users), but as the internet of things and wearable technology increase this data collection won’t be tedious, it will be automatic. Many of my less techie friends who wouldn’t dream of doing a Foursquare checkin are already starting to use  Fitbit Flexes and Nike Fuelbands to put a number on just how much their cat is disturbing them at night, or just how sedentary their desk job is making them.

Self quantification is about to open up in an entirely new way, but will we be able to do anything actionable with this information once we have?