Joshua Saunders

I like to think we are now in an unprecedented age of data.  Mass amounts of data.  Data is multiplying like a virus, growing exponentially, and it’s going to consumer us all!  Well, not really, but it is the next great engineering challenge that is going to offer spendid rewards to those that learn how to manage it efficiently.

Cloud services have been popping up like crazy for all different purposes.  Dropbox, the personal cloud storage service, is gaining lots of attention, as are many of its competitors.  And while this presents unique opportunities for entrepreneurs, it represents significant challenges for businesses.  As more and more data accumulates, it is harder to keep track of, access, find what you’re looking for, standardize, normalize, match, ……

In the last decade, we saw how the age of Search exploded, and was won by Google.  The biggest problem now isn’t how do we search through the data, it is How do we aggregate and match all the data we do have into something that is useful.

In building Greencurement, like many companies who deal with product data, we knew from the beginning how hard collecting, aggregating and managing data was going to be – except it ended up being much, much harder anyways.  There are multiple variables that impact this challenge:

• Standardization – What one company calls an attribute, another calls something else.  How one company reports a variable, another uses something completely different.
• Normalization – Data with different reference points is misleading (or useless) when combined together.  You need to understand how the data was measured and bring all data points to the same measurement points.
• Matching – After you have clean data, now mashing it together is still an enormous challenge.  Different parties call things different names and use unique identifiers.

At Greencurement, we’ve focused hard on developing processes to get good, clean data.  We’ve also built a really cool matching engine that makes our lives easier when matching products and data.  But it is still a long uphill road and a lot of people are fighting the same battle.

Take for instance, FindTheBest.  Completely different industry, but very similar challenge.  With FindTheBest, you can select a topic and compare data on that topic to Find the Best – but the challenge the company has is not just in finding the data, it is in matching, normalizing, and standardizing.  Don’t believe me, listen to Kevin O’Connor, co-founder of Double Click.

If you’re an entrepreneur looking for your next challenge, there are few better places to look than solving the data problem.  Not collecting data – making data useful.

P.S. – just as I’m about to post, I see that Box.net, the cloud storage platform, has just raised $81M at a $600M+ valuation.  What timing!