Big Data, Small Startup

Big Data is a daunting prospect for businesses of all sizes. A little over fifteen years ago, John Mashey told us that the volume of data created by human beings far outstrips our ability to make use of it. The Library of Congress processes 2.5 petabytes of data annually- and every item bears metadata. This load is set to increase by 400% in the coming years. The library stores information on 15,000 to 18,000 spinning disks attached to 600 servers in two data centers. Over 90% of the data, or more than 3PB, is stored on a fiber-attached SAN. As startups, we do not have to deal with anything like that volume of data, but I’m telling you about it to make a point. The Library of Congress holds more data than anyone can ever use (as an individual) and we have to consider that all the data we glean is not always useful. We are not librarians, we are in business. The data we obtain should be useful and foster profits. A library has public funds to hang onto everything – we do not.

What this means is that while it’s posited that 1.5 million more data managers are required in the US alone to deal with Big Data which could leverage a 60% increase in operating margins, the data management task starts with you. The problem is there really is no ubiquitous solution to data management. There is help for us little guys out there though. Here’s three cool tools to make your data management more intuitive and an asset, instead of a bleed on resources and money.

1. Watson Analytics by IBM

I have to say this one is pretty great. I’m not much of a data miner, and IBM knows that the vast majority of businessmen are pretty busy types. Watson automates everything for you- all the tools are presented in the operating system so you can easily refine and present data. One major advantage Watson has is that all data analysis is combined onto one platform, so you can identify patterns and meaningful insights in a user friendly way. In UX design terms, it’s pretty much perfect. It’s out in Beta right now, and the freemium launch is this month. Keep an eye out for it.

2. Canopy Labs

If you’re looking at more customer specific data rather than broad trends, the 360° Customer View combines all your customer data into individual files – engagement, value and purchase history as well as metrics on email behavior and the like show you which customers you are likely to reach out to with marketing and which ones are beginning to find you a bit annoying. With this tool you will be able to better manage and improve your outreach campaigns based on the data analytics you receive. It’s also free for up to 5000 customers, so it’s pretty ideal for a startup in the early stages.

3. Qualtrics

So what happens if you don’t have any data to analyse? Qualtrics is what I’d suggest to you. Their product allows you to conduct and apply market research on a small budget; their work is on a quotation basis, but this is a minimal expense option to gauge the market. If you’re an ingénue to the world of Big Data, it might be worth your while to get these guys to walk you through your first research projects and get some analysis.

These days it can seem daunting dealing with all the information flying around the Internet, but thanks to technology we are able to compete against big business in the big data world. We might not be able to afford data scientists and market research companies, but we can use tools to get a competitive edge. That to me is not just the essence of good business; it’s what human beings have been doing for thousands of years. What solutions do you use for data analysis? Let me know in the comments if you’ve come across something that has changed the way you work.

Click here to tell us about your project