Big data or better data?

Much has been said and written about ‘big data’ over the past few years, but how much of all the information that we’re keeping hold of do we actually use? Control F1 Product Development Director Dale Reed discusses.

The term ‘big data’ – the ability to compile a mass of data for each of your customers or prospects – is becoming the latest moniker of the digital age. But the reality is that much of the data we hold on our contacts has little or no relevance to their purchasing patterns or when they are in the market to buy our products, and the data that we gather can fast become obsolete and simply bloat our systems with useless, or worse, USED, but inconsequential and often misleading data.

Because servers and storage are relatively cheap these days, the temptation, like a hoarder stacking their newspapers or other treasures, is to accumulate every microscopic bit of information you can and add it to the pile, ‘just in case’. But if we carry that analogy forward, our hoarder will soon find it difficult to put their finger on an important article they wanted to read, when it’s buried under thousands of other pieces of information, and soon the obsession becomes the collection and storage of data, rather than the information it contains.

Conventional wisdom tells us that when dealing with big data, we should be concentrating on the three V’s. Volume, velocity, and variety. Volume, obviously, is the amount of data we can expect to have to store. Velocity is the speed at which that information is going to be accumulated. And variety is the variability of that data, as each source will often have a different format and purpose. By looking at these three factors, we can calculate the storage space and bandwidth we need, as well as the layout of our big data warehouse.

However, perhaps even before looking at the conventional three ‘V’s, we should instead focus on a fourth; Vision. What do we want to use the data for? In our efforts to keep up with big data, this simple and obvious point is often overlooked. By determining the purpose of the data, and the end result we want to achieve, we can drive everything else in the process including, crucially, what information we actually need to store to achieve our goal.

By spending some time on our vision, we can decide what data actually helps us identify and target customers for marketing purposes. Should we be capturing information from social media? How much transactional data should be kept before it becomes inconsequential or even misleading? Can some data be summarised, and therefore eliminated from the data? 

As a simple example, for transactions over three months old, just keep the dates and basket totals. Transactions over 12 months old, just keep a monthly, weekly or daily total. Transactions in the last three months, we can keep all of the basket and individual product purchases.

By keeping a consolidated, clean, and trim marketing database that contains only the data that we know from experience influences customer decision making, we can expect to reap the rewards of the data we hold, and not get bogged down in searching through terabytes of useless or out of date information for the one nugget that can actually help us achieve our targets.

So take the time to analyse your existing data. Find the information that will be useful. Make the painful decision to let go of data that you know will lose its usefulness over a period of time. Hoarding information will not, in the long term, improve your ability to qualify customers for a campaign. Only storing the right information will allow you to do that.

Let’s put aside the ‘big data’ terminology for now, and try for ‘better data’ instead.

This post is an extract, taken from a piece originally written for dmbgroup. View Dale’s original piece here.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s