The Messy and Scary World of Big Data

/in , , , , , /by

Big Data is frequently a topic that is overheard and discussed in its most generic terms and subsequently never fully explained. Even within our own articles we have avoided explaining this revolutionary sector of the business intelligence field due to its depth and level of complexity. This article will not give you detailed information about big data but hopefully it will give you an understanding of its nature and get you excited about the implications for businesses. With that being said there are dozens of books out there that delve into exactly what big data is and what its broad implications are. Few people realise that everyday some element of their life has been influenced by big data analytics; the broad impact of this field ranges greatly. Some examples of the influence of big data include: improving sea travel since the 1800’s, preventing flu outbreaks, predicting stock market trends, determining who to insure, finding the cheapest flight periods to preventing exploding manhole covers in New York City. The impact of big data is only increasing as our world is quickly undergoing rapid “datafication”.

If any of the above seems interesting I implore you to check out the book “Big Data: A Revolution That Will Transform How We Live, Work and Think” by Viktor Mayer-Schonberger and Kenneth Cukier. This book looks at the morass that is big data and breaks down its nature, implications and challenges through real world cases and simple concepts.

In this article we won’t delve into too much detail but we are going to outline a few of the key concepts of big data and explain to you why the world of big data is a messy and scary place with a level of potential that can only be characterised as revolutionary.

Big data as the name suggests is at its core basically a huge collection of information. Just one of the elements that distinguishes a big data source from a ‘small’ data source is the sheer volume and frequency that data is collected. An example of a big data source from Schonberger and Cukier’s book is when Google launched a project to predict the outbreak of flu within the United States based off fifty million of the most common search queries. Google then applied millions of mathematical models to the data which eventually lead to a list of forty five search terms that could successfully predicted where and when an outbreak of flu would occur to a high degree of reliability. The scary facet of this is not only Google predicting future disease spread from simple searches that might resemble “headache”, “runny nose” or “flu medication” but the fact that this data set is quite small in comparison to those that number in the tens of billions of rows. Where the messy component starts is in what the data actually looks like.

In most business intelligence systems companies are dealing with small, structured data such as point of sales systems or accounting systems. This is data that contains categories/fields such as booking number, item type or id, transaction value, location, date, time etc. Big data in contrast is data that is normally unstructured or commonly less structured for example think about the Google example earlier regarding search queries what does your typical search query look? To think about this lets think about some of the forms a search for flu “headaches” could generate. Not only does the logic that sorts and organises the data have to include the vast range of typos but it also needs to be able to read and categorise complex sentences such as “how to get rid of my cold headache” or any of the other possible variations. Now with that in mind think about the computer processing power and complex set of rules your system needs to have to sort and categorise all that diverse and unorganised data. This complex logic and huge processing power is a major hurdle for big data analytics however the benefits of processing this much data is astounding as it allows for identification of connections and events that wouldn’t normally be possible. Big data allows data analysts to look beyond trends and outputs and look at what is occurring on a macro scale without finding out why it is occurring. If you have enough data that says those cities that search “warm beanies” and “runny nose” end up with a flu outbreak 95% of the time then you don’t need to know why that happens you just need to know that statistically most of the time that is what occurs. This provides companies with clear correlation based actionable intelligence which is exactly what business intelligence and big data is about.

Dozens of companies use the inexact nature of big data to their advantage often finding correlation where no one would have guessed it existed. Amazon might find that people who read “Fear and Loathing In Las Vegas” also end up reading “Lord of the Flies”. Alternatively Netflix may find that statistically those individuals who view a drama such as “House of Cards” will also eventually watch the TV shows “Scandal” or “West Wing”. All of these hypothetical examples are basic and simple uses for the correlations that can be pulled from within big data sources. The more data you have the easier it is to draw accurate conclusions as there is less chance of small deviations in the data casing an incorrect assumption.

What this article aimed to achieve was to explain that big data at its core is all about huge quantities of messy, unorganised and complex data. This data is then analysed not to find out ‘why’ something is occurring but to find out ‘what’ is occurring. It is this ‘what’ that provides companies with the actionable intelligence that can be exploited within the marketplace.

If you enjoyed this article please go and check out Schonberger and Cukier’s book as it is a much clearer and informed examination of big data then I could ever hope to manage.

Mass Marketing With A Twist: Data-Driven Electronic Direct Mail (eDM) Campaigns

/in , , , , , /by

In the digital age, everyone is bombarded with information in particular email information. With that in mind, it can be difficult for businesses launching email based marketing campaigns to distinguish their campaign from the countless others that are sent to people on a daily basis. How businesses can make their campaigns stand out and increase their business’s sales and profits is by launching a data-driven Electronic Direct Mail (eDM) campaigns.

What is a data-driven eDM Campaign?

An eDM is a campaign that targets a group of prospects or clients and is focused on developing relationships or generating leads for businesses. While this may sound similar to a standard email marketing campaign it is actually much more rigorous. An eDM is not simply about emailing as many prospects as you can and hoping for natural conversions from the sheer number of targets. An eDM campaign is about distributing personalised marketing material to a targeted group of prospects generated from data (point of sale data, purchase histories, demographics, supplier incentive programmes etc..) analysis or a strategic objective and then using the data captured from your emailing to engage the prospects with other marketing tools. Emailing is just the first step in an eDM campaign that looks to turn leads into profit for a business.

How does an eDM Campaign use data?

At the initial and most basic level an eDM campaign will allow businesses to examine the opening habits and click through rates of their prospects from their emailed material.  While this may not seem very useful it can be an exceptionally powerful marketing device when you consider the implications of this information when implementing other lead conversion tools such as direct sales calls, SMS notifications, physical mail outs, traditional advertising and remarketing. To better explain the importance of using data to drive an eDM campaign we will use an example of a travel agency although any business that captures customer information through their point of sales system or transactional data would be a prime candidate.

A simple example of an eDM campaign

For the example let us consider a travel agent that wants to run an eDM campaign that targets past customers who have booked flights and hotels to Europe two years ago (This is called list segmentation, an important step to any marketing). The initial step might  be that the business owner sends out an email campaign showcasing specials, discounts and advertising related specifically to Europe as they know these individuals have an interest in the region. After the emails are sent business owner “Sarah” notices that of all the links within her email a high percentage of individuals click through to her website or the campaign page about “European Bus Tours” and the same users also click through on the 2016 European flight specials. This tells Sarah two things, firstly there is an interest by customers in bus tours in Europe and also a potential opportunity to package flights plus bus tours.

Sarah upon analysing this data decides to then takes initiate a two-pronged direct calling and mail out campaign targeting those two products(Europe flights and bus tours) for just those individuals who clicked through on the links. At this moment the campaign is highly segmented, personalised, targeted and driven by genuine interest in a product, all signs are good for a decent conversion rate. Sarah keeps her sales team busy and probably sees a fairly decent return on investment.

You might be thinking well that all sounds great but it sounds a little difficult to do what with organising mailing platforms, graphics, list generation etc.. or it doesn’t seem like there is much data analysis actually going on…Well, that’s because this brief scenario is only the very tip of the data analysis possible with eDM campaigns. A business intelligence company such as Resurg could in fact further enrich the campaign by providing an even higher level of data analysis. For example with enough expertise in data analysis, it is possible for even more highly segmented lists to be generated based on a business’s point of sales data and it may also be possible to use some predictive analytics to qualify prospects.

As mentioned previously where the true value of using data specialists unfolds is in when complex analysis occurs across your segmented list to provide predictive scores for each targeted customer. What this would look like is that instead of having a list of 100 hundred prospects who all clicked on your email links and are segmented to match your campaign, you would have a list of 100 prospects all ranked against each other based on captured and available data to predict which prospects would be the most likely to participate in your campaign and or those that would possibly provide the largest yield for your business.

This article has been to provide a brief look at the potential eDM campaigns have for shaping and changing the way your business conducts its marketing. It doesn’t matter whether you are a travel agent, retailer or specialist vendor; anyone that captures customer data can reap the financial benefits of data-driven Electronic Direct Mail campaigns. Don’t settle for long standing, less effective marketing methods to grow and develop your business, engage with the revolution that is data-based marketing.

Thanks for reading,
Resurg