I typically respond that our competition is the status quo. By status quo, I am referring to the traditional approach of hiring statisticians to sit in front of advanced software tools, building and studying complex analytic models trying to discover important insights in data. In my opinion, this process has not kept up with the onslaught of data and the demand for instant turnaround that businesses require. There simply is not enough time to build and refine models following the approach of the status quo.

Consider the business era of the 1950s pre-desktop computing. The age of the typing pool. The age of manual correction tape. The age of paper. Some of you are probably not even old enough to know what some of these things are. They are, more of less, extinct; they are a part of the retired fabric of a business world long since revolutionized by technology. They are extinct because breakthroughs in technology put the power of desktop publishing into the hands of every business person at an affordable price. The investment in teams of manual typists was replaced with more powerful and more cost effective desktop publishing software fully equipped with spellcheckers, grammar checkers and advanced layout tools. Technology challenged and usurped the status quo.

One could mourn the loss of the typing pool. One might argue that, for the good of our economy, it is important to create or at least maintain jobs. However, those people in the typing pool didn’t disappear. They found new ways to make the business more effective and more efficient. The shift from manual, rote, mundane activities to creative activities enables business people to search for new and innovative ways to beat their competition.

Let me bring you back to the world of data analysis. Do I think human data crunching automatons (aka data analysts) are headed for extinction? No, absolutely not… yet. Do I think that the role of the data analyst will change? It already has changed significantly over the past ten years and will continue to evolve rapidly. The tools available today enable analysts to build sophisticated models in record time, but that time savings is only significant when compared to the status quo: a human-intensive process. With a new paradigm of machine intelligence working on the intensive data crunching challenges, the role of the data analyst can be transformed into a master of creativity seeking out important questions to ask of machine intelligence-driven analysis in order to identify the best strategic course of action.

The data analyst will not disappear any time soon. The role – like all roles in business – will continue to adapt and evolve. Interaction with machine intelligence will become increasingly more integral to the process of data analysis. The machine intelligence will do the investigation and present back appropriate insights to the creative business person.

Meanwhile, the new owner of his farm picked up an unusual looking rock about the size of a country egg and put it on his mantle as a sort of curiosity. A visitor stopped by and in viewing the rock became so excited he could barely contain himself. He told the new owner of the farm that the funny looking rock on his mantle was about the biggest diamond that had ever been found. The new owner of the farm said, “Heck, the whole farm is covered with them” – and sure enough it was.

The farm became known as the Kimberly Diamond Mine. It is the richest diamond mine the world has ever known. The original farmer was literally standing on “Acres of Diamonds” but instead of looking in his own property, he sold that very farm to seek diamonds on a distant continent.

Think about this story as it relates to your business. You could consider data in the your business to be like the farm in the story, laden with diamonds simply waiting to be extracted. How many businesses do you suppose there are that spend inordinate amounts of energy, time and resources searching for key answers from outside experts, outside data vendors, and external market information. Surely, someone out “there” has the answers. But what if the answers, like the diamonds, are right there in your very own backyard. You only need the tools and the desire to mine them. The idea of “data mining” in the traditional sense is not what I’m referring to. Simply producing reports of different views of data hoping to find some diamonds is not the answer. If instead you develop and organize specific critical business questions and apply the right type of analytics against your data, you may find the diamonds you seek.

Every day I hear clients say, we need to acquire better data, we need more data, we need to buy a new set of data from the vendor, and in some cases… I need a new data vendor. I often ask, “What are you doing with the data you bought last year, or last week?” After a shrug, the reply is typically, “Well that data is old, I need new stuff, because that is what my competition is buying…new stuff.” While I’m not diminishing the value of current information, is getting more and more data really the answer? Is new data and fresh data the only thing that matters? Not if it is going to sit on the shelf like the last 50TB of data you accumulated.

If used properly, discovering patterns in your historic data can yield tremendous insights. Consider this fictitious example: every year between May 1 and June 1 on the East Coast for the past ten years new car buyers were more likely to buy white 4 door sedans than any other vehicle. If you simply focused on this month’s numbers you’d miss that pattern and the opportunity to overstock white 4 door sedans in late April on the East Coast.

Is there an equivalent in your business? Are there undiscovered opportunities sitting in your data right now? How many of these diamonds are you searching for on a distant continent instead of in your current data store? The answer may be to start looking in your own backyard. It is very likely that you have a lot of data and have yet to harvest the diamonds. The key to beating your competition may be using the very resources that are sitting quietly in a spreadsheet or database just waiting to be harvested.

To see why, we need to examine what human beings are good at and how they are different from artificial neural networks. Although we have a large, long-term memory store at our disposal (it has been estimated that we can remember up to 50,000 distinct faces), our short term memory is rather limited – it varies between a few and up to ten distinct items depending upon how they are organized internally. The implications of this for human intelligence are profound. We tend to develop models and theories from a few well-chosen characteristic examples and we are able to examine only the most salient aspects of these examples. This works extremely well when the number of variables at hand is relatively small and there is a great deal of regularity in the data. It has allowed us to develop everything from MRI machines to the theory of relativity.

But consider the situation when thousands or millions of records exist and each of these consists of many hundreds of independent variables. For example, a company may have a record for each financial transaction each with dozens of variables describing the customer’s actions. This may include demographic and past behavior information. Moreover, this data may be noisy or even contradictory, either because the information was recorded incorrectly or simply because of quirks in purchasing behavior. It is well beyond the ken of human processing to take all of this data and develop a coherent model. Of course, it may be possible to develop intuitions regarding why certain people buy certain products, but intuition will break down in all but the most obvious cases.

This is where Neuron comes in. Neuron has no trouble examining hundreds of factors, extracting out the most important of these, and determining the correlation between the input features and the desired output. In an important sense, it is Neuron’s lack of common sense – or preexisting bias – that allows it determine non-obvious relationships in the data. This combines with the ability to entertain thousand of possibilities at once to give it insight that can surpass native human capabilities.

In summary, we can state the following. When working in a relatively confined domain in which the determining factors are well-understood and cleanly relate to the desired outputs, humans will often outperform computational systems because of the flexibility of their model-building behavior. But in the increasingly common case in which vast amounts of data must be pored over to extract meaningful relationships, Neuron can often outshine even the most qualified expert in the field in question.

1. Why do old school analytics solutions require years of training and expertise?

In the old days, it was assumed that experts with advanced training in statistics and computer science would be the only people that would ever need to build predictive models. After all, they were used for scientific research purposed primarily. But that has all changed now that many businesses are realizing the power of using data to make smarter decisions. Businesses that might benefit from advanced data analytics find themselves facing the daunting task of hiring people with years of expertise and training to build and maintain predictive models.

2. Why does it take so long to get results?

Traditional data analysis is time consuming because it is labor intensive. Even with the advanced tools available to scientists, it takes a long time for experts to build and perfect predictive models. The process is trial and error to a large extent, and there are only so many trained experts to go around.

3. Why are the results so hard to understand the results and to put those results into action.

Traditionally, the results of advanced data analysis were meant for scientists, not business people. So, the results are often hidden within complex language and equations that require an expert to interpret. After all, the analytic tools were designed for scientific research. Does the average business person really understand the difference between ordinary or weighted least squares… do they really need to?

4. Why is it so expensive to get results with older solutions?

Expert People + Hardware + Software Licenses = Expensive

5. Why can’t it be easier and more cost effective for my business to get predictive actionable insights from my data?

It can. It is. The answer is Neuron™. The world’s first automated analytic engine from ColdLight. Neuron is Software as a Service, designed to automate the entire analysis and predictive insights process for you.

ColdLight’s Neuron is powered by next-generation technology that automatically discovers patterns in massive sets of data and offers action plans to achieve your critical goals. Neuron automates data analysis for you, making it easier and faster for you to turn large amounts of data into competitive action

Neuron requires absolutely NO expertise in statistics or data mining
Neuron performs the work of thousands of statisticians in minutes.
Neuron is the ONLY product on the market that delivers automated,. discrete action plans that show you how best to achieve specific business goals (“To achieve your goal follow this plan…”)
Neuron is Software as a Service, no hardware to install or maintain
Neuron is fast, easy, cost effective and designed for your business.
Neuron integrates easily into 3rd party products and services

Neuron can work as a stand-alone solution to help you make smarter, more informed decisions or plug right into your software products to literally turn them into “intelligent” machines. If you have highly transactional software products (CRM system, financial services platform, healthcare product, call center platform, or other data rich products), you can embed Neuron Prescriptive Intelligence directly into your product.