Myths exist for reasons. Like fairy tales, or other stories, a myth helps us to explain what was going on around us when there isn’t any science. There are always shreds of truth within the myth because they are based on some kind of reality, or at a minimum, have something in them because of a great story teller. Sometimes myths become reality, and sometimes a reality becomes a myth.
Take Big Data. When the era of Big Data was born, there was a lot of hoopla about how it was the greatest thing ever. We’d no longer have to think; big data would do it all for us. You would be able to target, mark, retarget, cultivate to your heart’s – and your bottom line’s – content.
But lately, there’s a chill in the air when the topic of Big Data comes up. That’s because there were a lot of myths created around it that are now being dispelled. Reality is setting in. Here is a taste.
- Big Data was never meant to be a Wizard of Oz-like man behind a curtain says Patrick Gray. During the 2013 holiday shopping season, many of the big data analytical tools employed by retailers failed to predict consumer behavior.
- Prakash Nanduri says in 2014, Big Data will lose its luster. This top CEO of Paxata says we need to admit that big data—up to this point—has been best suited for needle-in-the-haystack analytics, as opposed to something that everyone benefits from.
- Doug Levin does a great job of dispelling some of the myths around big data and predictive analysis.
- Gil Press has an excellent review in Forbes of Tom Davenport’s book big data and its myths.
- Gartner has a good piece on dispelling them as well. In fact, their competitor, Accenture, suggested once that you have to connect the dots to handle Big Data. But, what if you can’t see them?
Erysichthon and Big Data
A Greek myth might help us understand Big Data better. The myth is about Erysichthon, a king of Thessaly who loved to eat. He decided he needed a bigger hall to have his feasts and hold more food. He decided to chop down some trees to build the facility. He found the perfect oak grove with some fine timber. However, the grove was controlled by the goddess Demeter. Erysichthon didn’t let that stop him. You see Erysichthon needed wood – a lot of wood – for his hall.
Now Demeter was the goddess of corn, grain and the harvest to be exact. She watched Erysichthon and his men begin chopping her trees. Erysichthon came to one tree of particular importance to Demeter. When he struck that one, the myth tells us that he saw bright red blood coming from the tree where he cut. Could it be a god in disguise? Erysichthon was surprised, but ignored the blood and resumed chopping. He needed wood – a lot of wood.
Here’s where there are different versions of what happened, which is almost always the case in a myth; over time, versions emerge, are blended, and it gets difficult to tell what really happened. Some versions say Demeter disguised herself as a priestess and went up to Erysichthon, begging him to stop. “The goddess Demeter owns this grove,” she said. Erysichthon ignored her, so she ordered him to leave in a stern voice. He laughed and kept on chopping. They argue some more, he threatens her with his ax. You know how arguments can get ugly. Anyway, she leaves, mumbling something under her breath.
What she was mumbling was a curse on Erysichthon. Now some versions of the story didn’t cover all these different scenes. One version said that as soon as she saw him cutting that tree, she put the curse on him immediately. Still another version has Erysichthon beheading a stranger who was passed by telling him to stop after Demeter left. Who knows the real story?
In any case, the myth – like other myths – demonstrates a lesson. For this one, the lesson is clear: When a human decides to challenge a god, nothing good ever comes of it.
Some versions say Erysichthon was arrogant so he couldn’t help himself. Other versions don’t explain why he decided to keep cutting the trees down, whether it was his love of food, defiance of the gods, or whatever. We’ll probably never know for sure, but the result was the same: the Demeter curse.
Here is what the curse said: Erysichthon would be cursed with an insatiable appetite. This how she did it. Demeter called in Famine (sometimes known as Peina, or Hunger) to infect Erysichthon with the insatiable appetite. “You love food. OK. Now you will never ever be able to satisfy your love. It will be insatiable.” That’s the rough translation, I think.
The curse has parallels with Big Data: the more Erysichthon would eat, the hungrier he got. The curse was also immediate. It didn’t wait until he finished building his hall. One of the versions said he needed more and more money to buy more food because he had used up his fortune within a few days of the curse taking effect. He ended up selling his own daughter to a ship’s captain as a slave. But that didn’t help that much in the end. He just couldn’t satisfy his hunger.
With nowhere to turn, and tormented each and every second by insatiable hunger, the myth explains that Erysichthon began to eat himself, starting with his legs. Eventually, he killed himself by “devouring his own flesh.” It’s the stuff that today’s horror movies love to depict. But actually, it is what will happen to all of us if we keep eating data.
Erysichthoning Your Data
When it comes to data, we’re all a little like Erysichthon. Anyone who has started down the data trail knows that it’s impossible to ever have enough. We have a saying in our company: “The more you look, the more you see; the more you see, the better you know where to look.” But with data, you inevitably keep seeing more. And wanting more. That’s the nature of data. But unlike food, where we have to eat to survive and eventually get full, the hunger for data is insatiable. We just keeping eating and eating, until suddenly we realize we’ve lost a large part of our self somewhere in that journey.
The reality is there is only so much you really need to eat in order to draw conclusions. Nate Silver, in his book The Signal and The Noise, says collect as much information as possible, but then be as rigorous and disciplined as possible when analyzing it. It’s the “as possible” that’s the question.
In the April 2012 issue of Quirk’s Marketing Research Review, my article “Analyzing a B2B magazine’s ad inquiries and their relationship to qualified prospects” pointed out that the number of interviews needed to be assured of some certainty when qualitative research is done is n=30. Peter DePaulo, in the December 2000 Quirk’s Marketing Research Review, wrote about the sample size. In addition, George Gaskell in Qualitative Researching wrote and discussed the number 30 extensively. Combined with our long history of studies, we felt confident we would help advertisers understand inquiries better, and, therefore, the advertising investment, with this number.
In other words, anything beyond 30 is simply redundancy and probably wasting time.
So the question is always: What is your number? How much should you eat?
As you begin chopping your trees for your data house, keep Erysichthon in mind, and don’t challenge the data gods. You might be cursed with the insatiable appetite and end up consuming yourself instead of the world around you.
FOOTNOTE
Even now, myths spring up despite science. I was in a meeting a few years ago where the client was discussing how they created molds for a product without drawings. If you know anything about manufacturing, you know you really need good — really good — technical drawings to create a mold that is going to mass produce a product with quality; the process has to be repeatable, and accuracy is essential. Some of the engineers found it unbelievable that products were being manufactured from molds without drawings. Others accepted it as par. “We’ll get the drawings eventually. But we have no time to waste.”
Afterwards, I was with a group from marketing and they asked me what happened in the meeting. I said this. “Well, there was no time to create the drawings for the product mold, so they called upon the old artisan himself to create the molds. You know, Peter. Peter, of course, is blind, but being a true craftsman, he couldn’t see any drawings anyway. So he created the molds by hand, a real artist. He had the touch. He created the molds for the products. They are being used in production now.”
They listened to me like I was Homer reciting the Iliad.
Two years later, I was in a meeting talking with a group about the latest production run of the product, and one of them said, “I’m glad we were finally able to get the drawings for the mold rather using the blind guy’s versions.” At that instant, I knew my myth had become reality — at least for a time. To this day, you can hear this story about the blind artisan who rose to the occasion and saved the company by creating molds by hand. Thank goodness for myths!
Got a good myth? Share it with us. Thanks for reading!