If you read my recent blog, True data about big data, in which I proposed that Zero Dark Thirty’s lead character, Maya was a near perfect example of a data scientist, you’ll understand today’s blog title. The unstated subtext of that blog was that it’s unwise to turn big data over to the IT group. Sure, you’ll need them around, but their discipline differs significantly from what’s needed. And. . . I’ve found plenty of IT execs who understand that reality.
I argued the case from my experience and knowledge about
the thinking styles and knowledge base of IT and other business functions. My
cognitive approach is one way to go at the issue. But Marchand and Peppard
arrive at the same conclusion from still a different tack in their October HBR article,
entitled Why IT Fumbles Analytics. They got
right to the crux of their understanding in the first paragraph:
In their quest to extract insights from the massive amounts of data now available from internal and external sources, many companies are spending heavily on IT tools and hiring data scientists. Yet most are struggling to achieve a worthwhile return. That’s because they treat their big data and analytic projects the same way they treat all IT projects, not realizing that the two are completely different animals.
What this implies is that rather than emphasizing information in the big data, information is viewed not as a resource that makes possible the design and implementation of more IT systems, but, rather as something that people themselves make valuable in their discipline and for their customers. To go back to the CIA analyst in Kathryn Bigelow’s film, Maya’s task as the most knowledgeable analyst was to question the validity and usefulness of the data. Inevitably, that’s a difficult task because she had to be able to ask “what” the data means, “why” the data is in such form and why and how it’s taking such a shape before she could get around to “what next?” Her questions were constantly iterated for years. What more data do I, as knowledgeable CIA agent, need? What conclusions can I draw thus far, and is this enough data to make a decision? Then, as I emphasized strongly, she had to sell her conclusions.
Strategic approach
to big data
Without focused theory development, the data scientist
will be overwhelmed by the amount of data available. All the IT processing
power in the world won’t help. Maya’s theory, of course, focused upon a
strategy to locate Osama Bin Laden. Her fundamental hypothesis was that if she
could locate his primary couriers, she locate him. The film began with torture
tactics for gathering courier identities, and, essentially stayed on that
fascinating and difficult track for nearly two hours before actually finding
the terrorist. Then, she faced the intriguing task of selling her conclusions
to the State Department, an issue that was just as difficult, perhaps, as
locating the terrorist. In sum, the first steps after creating a hypothesis are
acquiring, harmonizing and mining the data sources. Of course, in the
iterations of this process, you will inevitably find new, exciting insights.
That also implies, as David Meer has written, that as you gather these insights, it will be important to be open to new
approaches and to challenge sacred cows.
If you’ve ever done a major hypothesis-driven, research thesis, the strategic approach to big data is exactly the same iterative process as the Zero Dark Thirty portrayed. The distinction between my two analogies (Maya and hypothesis driven thesis) and the “big data approach” is that once the hypothesis is in place, IT tools are necessary to mine the data.
At the heart of
the initiative
It can’t be emphasized strongly enough that people are at
the heart of the initiative, not IT tools, big data initiatives or the IT group.
Marchand and Peppard are devastating in their critique of the IT logic:
The IT approach assumes. . . that giving managers more high quality information more rapidly will improve their decisions and help them solve problems and gain valuable insights. That is a fallacy. It ignores the fact that managers might discard information no matter how good it is, that they have various biases, and that they might not have the cognitive ability to use information effectively.
As I have argued previously, most IT professionals come from computer science, engineering and math backgrounds resulting in a focus on technology and the use of different cognitive approaches rather than what’s necessary. Thus, a different animal will be needed to create and make the best use of information. I have no doubt that Marchand and Peppard are correct in their recommendation of the use of cognitive and behavioral scientists. But that’s a personnel expense only the largest of firms are currently willing to fund. People who have a deep and rich understanding of the business are absolutely necessary for big data success. And then, gradually, as business understands the opportunities provided by big data, they will need to add the cognitive and behavioral scientists who. . . understand how people perceive problems, use information, and analyze data in developing solutions, ideas, and knowledge.
The evidence is already clear that data-driven decisions tend to be better decisions. Thus, I’ve attempted to lay out a big-picture of the fundamental processes to make advanced analytics work for a company. The process—simplified--is to get the right hypothesis from the appropriate people, mine the data iteratively, create the new business information, sell it to the relevant executives and implement.
Flickr photo: UWWW ResNet