How “Dark Data” is damaging your business efficiency—and what to do about it

Move over, content—data is the new king of business strategy and success. But with companies drowning in a sea of numbers and statistics, new research from data actions and outcomes firm Splunk shows companies are ignoring potentially valuable data—most likely because don’t have the resources they need to take advantage of it.

Although business execs recognize the value of using all of their data, more than half (55 percent) of an organization’s total data is “dark data”—meaning they either don’t know it exists, or don’t know how to find, prepare, analyze or use it.

In an era where data is connecting devices, systems and people at unprecedented growth rates, the results show that while data is top of mind, action is often far behind.

• 76 percent of respondents surveyed across the U.S., U.K., France, Germany, China, Japan, and Australia agree “the organization that has the most data is going to win.”
• 60 percent of respondents said that more than half of their organizations’ data is dark, and one-third of respondents say more than 75 percent of their organization’s data is dark.
• Business leaders say their top three obstacles to recovering dark data is the volume of data, followed by the lack of necessary skill sets and resources.
• More than half (56 percent) admit that “data-driven” is just a slogan in their organization.
• 82 percent say humans are and will always be at the heart of AI.

“Data is hard to work with because it’s growing at an alarming rate and is hard to structure and organize. So, it’s easy for organizations to feel helpless in this chaotic landscape,” said Tim Tully, chief technology officer at Splunk, in a news release. “I was pleased to see the opportunity people around the world attach to dark data, even though fewer than a third of those surveyed say they have the skills to turn data into action. This presents a tremendous opportunity for motivated leaders, professionals and employers to learn new skills and reach a new level of results.”

Respondents are slow to seize career and leadership opportunities

While respondents understand the value of dark data, they admit they don’t have the tools, expertise or staff to take advantage of it. Plus, the majority of senior leaders say they are close enough to retirement that they aren’t motivated to become data-literate. Data is the future of work, but only a small percentage of professionals seem to be taking it seriously. Respondents agree there is no single answer, though the top solutions having potential included training more employees in data science and analytics, increasing funding for data wrangling, and deploying software to enable less technical employees to analyze the data for themselves.

• 92 percent say they are “willing” to learn new data skills but only 57 percent are “extremely” or “very” enthusiastic to work more with data.
• 69 percent said they were content to keep doing what they’re doing, regardless of the impact on the business or their career.
• More than half of respondents (53 percent) said they are too old to learn new data skills when asked what they were doing to educate themselves and their teams.
• 66 percent cite lack of support from senior leaders as a challenge in gathering data and roughly one-in-five respondents (21 percent) cite lack of interest from organization leaders as a challenge.

AI is believed to be the next frontier for data-savvy organizations

Globally, respondents believe AI will generally augment opportunities, rather than replace people. While the survey revealed that few organizations are using AI right now, a majority see its vast potential. For example, in a series of use cases including operational efficiency, strategic decision making, HR and customer experience, only 10 to 15 percent say their organizations are deploying AI for these use cases while roughly two-thirds see the potential value.

• A majority of respondents (71 percent) saw potential in employing AI to analyze data.
• 73 percent think AI can make up for the skills gaps in IT.
• 82 percent say humans are and will always be at the heart of AI and 72 percent say that AI is just a tool to solve business problems.
• Only 12 percent are using AI to guide business strategy and 61 percent expect their organization to increase its use of AI this way over the next five years.

Regional differences fuel range of opinions: China is furthest ahead in understanding the potential of dark data

The research also discovered some distinct differences in attitude and opinion between the seven countries polled. For example, French, German and Japanese respondents seem less concerned about the value of data skills to their careers, with affirmative answers roughly 25 percent lower on average, than their counterparts in other countries. Respondents in China overwhelmingly voice the most enthusiasm and confidence in AI but their current adoption is only slightly higher than the global average. (20 to 16 percent).

• Australian respondents implied the lowest AI adoption rates among all countries surveyed with 43 percent saying AI is already—or will in the near future be—an important part of their organizations’ operations compared to the global average of 52 percent.
• Although China leads response rates on the value and impact of AI across the research, 93 percent of Chinese respondents also believe machines can never replace human qualities like curiosity, creativity and initiative—the highest of any country.
• Only 64 percent of French respondents think data is a central component of an organization’s success compared to 81 percent globally.
• Only 58 percent of German leaders think data will grow more valuable over the next decade compared to 71 percent globally.
• Nearly four out of ten people in Japan (38 percent) say they are excited about working with data, lagging behind the global response of 57 percent.
• 39 percent of people in the United Kingdom strongly believe AI can make up for the skills gap versus only 27 percent globally.

Read an executive summary of the report here.

“The State of Dark Data Report,” built using research conducted by TRUE Global Intelligence, surveyed more than 1,300 global business managers and IT leaders about how their organizations collect, manage and use data.