Data-quality open-source platform Great Expectations recently announced the results of a survey highlighting top pain points and consequences of poor data quality within organizations. Insights from 500 data practitioners (engineers, analysts, and scientists) reveal that 77 percent have data quality issues—and 91 percent said it’s impacting their company’s performance.
“Poor data quality and pipeline debt create organizational friction between stakeholders, with consequences like degraded confidence,” said Abe Gong, CEO and cofounder of Superconductive, the company that makes Great Expectations, in a news release. “This survey made it clear that data quality issues are prevalent and they’re harming business outcomes.”
Data quality issues can make it difficult or impossible to see a “single view” of an end-user or service
These issues can also lower productivity, obscure reliable performance metrics, and overwhelm development teams and budgets with data migration tasks. Data practitioners blamed poor data quality on lack of documentation (31 percent), lack of tooling (27 percent), and teams not understanding each other (22 percent). They said data scientists spent too much time preparing data for analysts, end-users complained about gaps in their data (such as lost transactions), and production teams were mired with delays.
Data confidence is critical for organizations to make informed business decisions
Fewer than half of respondents expressed high trust in their organization’s data, and 13 percent had low trust in data quality, which stemmed from broken apps or dashboards, decisions based on unreliable or bad data, teams having no shared understanding of metrics, and siloed or conflicting departments. Additional issues impacting data trust included alert fatigue, misalignment on certain metrics, and friction between teams.
“Data quality is critical to facilitate the making of decisions with confidence across the organization, enabling a singular understanding of what that data means and what it’s being used for. That’s why support for data quality efforts should be found at every level of an organization, from data scientists and engineers to the C-suite and board who have confidence in outcomes for decision-making,” Gong said.
Nine in 10 (89 percent) respondents said their leadership was supportive of data quality efforts
Meanwhile, 52 percent believe leadership regards data quality with high trust. When asked about their company’s current approach to data quality, 75 percent said they validated data. Only 11 percent do not think they have a data quality issue. Data quality efforts included having a data quality plan scoped and budgeted (22 percent), using a specific data quality tool (19 percent), checking data manually (14 percent), and building their own systems (15 percent).
This survey was conducted in May 2022 by Pollfish, an independent research platform, leveraging responses from 500 information services and data professionals based in the United States (57% men and 43% women aged 18-54). 60% of respondents were employed at companies with 250 or more employees.
