IT and business leaders face big gap in understanding of data priorities

Over three-quarters (77%) of business leaders claim it is easy to use the data they need for their jobs, yet IT people struggle with data management, research has found.

The Morning Consult survey of 4,000 business leaders and tech practitioners for Capital One found that 70% of IT staff report spending up to four hours a day fixing data issues, conducting quality checks or correcting errors.

According to Capital One, these ongoing struggles not only slow down workflows, but also point to deeper issues with data management and governance to ensure high-quality data.

The survey reported that although data culture is a top indicator of artificial intelligence (AI ) success among those polled, only 35% of respondents said they have a strong data culture, citing inconsistent support and education. In fact, over a fifth said their organisation lacks a strong data culture or there is inconsistent leadership support, talent development and education around data.

The majority (87%) of business leaders who participated in the survey believe their organisation has a sufficiently modern data ecosystem to build and deploy AI. The fact that only 13% of the technical people polled were confident they could fix data issues in less than an hour shows a disconnect between business goals and the technical implementation challenges that must be overcome to meet these objectives.

Companies are looking to deploy advanced artificial intelligence such as multi-modal AI, which requires the ability to process unstructured data in various formats and on a massive scale. However, according to Capital One, the contrast between perceived ease and the time spent resolving data problems highlights how many organisations overlook the result of poor data management in an increasingly complex environment.   

The survey also revealed inconsistency in how business leaders rank data security. While 76% ranked data security as their top concern in AI initiatives, followed by data quality (73%) and data management (65%), over half (53%) said their organisation prioritises data management to mitigate risk. In fact, 38% admit data management is given only moderate importance. Efforts to address security risks vary, with 58% of business leaders using data encryption, but only 20% using tokenisation.

When asked about their organisation’s progress with cloud integration, 41% of leaders and 33% of practitioners with advanced implementations said they are scaling automation technology across the enterprise. However, in those organisations at an early stage of implementation, 15% of leaders and 18% of practitioners are running pilots in some parts of the business.

Cloud integration is considered an important step in improving the data management required for AI. Terren Peterson, vice-president of engineering at Capital One, said: “Doing AI and ML [machine learning] is hard enough on its own.”

Organisations need to start with a good foundation of data, he said, and to achieve this, they should aim to build a data platform based on standardised data using a single data pipeline.

According to Peterson, those organisations that have fully embraced cloud technologies are in a better place in terms of the data management required to support AI, compared with those that rely on on-premise IT infrastructure. “Cloud-native organisations are just two or three clicks away from building their data platform,” he added.