Follow ZDNET: Add us as a preferred source on Google.
ZDNET's key takeaways
- Eighty-two percent of CDOs are hiring for data roles that didn't exist last year.
- Most executives are struggling to fill new data roles.
- Only 26% of CDOs are confident their data is ready for AI agents.
If you want to consider how fast our world is changing, look at occupations that didn't even exist a couple of years ago, such as AI engineers, prompt engineers, quality assurance analysts for AI and large language models, generative AI data scientists, and AI agent supervisors.
These roles all have something in common: their tasks focus on assembling and confirming that the right data is in the right place for AI analysis. To this end, 82% of chief data officers responding to a new survey by the IBM Institute for Business Value said they're hiring for data roles that didn't exist last year related to generative AI, up from 60% in 2024.
Also: AI agents see explosive growth on AWS Marketplace - over 40x the team's initial targets
Such data skills are urgently needed, and the research suggests that professionals will have their work cut out for them. The IBM CDO study identified data readiness as a major issue in deploying AI and AI agents. Only 26% of CDOs were confident their data capabilities can support new AI-enabled revenue streams, the survey found. This result aligns with a recent Salesforce survey that also spotlighted this issue.
A majority of respondents (77%) said they're struggling to fill these key data roles. Only 53% said their recruiting and retention efforts deliver the skills and experience needed, down from 75% in a similar survey from one year ago.
Also: OpenAI bets $38B on Amazon's cloud to power ChatGPT and the future of AGI
Still, market pressure is on, and companies are forging ahead with plans for AI and AI agents. Many CDOs (83%) suggested the potential benefits of deploying AI agents in their organizations outweigh the risks, and 77% reported being comfortable with the outcomes from AI agents.
The top challenges limiting the use of enterprise data by AI included the following:
- Accessibility: "Slow response times to data requests by authorized users and low user satisfaction."
- Completeness: "Null or empty field percentages, missing data rates, and low mandatory field compliance rate."
- Integrity: "Limited data lineage tracking and inconsistent data entry across systems."
- Accuracy: "High error rates, percentage of incorrect data, and data validation failure rates."
- Consistency: "Inconsistent data formats, including code and nomenclature."
Wider evidence suggests there may be cutbacks or stagnation in IT budgets, but not when it comes to AI data, as money is flowing to AI and AI agents. The IBM research found that 13% of a typical organization's IT budget is allocated to data strategy, up from just 4% in 2023. Most (81%) CDOs said their data strategy needs to be integrated with their overall technology roadmap and infrastructure investments, up from 52% in the previous year's survey.
A lot of data is prepared at the functional or project level, as noted by 81% of respondents. There is a strong demand for proprietary enterprise data.
Also: Stop saying AI hallucinates - it doesn't. And the mischaracterization is dangerous
If your organization is exploring new types of data and AI roles, or needs to understand the dynamics of AI support, the IBM authors provided the following recommendations:
- Use AI to improve AI: AI agents can manage the data lifecycle, from sourcing to retirement. AI agents can "autonomously cleanse data, detect anomalies and potential errors, and validate it against predefined rules and standards to boost accuracy," the IBM team pointed out. AI can also help predict upcoming data requirements.
- Open up data resources to the entire organization: Prioritize data democratization, the researchers urged. Democracy is the path to AI adoption and innovative experimentation.
- Develop a forward-looking data strategy: Learn what data is available, what's coming next, and who needs it. Then ask how an AI system can benefit from this data. "Do this in partnership with other business and technology leaders," the authors added.
- Leverage unstructured data: The next generation of tools -- natural language processing, computer vision, and machine learning -- should be able to read text, images, videos, and audio files. This approach calls for multimodal integration to support all these types of data from different sources.
- Establish KPIs that link data initiatives to business outcomes: These outcomes may include "increased sales conversions, reduced customer churn, or operational cost savings. Implement regular ROI reporting that quantifies the financial impact of data projects, making it easier to secure continued investment."
- Advocate for data literacy across the organization: Every role, from the top to the bottom of the organization, should be a data role. The researchers said: "Ensure that all leaders understand the strategic value of data and support data-driven decision-making. Integrate data and insights across functions to improve engagement and efficiency."
- Invest in intuitive data interfaces and user-friendly analytics tools: These systems "can simplify the interaction with business data for non-technical users." A majority of CDOs (82%) agreed that data and AI systems must be easy to access and use.
English (US) · 
