Tackling AI sprawl in the modern enterprise

As enterprise AI becomes more embedded into the fabric of everyday tools, the biggest challenge facing organizations isn’t AI adoption; it’s AI management. Gone are the days when AI features like meeting transcriptions or document summarization stood out as cutting-edge.

Today, they are expected. According to McKinsey’s 2024 State of AI report, 72% of organizations have adopted at least one form of generative AI, and over half report using it in more than one business function. But this surge in adoption has led to a new operational crisis: AI sprawl.

Tony Gentilcore Social Links Navigation Co-founder and Product Engineering Lead at Glean.

What Is AI Sprawl and Why Does It Matter Now?

AI sprawl is the unchecked proliferation of AI tools and systems across departments, applications, and infrastructure without a unified strategy. The result? A chaotic digital ecosystem where redundancy is rampant (e.g., multiple summarization tools embedded in different apps), user experiences are inconsistent, data governance becomes unmanageable, and security vulnerabilities go undetected.

For example, companies eager to integrate AI across their tech stacks often deploy similar capabilities in silos—an AI assistant in a messaging platform, a different one in email, another in help desk software—without a shared interface or policy layer. This fragmented approach increases operational costs, confuses users, and makes compliance audits a nightmare.

The Rise – and Limits – of Vertical AI

Most enterprise AI today is what we call “vertical AI”: narrow capabilities embedded directly into a specific tool, often by that tool’s own vendor. These AI features are excellent at solving bounded problems but struggle at scaling across workflows or departments.

IDC research notes that organizations are spending up to 30% more per seat due to overlapping AI functionality across their application ecosystems. While each solution may serve a use case in isolation, collectively they add inefficiency and cost.

The Real Cost of Fragmentation

Here’s where AI sprawl hurts the most:

• Wasted Spend: Gartner estimates that up to 25% of enterprise AI investment is duplicative, particularly in use-case specific tooling.
• Poor AI Literacy: Employees have to relearn how to interact with each tool’s AI assistant, eroding trust and slowing adoption.
• Regulatory Risk: Privacy settings and data policies vary app by app, creating blind spots for security teams and legal counsel.
• Broken Context: AI models can’t share knowledge between systems, meaning insights are trapped inside individual tools.

A Smarter Alternative: Interoperability as Strategy

Instead of asking, “How many AI tools do we have?” CIOs and CTOs must ask, “How well do our AI systems work together?”

Interoperability means more than just integrations or connectors; it requires AI tools that can share context, adhere to consistent governance, and surface insights across platforms. This horizontal approach avoids the trap of buying more features and focuses instead on making those features work in concert.

Three Core Benefits of AI Interoperability

• Holistic Intelligence: AI-driven insights from one tool (e.g., CRM) can inform decisions in customer support, marketing, and HR when systems talk to each other.
• Trustworthy User Experience: Employees get consistent behavior, language, and recommendations regardless of the app they’re using.
• Centralized Oversight: IT and security teams can manage data policies, model updates, and risk controls from a single pane of glass.

Charting a Coherent Path Forward

To navigate from fragmentation to function, enterprise leaders must pursue both operational alignment and robust governance practices. The good news is that AI sprawl is not an inevitable cost of innovation—it can be addressed proactively.

By taking a strategic approach that blends centralized governance with interoperable infrastructure, organizations can rein in AI fragmentation before it becomes unmanageable. The way forward is clear, actionable, and within reach.

• Build a centralized AI governance council that includes IT, compliance, legal, and business users.
• Define enterprise-wide AI usage policies and audit mechanisms to ensure consistent, responsible practices.
• Implement monitoring tools that track model performance, data lineage, and access across platforms in real time.
• Consolidate and rationalize tools to eliminate duplicative spend and improve oversight.
• Audit the AI landscape by inventorying every AI-enabled tool, feature, and data dependency across the organization.
• Invest in AI infrastructure by adopting open standards like MCP, APIs, and orchestration platforms that promote interoperability.
• Upskill employees through literacy programs that demystify AI, reduce risk, and build trust in intelligent systems.

In fragmented environments, IT and compliance teams are often required to support multiple incompatible permissioning models, audit trails, and deployment protocols. A centralized platform enables governance teams to monitor model performance and data lineage in real-time, reducing exposure while aligning AI use with evolving regulatory expectations.

Less Hype, More Harmony

Enterprise leaders need to stop chasing the next flashy AI feature and start focusing on cohesion, governance, and usability. The future isn’t about having the most AI; it’s about having the most effective, connected, and secure AI.

The maturity curve for AI adoption will increasingly reward organizations that move beyond fragmented experimentation. Those who consolidate capabilities and embed AI within core processes will unlock sustainable growth, resilience, and competitive advantage.

In the age of ubiquitous AI, everyone has tools, but not everyone has traction. The innovators aren’t the ones with the most features; they’re the ones who make it all work together. AI sprawl may be a modern challenge, but orchestrated intelligence is the competitive edge of tomorrow.

We list the best employee experience tool.

This article was produced as part of TechRadarPro’s Expert Insights channel, where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing, find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro.

Original Author: Tony Gentilcore | Source: TechRadar