New research from Uplevel reveals a gap between engineering leaders’ confidence in AI-driven software development improvements and their ability to measure its actual impact. According to the findings, 96% of engineering and technology leaders believe AI is contributing to positive development outcomes, yet only 40% use controlled comparisons between AI-enabled and non-AI teams to evaluate its effectiveness. The results highlight challenges organizations face in accurately measuring AI’s influence on software development productivity and performance.
These findings are part of Uplevel’s new report, “Would You Recognize AI ROI If You Saw It?,” based on an online survey of 428 engineering and technology leaders. The study explored how engineering organizations measure AI’s value and attribute performance improvements, as well as the barriers they face in realizing AI’s potential.
“Engineering leaders are seeing real benefits from AI, but confidence alone doesn’t tell us how much of that improvement is driven by AI rather than other influences,” said Joe Levy, CEO of Uplevel. “As organizations invest more heavily in AI, they need a clearer picture of its impact: one that shows where the gains are coming from, what’s potentially holding them back, and whether they’re translating into meaningful business outcomes.”
Confidence Outpaces Measurement
The survey revealed additional gaps in how engineering leaders measure and demonstrate AI’s value:
- Existing approaches dominate: Nearly three-quarters (74%) of engineering leaders have extended their pre-AI ROI measurement approaches to AI initiatives.
- AI-specific measurement lags: Just 4% are building new measurement approaches specifically for AI.
- Business outcomes take a back seat: Only 29% primarily evaluate AI’s value through business outcomes.
Faster Coding, New Bottlenecks
Speed was one of AI’s earliest selling points in software development—enabling engineers to generate code faster and automate routine tasks. And while AI has since expanded into other parts of the development process, organizations continue to look to it primarily to save time and improve efficiency.
The study found:
- The need for speed: 39% of engineering leaders identified speed as the primary factor they’d consider when determining whether AI creates value, more than any other area. This preference held across most seniority levels, including managers (41%), directors (40%) and VPs (40%).
- Faster coding is a major draw: When asked separately how AI primarily creates value, 39% of engineering leaders pointed to helping developers write code faster.
- Review creates new friction: Speed is the promise, but additional review work can complicate the payoff. This year, nearly three in 10 engineering leaders (29%) cited increased time reviewing AI-generated work as their biggest new source of friction over the past quarter—a higher percentage than for any other challenge. Uplevel’s 2025 research foreshadowed this roadblock: Two-thirds of engineering leaders had identified validating AI-generated outputs and quality assurance as a critical skill for their teams in the AI era.
“AI is making parts of software development faster, and we’ve always been told that time is money,” said Amy Carrillo Cotten, director of client transformation, Uplevel. “But faster coding doesn’t automatically mean faster or better delivery. As AI accelerates certain tasks, it’s also exposing where work gets stuck. Engineering leaders need to look at the entire delivery process—not just how quickly code gets written—to understand where improvements will make the biggest difference.”
The AI Maturity Continuum
Uplevel’s study shows that engineering organizations are at different stages of their AI journeys, from using AI to assist with individual development tasks to orchestrating autonomous agents across software development. Leaders rated their organizations’ maturity on a 5-point scale, revealing a broad distribution:
- 38% are in the earlier stages: This encompasses organizations that consistently use AI tools for tasks such as code completion (Level 1) or have begun using AI agents for specific, bounded areas, with limited organizational context (Level 2).
- 23% are in the middle: These Level 3 organizations configure AI agents with organizational context, such as codebases, conventions and runbooks. Engineers increasingly focus on reviewing, correcting and directing agent output.
- 38% are at more advanced stages: These organizations develop, maintain and share AI agents and capabilities across teams (Level 4). At Level 5, they orchestrate agent teams that deploy software independently, with development workflows restructured around AI and minimal human intervention.
Obstacles Shift Across AI Maturity Levels
When identifying the single biggest obstacle to getting more value from AI, engineering leaders pointed to a range of challenges, with no one issue dominating.
Governance, security and compliance concerns topped the list (24%)—followed by gaps in engineering skills or AI literacy (21%) and missing context, such as documentation, data or institutional knowledge (also 21%). Workflow friction and infrastructure limitations each accounted for 17%.
But the obstacles looked different across the five AI maturity levels:
- Earlier stages: Missing context holds organizations back. At Levels 1 and 2, missing context was the biggest impediment to getting more value from AI (25% and 27%, respectively). At Level 3, no single obstacle clearly dominated.
- Advanced stages: Governance and skills are bigger hurdles. Gaps in engineering skills and AI literacy (26% at Level 4; 30% at Level 5) and governance, security and compliance concerns (28% at Level 4; 26% at Level 5) emerged as the largest barriers to realizing AI’s potential.
Explore the Full Findings
For more information on how engineering organizations are measuring AI’s impact, navigating challenges, and identifying where AI creates value, please visit https://uplevelteam.com/resources/ai-leadership-survey-2026
Related News:
Gravitee Report Reveals AI Agent Adoption Outpacing Security Governance and Controls
Self-Managed Keeper Secures Critical Infrastructure with Zero-Trust PAM
