A Metric Sherpa study produced with Sprinklr found that 92% of organizations regularly using customer-facing AI reported satisfactory or better performance. Yet only 27% said they could demonstrate those results. For marketing leaders, that gap raises questions about how deployment decisions connect to business performance. Positive user assessments can support adoption, but they offer limited grounds for allocating budgets or expanding applications.
Measurement Readiness Trails Deployment Confidence
The study’s strongest structural signal is the distance between stated objectives and the preparation needed to measure them. Metric Sherpa found that 99% could identify their intended outcome, while 26% had specified measures of success. Just 10% had also established a starting benchmark.
Those findings suggest a problem extending beyond individual tools or disappointing implementations. Teams can articulate what they want technology to achieve without establishing how they’ll judge its contribution.
Cost claims illustrate the consequence. Among respondents reporting faster work, 67% also reported savings, the study found. Within that group, only 32% had measured costs before deployment.
Without that comparison, reported savings remain difficult to separate from perceptions of efficiency. For budget holders, faster production and lower expenditure require different evidence.
The figures support concern about measurement readiness across the surveyed organizations. They don’t establish that AI delivers little value, or that these shortcomings characterize every marketing team.
Channel Improvements Need a Business Reference Point
The broader marketing implication concerns how local improvements translate into commercial results. Conversion, engagement, and service containment each describe part of a customer interaction. None independently establishes whether an application improves the business outcome behind its deployment.
A hypothetical campaign could generate more conversions while leaving acquisition costs unchanged. Likewise, containing a service conversation could leave the customer needing another contact. These possibilities explain why platform reporting needs a reference point beyond the activity being optimized.
Metric Sherpa found that 85% reported gains within a measurement category they hadn’t monitored before deployment. Researchers couldn’t determine whether respondents introduced tracking afterward or drew evidence from elsewhere.
That uncertainty limits interpretation of the figure. It also reinforces the need to distinguish a newly visible metric from a demonstrated improvement.
Marketing operations can connect channel reporting with CRM records, financial information, customer outcomes, or experimental comparisons. The relevant connection depends on the original objective. Otherwise, platform activity risks becoming the default definition of success.
Outcome Ownership Supports Stronger Evaluation
Measurement also depends on someone answering for the commercial result. Metric Sherpa found that only 17% assigned responsibility for AI outcomes.
Organizations assigning that responsibility quantified results in 46% of cases, compared with 23% among organizations without it. They also reported greater use of controlled comparisons and financial reporting.
The study doesn’t demonstrate that assigning an owner produced better measurement. Better-managed organizations might adopt both practices together. Still, the association supports treating responsibility for results separately from maintaining a tool or running a process.
For one application, a marketing leader can establish a review with three concrete requirements:
- Record the intended business outcome and its pre-deployment benchmark.
- Specify the comparison and performance threshold needed to judge progress.
- Assign responsibility and document criteria for expansion, revision, or withdrawal.
This brings measurement into the investment decision rather than leaving it as a reporting exercise.
The Evidence Warrants Discipline, Not a Verdict on AI
Several findings leave room for less pessimistic interpretations. Missing benchmarks don’t prove that reported benefits are imaginary. Later measurement or another information source could explain some claims.
More rigorous evaluation can also reveal additional problems. Organizations using at least three measurement practices reported more handoff difficulties, adoption challenges, and friction involving data quality.
Researchers couldn’t distinguish greater visibility from greater deployment complexity. A program reporting more problems may therefore be examining performance more thoroughly.
Regular reviews remain uncommon: Metric Sherpa found that 37% scheduled them, while 28% had agreed criteria governing subsequent deployment decisions. This makes the practical priority clearer than any sweeping verdict about AI returns.
Marketing leaders need evidence connecting an application’s activity to its intended business result before committing further resources. The study supports that discipline; it doesn’t establish whether any particular application deserves expansion or closure.
