AI KPIs: how to measure the ROI of AI projects

In short: to measure the ROI of an AI project you need four families of KPIs: efficiency, quality, revenue and adoption. Set each one before launch, with a baseline, a numeric target and a deadline. ROI then compares the measured benefits with the full cost of the project, including licences, API usage and internal hours.

Key takeaways

• An AI project without a baseline is impossible to defend: measure the process as it runs today, then introduce the tool.
• Technical metrics such as accuracy and latency tell you whether the AI works, not whether it pays off. Read them alongside efficiency, revenue and adoption.
• Every KPI should be SMART: specific, measurable, achievable, relevant and time-bound.
• Hours saved only count as ROI when they are redeployed to valuable work or remove a real cost.
• Three to five KPIs per project are enough. More than that and nobody reads the dashboard.
• To credit a result to AI you need a credible comparison (control group, staggered rollout or A/B test) and a counter-metric next to every KPI.

Why measuring AI value matters now

AI adoption among European businesses is moving fast. Eurostat reports that in 2025 almost 20% of EU enterprises with 10 or more employees used at least one AI technology, up from 13.48% the year before. The gap by size is wide: 55% of large enterprises against 17% of small ones. In Italy, Istat puts the 2025 figure at 16.4%, double the 8.2% recorded in 2024.

Another Istat figure shows the measurement problem clearly: among Italian companies using AI, the share that cannot point to any specific business purpose rose from 15.5% in 2024 to 33.4% in 2025, mostly small firms. Among companies that considered AI but did not adopt it, the most cited obstacles were lack of skills (58.6%), unavailable or poor-quality data (45.2%) and high costs (43.0%). A defined purpose, reliable data and controlled costs are exactly what a KPI system forces you to pin down.

When adoption doubles in a year, plenty of projects start out of fear of falling behind rather than from a clear goal. Anyone working with online stores or B2B companies has seen the outcome: active subscriptions, a few automations that run, and nobody able to tell the board or a business partner what they return. KPIs exist to turn "it seems useful" into a number that can support a budget decision.

What are AI KPIs?

AI KPIs are indicators that show whether an AI solution delivers the business result it was introduced for: less time spent, fewer errors, more sales or more people actually using it. They are different from model metrics, which describe how the algorithm performs.

Three definitions worth pinning down:

• A KPI (key performance indicator) is a numeric measure tied to a business objective, with a threshold to reach by a set date.
• A baseline is the value of that KPI before the change, measured with the same method you will use afterwards.
• ROI (return on investment) is the ratio between the net benefit generated and the cost incurred, expressed as a percentage.

A practical example: for an AI assistant in customer service, "answer accuracy" is a model metric, while "share of tickets closed without a human" and "average handling time" are KPIs. The first tells you the system is reliable; the others tell you whether it saves money.

Why aren't traditional KPIs enough for AI projects?

Because an AI project can fail in places that classic KPIs do not see. ROI and productivity remain the end goal, but an AI system adds three risks that need their own measures.

1. Quality can slip unnoticed. A model that misclassifies 10% of leads raises no alarm. It simply sends sales reps after the wrong people. You need regular human spot checks.
2. People may ignore the tool. A licence that is paid for and unused has negative ROI by definition, so adoption rate is a KPI in its own right.
3. Improving one metric can hurt another. In its guide to generative AI KPIs, Google Cloud gives the example of a more engaging chatbot that raises cart size but also increases time-to-cart, a metric retailers previously worked to keep low. KPIs have to be read together, never one at a time.

Then there is risk: reliability, security, transparency. The NIST AI Risk Management Framework, a voluntary framework published in 2023, devotes one of its four core functions (Govern, Map, Measure, Manage) to measurement. If that side interests you, I cover it in the article on AI governance and explainability for businesses.

Which KPIs should you use to measure AI ROI?

Think in four families and pick one or two indicators from each, depending on the use case. The table gives definitions and examples for an online store and a B2B company.

Family	What it measures	E-commerce example	B2B example
Efficiency	Time and cost to complete a task	Average handling time per ticket; waiting time before the first reply; minutes to write a product description	Hours to prepare a quote; time to first response on a new lead
Quality	Errors, accuracy, satisfaction	Share of correct answers in spot checks; first-contact resolution; CSAT after a chat	Lead classification accuracy; data errors entered into the CRM
Revenue	Effect on sales and margin	Conversion rate; average order value; recovered carts	Lead-to-opportunity rate; sales cycle length; revenue from reactivated accounts
Adoption	How much the tool is actually used	Share of conversations handled by the assistant; share of visitors who engage with the assistant; weekly use by the support team	Sales reps active on the tool each week; share of quotes produced through the AI workflow

Alongside these four families you need a few technical guardrail metrics. They do not go into the ROI calculation, but they protect it: system response time, error rate, uptime. If the assistant takes 20 seconds to reply, adoption will collapse before you can measure any savings.

One finding shows why segmenting matters. A National Bureau of Economic Research study of 5,179 customer support agents found that an AI assistant raised issues resolved per hour by 14% on average, by 34% for novice and lower-skilled agents, with minimal effect on the most experienced ones. Look only at the average and you will miss where AI earns its keep and where it is just a cost.

Which technical metrics keep AI quality in check?

Technical metrics do not go into the ROI calculation, but they are the alarm system that protects it. Which ones to use depends on what the AI produces. Google Cloud's guide to generative AI KPIs separates systems with bounded outputs (a category, a product, a yes or no), which can be compared against a set of correct answers, from generative systems with open-ended outputs (text, chat replies, code), which have to be judged against descriptive criteria.

System type	Example	Metrics	How it is measured
Bounded outputs	Lead classification; product search in the catalogue	Precision (how many suggested results are relevant); recall (how many relevant results were found); F1 (a balanced average of the two)	Comparison against a sample of hand-labelled cases
Generative outputs	Chatbot replies; product descriptions; draft quotes	Score on a rubric (for example 0 to 5) for correctness, groundedness in sources, instruction following and safety; win rate in a pairwise comparison with the previous version	Human review of a sample; optionally a model-based auto-rater calibrated against human judgements
Infrastructure	Any AI application in production	Response time; request error rate; uptime; time to retrieve data from the CRM or catalogue	Application and platform logs

Two practical points. First, Google Cloud describes auto-raters as still somewhat experimental and notes that many organisations rely on human evaluators; for a typical business, a fixed sample of cases reviewed every month with the same rubric is the simplest way to spot a drop in quality. Second, AI quality does not stand still. Catalogues, price lists and customer questions change, which is why the guide also lists the share of models monitored for shifts in data or performance degradation. In practice: repeat the same spot check every month and compare it with the one you ran at launch.

How do you calculate the ROI of AI in a business?

The formula is the same as for any investment: ROI = (measured benefits − total costs) / total costs × 100. The hard part is counting every cost and putting an honest value on the benefits.

Total costs include licences and subscriptions, API consumption (tokens), development and integration with existing systems (CRM, store platform, ERP), internal hours for testing and training, and ongoing maintenance and review of the outputs. Benefits come in three kinds: costs avoided, extra revenue and reduced risk. Risk is the hardest to quantify; if you have no credible method, leave it out of the calculation and mention it separately.

Take a hypothetical scenario with illustrative numbers. A furniture online store handles 2,500 support tickets a month, at an average of 8 minutes each. It introduces an AI assistant that resolves 35% of tickets on its own (875 a month). That saves roughly 117 hours a month; at a fully loaded cost of €25 an hour, about €2,900 a month or €35,000 a year. The project costs €12,000 to build and €800 a month to run, including API usage: €21,600 in year one.

• Year-one ROI: (35,000 − 21,600) / 21,600 ≈ 62%.
• Payback on the initial build: 12,000 / (2,900 − 800) ≈ 6 months.

One caveat changes everything: those 117 hours are worth €35,000 only if the support team uses them for something else (assisted selling, complex returns) or if they avoid a planned hire. If they simply become slack, the real benefit is zero. Put that in writing in the business case before you start.

How do you value benefits that are not revenue?

Many AI projects deliver real but indirect benefits: happier customers, fewer errors, staff who stay. The rule does not change: only what you can turn into money with a documented method goes into the ROI; everything else is reported separately as an indicator.

• Customer satisfaction and churn. Google Cloud notes that there is often a strong inverse correlation between customer satisfaction (CSAT) and churn: as the first rises, the second tends to fall. For an online store with repeat customers, or a B2B company with annual contracts, lower churn means retained revenue you can calculate from average customer value.
• Errors avoided. Multiply the errors avoided by their average cost: a return caused by a wrong product description, a B2B order that has to be redone, a credit note. Take the average cost from your own data rather than guessing.
• People who stay. Besides the productivity gain, the NBER customer support study found that AI assistance improved customer sentiment in conversations and increased employee retention. Staff turnover has a concrete cost (recruitment, training, ramp-up time) that you can work out from your HR data.
• Reduced risk. Security incidents, compliance errors, service outages: if you have no history to estimate their frequency and cost, leave them out of the calculation and report them as a qualitative benefit.

How do you know the improvement really comes from AI?

It is the first question a business partner or finance director will ask, and a simple before-and-after comparison often cannot answer it. If you launch a chat assistant in October and sales rise in November, part of that growth may come from seasonality, a promotion or a campaign that started at the same time. There are four ways to isolate the effect, from the simplest to the most rigorous.

1. Before and after, adjusted for seasonality. Compare with the same period last year, not the previous month, and log every other initiative running in parallel (promotions, campaigns, new price lists).
2. Control group. Part of the team keeps working without AI for a few weeks: for example, half the sales reps qualify leads with the new workflow and half with the current method. The difference between the two groups over the same period is the effect of the tool.
3. Staggered rollout. Switch the tool on for different groups at different times and compare those who already have it with those who do not yet. This is the approach of the NBER study mentioned above, which analysed exactly that kind of gradual introduction among support agents.
4. A/B test on the website. For an online store this is the cleanest option: half the visitors see the assistant or the AI recommendations, half do not, and you compare conversion rate and revenue per visit.

Whichever method you choose, pair every main KPI with a counter-metric: an indicator that tells you whether the improvement carries a hidden cost.

Main KPI	Counter-metric	Why
Share of tickets closed by the assistant	Reopened tickets; CSAT; requests to speak to a person	A ticket closed badly comes back and costs twice
Conversion rate with AI recommendations	Return rate; average margin per order	Selling more of the wrong products means more returns
Time to prepare a quote	Quotes corrected after sending; acceptance rate	A fast but wrong quote loses the deal
Qualified leads per day	Lead-to-opportunity rate; sales reps' assessment	More "qualified" leads are useless if they never become opportunities

For more AI use cases in online selling and retail, see the article on AI and retail.

How do you make AI KPIs measurable with SMART criteria?

A well-written KPI is one where two different people looking at the same data reach the same number. SMART criteria help you write it that way.

1. Specific: "improve customer service" is not a KPI; "reduce average first-response time in chat" is.
2. Measurable: a data source (CRM, GA4, assistant logs, help desk) must record it automatically. If you have to count it by hand every week, you will stop.
3. Achievable: the target starts from your real baseline, not from a case study you read on LinkedIn.
4. Relevant: it must link to something the budget holder cares about, meaning cost, margin or growth.
5. Time-bound: each KPI has a review date, usually 30, 60 and 90 days after launch.

Here is what a baseline and target table looks like for a hypothetical B2B scenario: a components manufacturer using an AI workflow connected to its CRM to qualify inbound leads. The numbers are illustrative; yours need to be measured.

KPI	Family	Baseline	90-day target	Where it is measured
Time to first response on a lead	Efficiency	26 hours	4 hours	CRM (lead creation and first contact timestamps)
Sales hours per month spent on unqualified leads	Efficiency	60	30	Activities logged in the CRM
Correct classifications in a sample of 50 leads	Quality	Not measured	At least 85%	Monthly review by the sales manager
Lead-to-opportunity rate	Revenue	12%	15%	CRM pipeline
Sales reps using the tool every week	Adoption	0%	80%	Access and usage logs

Note the sequence: efficiency and adoption show up within weeks, while revenue needs at least one full sales cycle. In a B2B business with six-month deals, the 90-day revenue KPI is a leading indicator, not the final result. For the wider picture on AI value inside day-to-day operations, see the article on enterprise AI and operational context.

How do you measure AI projects that are already running?

Many companies are not starting from scratch: they already have subscriptions, assistants and automations that have been running for months without KPIs. There is no need to stop everything. A five-step review is enough.

1. Take an inventory. List every AI tool in use, the process it touches, who uses it and what it costs per month in licences, API usage and maintenance.
2. Check real usage. Pull active users and frequency of use from the logs. According to Google Cloud, the shape of the adoption curve points to the cause: if it stays low from the start, people usually do not know about the tool; if it starts high and then drops, the tool is not performing well enough.
3. Rebuild a baseline from historical data. Your CRM, help desk, GA4 and ERP hold the data from the months before launch: use it for the comparison, or set up a control group if the data is not enough.
4. Decide for each tool: keep, fix or drop. A recurring cost that still has no linked KPI after 90 days is a candidate for cancellation.
5. Review the KPIs every quarter. KPIs are not fixed: once a target is reached and stable, replace it with the next one, for example moving from adoption to quality and then to revenue. Start with a few indicators and let them evolve with the project.

How do you measure the value of AI agents?

An AI agent does more than answer: it runs a sequence of steps, picks the tools to use and completes a task, such as drafting a quote or updating an order in the ERP. That is why, according to a February 2026 Google Cloud guide on agents in production, the metrics used for chatbots (including a simple thumbs up or down) are not enough. These are the indicators that matter.

• Cost per successful task. Cost per run is misleading unless you pair it with the success rate: in the guide's example, an agent that costs $0.10 per run but fails half the time actually costs twice as much per useful outcome.
• Acceptance rate. How often the agent's work is accepted as is, accepted with edits or reverted. A revert is a more reliable signal than explicit feedback, which people rarely give.
• Time to verify. The gap between the agent finishing its work and a person approving it. If checking takes longer than doing the task by hand, the agent does not pay off.
• Human interventions. How often someone has to take over a task the agent started. A high rate points to a trust or reliability problem.
• Path reliability. Whether the agent picks the right tool, follows the planned order of steps and avoids inventing the data it passes to your systems, such as a product code that does not exist.

A B2B example: an agent that drafts replies to requests for proposal should be measured on time per proposal sent, the share of drafts sent with few edits and the time sales reps spend checking them. With agents, variable costs also grow with usage; I cover this in the article on AI agents and SaaS models.

How to apply it in your business: first steps

This is the path I recommend to an owner or manager who has to pitch an AI project and defend it three months later.

1. Pick one process with a visible cost. Customer support, lead qualification, product descriptions, quotes. For a map of the processes that fit best, start with the article on business process automation.
2. Measure the baseline for at least four weeks. Times, volumes, errors, conversions, using the same sources you will use after launch.
3. Write three to five SMART KPIs, at least one each for efficiency, quality and adoption, plus a revenue KPI if the process touches sales. Decide who reviews them and when.
4. Calculate total cost and break-even before you sign: licences, API usage, integration, internal hours. If break-even is more than 18 months away, shrink the scope.
5. Feed the data into an automated dashboard (CRM, GA4, assistant logs) and schedule reviews at 30, 60 and 90 days, with a rule to pause or fix the project if quality KPIs fall below threshold.

If you are still deciding where generative AI makes sense for you, start with the article on generative AI and business strategy. More use cases are in the AI for business section of the blog, and the systems I build to measure and automate these workflows are described under solutions.

Mistakes to avoid

• Starting without a baseline. After launch you can no longer reconstruct how the process worked, so every improvement becomes a guess.
• Measuring only the technology. Model accuracy and speed do not pay invoices. They protect business KPIs; they do not replace them.
• Measuring activity instead of outcomes. How many messages the chatbot handled or how many drafts the agent wrote says nothing about value. What counts is how much time or money it saved, and at what quality.
• Mistaking efficiency for the goal. Google Cloud observes that many organisations confuse operational efficiency gains with end-state goals. Answering faster matters only if it leads to more sales, lower costs or customers who stay.
• Counting hours saved as cash in the bank. Freed-up time is only worth something if it is redeployed or removes a cost.
• Forgetting recurring costs. API usage, prompt maintenance and human review add up every month, even when the project looks finished.
• Tracking too many indicators. With fifteen KPIs nobody opens the dashboard. Three to five, read together, are enough.

Frequently asked questions (FAQ)

What are the most important KPIs for an AI project?

It depends on the process, but every project should have at least one KPI in each of four families: efficiency (time or cost per task), quality (errors or satisfaction), revenue (conversion or margin) and adoption (real usage by the team). Add technical guardrail metrics such as system response time and error rate.

How do you calculate the ROI of AI in a business?

Subtract total costs from measured benefits, divide by total costs and multiply by 100. Costs include licences, API usage, integration, internal hours and maintenance. Benefits are the costs avoided and the extra revenue measured against the baseline.

How long before you can measure the ROI of an AI project?

Efficiency and adoption KPIs can be read after 30 to 60 days. Revenue KPIs need at least one full sales cycle, which in B2B can take several months. That is why it makes sense to review at 30, 60 and 90 days and run a full assessment at 12 months.

What if I have no baseline for the process?

Before switching the tool on, measure the current process for at least four weeks with the same data sources you will use afterwards. If the project is already live, use a control group: part of the team or customer base continues without AI and you compare results over the same period.

Do hours saved with AI count as financial return?

Only when they turn into a real saving or extra revenue. Freed-up hours are worth their fully loaded cost if they are redeployed to work that creates value, or if they avoid a hire or an outsourcing contract. If they go unused, their value in the ROI calculation is zero.

How many KPIs does an AI project need?

Three to five per project. With fewer than three you lose sight of quality or adoption; with more than five the dashboard becomes hard to read and nobody checks it. What matters is reading them together, because improving one indicator can worsen another.

Which KPIs should an online store use for an AI chatbot?

For efficiency and quality: containment rate (conversations resolved without a person), average handling time, first-contact resolution and CSAT after the chat. For revenue: conversion rate and revenue per visit for sessions with chat, measured with an A/B test. As counter-metrics: requests to speak to a person, reopened tickets and returns.

Sources

• Eurostat, Use of artificial intelligence in enterprises
• Istat, Imprese e ICT, 2025 (press release, 15 December 2025)
• Brynjolfsson, Li, Raymond, Generative AI at Work, NBER Working Paper 31161
• Google Cloud, Measuring gen AI success: a deep dive into the KPIs you need (2024)
• Google Cloud, The KPIs that actually matter for production AI agents (2026)
• NIST, AI Risk Management Framework

If you need to justify an AI project and want to start from the right KPIs and baseline, request a free diagnosis.