A company deployed an AI customer service system in January. By April, the CEO asked: "Is this working? Are we better?"
The head of customer service said: "Yeah, I think so. Response times feel faster. Customer feedback seems better."
That's where most companies fail at AI. Not at deployment. At measurement.
They can't answer: Is this working? By how much? How do we know? What gets better next?
This article tells you how to fix that. How to measure your current state. How to benchmark. How to track improvement.
The Three Numbers You Need
For any AI deployment, track three things:
1. The Baseline (Where You Are Now)
Before you deploy AI, measure your current performance. Not approximately. Precisely.
If you're automating customer service: How many inquiries per day? What's the average response time? What percentage get resolved on first contact? What's the customer satisfaction score?
If you're automating invoice processing: How many invoices processed per week? How many errors? How much time per invoice? What's the cost per processed invoice?
If you're automating recruiting: How many candidates screened? Time-to-hire? Cost-per-hire? Quality of hire (retention rate)?
Write these numbers down. Don't approximate. Don't estimate. Measure them for 30 days and get actual data.
2. The Target (Where You Want to Be)
Based on industry benchmarks and vendor claims, what's realistic?
An AI customer service system might reduce response time by 60% (industry standard). An AI invoice processor might reduce processing time by 70% and cost per invoice by 60%. An AI recruiting system might reduce time-to-hire by 40% and cost-per-hire by 30%.
But don't use vendor claims directly. Adjust down by 20% to account for reality. If the vendor says "60% improvement," plan for 48% as your target.
Your target for customer service: "Reduce response time from 6 hours to 3 hours (50% improvement, accounting for vendor optimism)."
Your target for invoicing: "Reduce cost per invoice from HK$50 to HK$20 (60% improvement)."
3. The Current Measurement (Monthly)
After deployment, measure the same KPIs monthly. Don't wait six months. Monthly. So you can see what's working and what's not.
Month 1: Response time dropped from 6 hours to 5.2 hours (13% improvement). Not as much as target but moving in right direction.
Month 2: Response time dropped from 5.2 hours to 3.8 hours (37% improvement). Good progress.
Month 3: Response time 3.1 hours (48% improvement). Approaching target.
Month 4: Response time 2.9 hours (52% improvement). Target exceeded.
Now you have a story. Month 1 rough, month 2-3 improvement, month 4 success. You can also point to why: "Month 1 was training and system tuning. Month 2 we optimized prompts. Month 3 team became fluent."
The Measurements That Matter
Different functions need different KPIs. Here are the ones that actually matter (not vanity metrics):
Customer Service:
- Response time (first reply)
- Resolution rate (% resolved on first contact)
- Customer satisfaction (CSAT score)
- Cost per resolved ticket
Recruiting:
- Time-to-hire (days from open to offer)
- Cost-per-hire (total recruitment cost / hires)
- Quality of hire (retention rate at 12 months)
- Hiring manager satisfaction (would they use this system again?)
Finance:
- Processing time per invoice
- Error rate
- Cost per processed invoice
- Days-to-payment (how fast invoices get paid)
Operations:
- Process cycle time
- Error rate
- Cost per transaction
- Capacity (number of transactions per hour)
Pick three KPIs per function. Don't track everything. Three metrics that tell the real story.
The Benchmarking Problem
Your baseline is internal. But what if your baseline is worse than industry standard?
You might be processing invoices slower than competitors. Your customer service response time might be worse. Your recruiting might be slower.
That's why external benchmarking matters.
Find out: What do companies like yours (same size, same industry) achieve?
For customer service: Industry average response time is 2 hours. You're at 6. You're below average.
For invoicing: Industry average is HK$30 per invoice. You're at HK$50. You're paying too much.
For recruiting: Industry average time-to-hire is 25 days. You're at 35 days. You're slow.
Now your AI improvement has context. If you improve from 6 hours to 3 hours on customer service, you're still worse than industry average (2 hours). So your target isn't "match vendor claims." Your target is "match industry average, then beat it."
That's ambitious. That's what drives real competitive advantage.
How to Track Progress Like a Leader
Once you have baseline, target, and monthly measurements, you need a system to track and communicate:
Dashboard (visual, updated monthly):
- Baseline (starting point)
- Target (goal)
- Current (month-to-date)
- Progress (% toward target)
- Trend (improving, flat, declining?)
Monthly narrative (one page, send to leadership):
- "Response time improved 13% this month (target: 50% by month 4)"
- "Resolution rate on first contact is now 78% (was 62%, target 85%)"
- "Cost per resolved ticket dropped HK$12 to HK$8 (target HK$5)"
- "What worked: Improved training for AI edge cases. What didn't: Prompt optimization took longer than expected. Next month: Focus on reducing escalations."
Quarterly business review (show stakeholders the impact):
- "Q1 summary: Customer service AI achieved 48% improvement in response time, 22% improvement in resolution rate, 28% reduction in cost-per-ticket."
- "This translates to: 200 fewer escalations per month, 8 fewer customer service reps needed, HK$80,000/month in labor savings."
- "Next focus: Expand to chat-based support and email automation."
That's how you communicate success. Not "it's working," but "here's the data showing it's working, here's the business impact, here's what comes next."
Why Most Companies Fail at This
They skip the baseline. They deploy AI without knowing their current performance. Then after three months, they try to measure impact but can't because they don't have a starting point.
"Are we better?"
"I dunno. We didn't measure before."
That's a wasted deployment. You spent HK$50,000 on AI and learned nothing about whether it worked.
Don't be that company.
Establish baseline before deployment. That takes 2-4 weeks of data collection. It's worth it.
Your AI Benchmark Score
Use UD's AI Rank tool at https://ai-staff.ud.hk/ai-rank to see where your company stands on AI adoption benchmarks compared to your industry peers. You'll get:
- Your baseline (where you are now)
- Industry average (where you should be)
- Top performer benchmark (where leaders are)
- Gap analysis (what you need to improve)
This external benchmark gives you context for your internal measurements.
The Bottom Line
If you can't measure it, it's not real. AI projects that lack clear metrics fail. Not because the technology fails, but because nobody knows whether to celebrate or fix.
Before you deploy AI, establish your baseline. During deployment, measure monthly. After deployment, communicate results.
That's how you know if AI is working. And that's how you prove the investment was worth it.
Email: sales@ud.hk | WhatsApp: (852) 9696 7545