Our team had just hit the week of go-live and everything was running smoothly. We felt we had met all the requirements and UAT was going well. After a successful cut-over from the old system to the new, we were staffed for any issues that may occur.
Our biggest measure of success would be determined by average call time/call length. This client needed short call times in order to fulfill the massive amount orders they receive during the summer selling season, which started the week of go-live.
Over the next few days only very minor issues were brought to our attention. Call times were meeting the benchmarks and life was good. Our project was a success; or so we thought.
What we didn’t know was why call times were so short, sometimes shorter than last year’s pre-new system times.
The call centers was getting calls from customers, realized the system couldn’t perform the requested action, and told the customer to call back later when the problem was fixed.
Call times were short because the reps couldn’t perform their jobs and would just say “call back later”. That’s one way to hit your bench marks.
We were measuring the wrong criteria for success and in return we got false positives which delayed our action in putting in the proper solution.
It’s important to understand what you’re measuring and all its shortcomings before relying on a single statistics to determine success. This is a lesson I’ll carry with me for future projects to come.
I wrote about how management should respond in these sort of crisis situations last year around the time this story took place. It’s called Don’t Point Fingers.