Using the Right Metric for the Job
Its a management truism that you get what you measure, so it is important that you know the difference between the various types of metrics, and when to use each one. The four primary types are:
- Measures of Activity: Is the plan being executed? (Project Managers care)
- Project Results: Was the plan successful? (Managers care)
- Business Drivers: Did the results of the project have their intended effect on the business drivers? (Executives care)
- Strategic Outcomes: Did the effects lead to the desired outcome? (Shareholders care)
Measures of Activity
Any measure of progress or activity falls into this category. Measures factors directly under the control of, or affected by, the project team members. How many calls are being made per day? How many clients contacted? How many training courses are being run? Are we on budget? Are we on schedule? How many bricks have been laid, pages written, blogs posted? Project Managers measure these to ensure projects are being executed.
For ABC ‘s training initiative, we might track how many users are contacted, how many sign up for training, how many training sessions are performed. We might also look at quality metrics – trainer evaluations for example.
Measures the direct results of the project, initiative or whatever. It would be appropriate to set goals or targets for the project using one of these metrics – sometimes these can also be used as termination criteria (“once the 500 users have been trained, we can stop the project”), or success criteria (“if we get 250 people trained the project would have paid for itself, and can be considered a success”). Project Managers and Managers measure these to ensure that projects are successful.
For ABC’s training initiative the number of people trained or certified would be appropriate.
The business drivers are what you are attempting to influence with your project or initiative. Hopefully these are measurable, and are attributable to the results of the project. If you think of a Strategic Execution framework such as the “Balanced Scorecard”, then the business drivers relate to customer perceptions of brand, quality, relationships, price, functionality, timeliness and the internal process ones of cost efficiency and asset utilization. A suitable metric is one that you can measure reasonably accurately and in a timely manner. Senior Managers and Executives measure these to ensure that the Business Strategy is being executed.
For ABC’s training initiative we are targeting the customers’ perception of functionality – directly related to how much the service is being utilized. So a suitable metric for us would be usage rates. Our theory is that if users know how to use the service better (i.e. have been trained) then usage rates would go up.
What is the business trying to achieve with this strategy? Market Growth? Market Share? Increased revenue? Decreased costs? Sales, renewal rates, win-loss ratios … all of these have one thing in common – they are not under the control of the project team members, and they are lagging indicators. Additionally they are normally influenced by a great many things – your initiative being only one of them – and you won’t know if you have been successful on these metrics until quite possible a year or more has gone by. For this reason, these metrics need to be used with great caution. Executives, Board members, and Shareholders measure these to ensure that the Business Strategy is successful.
For ABC’s training initiative we are interested in renewal rates for users that have been trained. Unfortunately for us, we won’t really know this for at least a year….