| Best Practices |
|
The
Impact of Forecasting Improvement on Return on Shareholder Value
|
| Conferences & Events |
5th Annual World Class Sales and Forecasting Management Conference, |
| Performance Management: Use It or Lose Out! | ||
|
By Anders Gjerde, Senior Manager, Business Development, Steelwedge Software |
||
|
As we say in Norway, "if you don’t know where you came from, how do you know where you are going?" This piece of ancient conventional wisdom appears to be true in forecasting and planning as well. The proverbial "You Are Here" is Performance Management, which is your point of departure on the roadmap to continuous forecasting and planning improvement.
Performance Management is the measurement of past performance in order to improve future forecasting and planning accuracy. This is a corner-stone of any forecasting and planning process. Surprisingly, however, few companies are able to effectively use Performance Management as a tool to improve business performance. Indications of suboptimal Performance Management include:
Performance Measurement at a Technology Company
Making Performance Management an integral part of a continuous improvement process requires a flexible, analytic framework that empowers all users and stake-holders to focus on metrics that are useful to them – metrics that can be acted on. Creating good performance measurements is not a “one size fits all” proposition – they must be crafted to fit the tasks they support. A simple example: Measuring quarterly corporate revenue forecasts can be done by adding forecasts and actual sales for each quarter, whereas someone working on monthly operational forecasts probably would not find quarterly aggregations particularly useful. In the technology company, senior management complained that even though forecast accuracy was excellent, they had problem with on-time delivery as well as excess inventory. Each month, management received a performance report like the one below: 
As can be seen, the forecast appears to be very accurate. However, when taking a closer look at how the numbers were generated, Steelwedge Software found that their MAPE (Mean Average Percent Error) was computed from aggregate data. This is a very deceiving performance measure. The problem with their report was that by adding different items within a family, over- and under-forecasted items would offset one another, and the computed forecast error was small even though the mix was way off. A surplus of apples doesn’t help a deficit of oranges. Steelwedge proceeded to build a new report, calculating errors for each item at detail. Management did not have time to review each item in detail, so to avoid information overload, the errors were added up as a weighted average – the total MAPE was then a one-number error that provided a clear picture of how well the overall product mix was forecast. The report also included the ability to slice the data in a variety of ways, like selecting specific product families, products for each forecast analyst, and the ability to include or exclude specific items. Using the tool for analysis, users could also drill down to view the performance of individual items. This provided all users with a unified mechanism of viewing performance metrics at levels that made sense to them, both conceptually and operationally. As the company started to use the report, they realized that aggregation is sometimes the right thing to do, such as when an over-forecast in one period is offset by an intentional under-forecast in the following period. Without aggregating these numbers, the performance measure would look much worse than it really was. Hence, the performance report was augmented to include a combination of non-aggregate and aggregate metrics. In the end, it provided a rich, yet manageable set of metrics on business performance. 
The above report would be more powerful if it also included the ability to separate systemic errors from random errors. Systemic errors are those that can be corrected; random errors cannot. Examples of systemic errors are bias and lead/lag errors(2). For the technology company, the analysis framework was augmented to include reports like the one below; the first graph shows bias within a given month – how many items were over- and under-forecast? The second picture shows systemic over- or under-forecasting across time – how many items were over- or under-forecasted in all time-periods, or all time periods except one? Bias over time is particularly useful towards identifying corrective actions. In the below example there is a clear tendency to over-forecast. Armed with this knowledge and the ability to drill down to the affected items, management could effectively direct corrective actions. 
The technology company was also wrestling with the issue of when to measure their forecasts. That is, what would be the best lag time for measuring forecasts? For example, measuring the forecast for December created back in September is equivalent to a three month lag. The benefit of a fixed lag time is that it puts all forecasts on a level playing field. An alternate approach would be to measure the October, November and December forecasts as they stood in September – this means that the October forecast had a one month lag time, the November forecast had a two month lag time, and the December forecast had a three month lag time. For the technology company, both approaches were used – they are both valid as long as they are used consistently and are designed based on their intended use. The fixed lag time approach was used for measuring forecast performance, whereas the flexible lag time approach was used for measuring the quarterly revenue forecast. For the fixed lag-time measurement, the lag time was set to two months, which was roughly the lead time for making resource commitments. In the end, creating a set of performance metrics that fit the company’s needs turned out to be a challenging, multi-faceted undertaking, but the insights gained during the process and the resulting forecast improvements were substantial: The company improved their product mix accuracy by 30% in the first 2 or 3 months, resulting in substantial savings in inventory carrying costs and a 8% improvement in on-time delivery.
Performance Management as a Business Process
In working with this technology company, Steelwedge also reviewed Performance Management as a business process. As we saw above, the company used to equate Performance Management with a standard, monthly report prepared by the forecasting group and reviewed by management. These reports fulfilled one important objective of performance management -- to give management a general notion that things are (a) stable, (b) improving, or (c) deteriorating. However, in this case, even though Performance Management – on paper-- was part of the planning process, it was a mute process step because the information was not incorporated back into the forecast. That is, the recipients of the reports were removed from the actual forecast development, and the information was not clear enough to help them take corrective actions. Another process issue was who should prepare the performance analysis? If the people who generate forecasts are evaluated solely on the basis of their forecast accuracy, this could be a conflict of interests. Said one manager: “...It’s like getting a self-prepared monthly account statement from my son in college. I can see if the balance is improving or deteriorating, but I cannot find out where the money is spent, if he really needs the money, or if the stated account balance reflects what’s actually in the account.” To deal with this issue, and instead of creating a separate forecasting audit function, the company modified how they evaluated forecasts: They pegged forecast accuracy measures against inventory levels and on-time delivery. In so doing, they aligned the interests of the people developing the forecasts with the interests of the company – to improve business results. The forecasting process in this technology company went through a chain of reviews, where each stake-holder could (and was expected to) modify the forecast. To provide specific feedback, performance analysis reports were created for each stake-holder’s forecast based on where they started and where they ended up. In summary, the company implemented the following process improvements to make Performance Management an integral part of a continuous improvement effort:
Conclusions
Making Performance Management part of a continuous improvement process can yield great benefits, but it requires a thorough review of both measurements used and how performance management fits into the overall business processes:
Footnotes (1) Name and numbers are obscured to protect the innocent. (2) Lead/lag errors: Forecast timing is off – for example a demand peak is forecast correctly, but in the wrong period.
About the Author
Anders Gjerde is a Senior Manager and Business Development Analyst at Steelwedge Software. Since joining Steelwedge in 2002, he has worked with customers to implement innovative solutions to help them solve a wide range of planning and performance management problems. Prior to joining Steelwedge, Anders was Director of Global Client Solutions at Decision Focus/Talus Solutions (acquired by Manugistics, Inc in 2001). Anders holds an MBA from the Norwegian School of Economics and Business Administration. |
||
3825 Hopyard, Suite 155, Pleasanton, CA 94588
Copyright 2005 STEELWEDGE, Inc. All rights reserved.