Dealing with uncertainty

How to include these soft metrics in a financial analysis has become a matter of much debate. Some companies don't include them at all because measuring them is thought to be too hard or subject to too much manipulation, says Nick Vitalari, an executive vice president at BSG Alliance. "Companies think they can't quantify the benefits side," he says, "but that's simply not true. You have to use some creativity."

Vitalari says one way that is becoming increasingly popular is to conduct "business experiments" -- which for IT could be system prototypes, possibly linked to test marketing -- before proceeding with a project that may or may not deliver value. Even fairly straightforward projects, where the costs and benefits are predictable, can gain from this type of business experimentation, he says.

Experimentation happens frequently at P&G, where there is not a single go/no-go decision point for a project. "More and more, we do quick pilots, so you do a lot of learning early on before investing heavily," says Scott. "Is this really something that will create value? Does the technology work? We call it an 'innovation funnel,' where the funnel is wide at the beginning but you have gates where you weed out options as the project progresses."

The Schwan Food uses NPV, IRR and payback period to evaluate IT projects costing more than $1 million. Cost estimates are developed by IT, but the benefits estimates come from the business units, says CIO Kate McNulty. When benefits are hard to quantify, IT works with business managers to help do that, she says.

Numbers with a great deal of uncertainty are dealt with in two ways, she says. They can be the subject of "sensitivity analysis. She explains: "We might ask, 'What if my benefits come in at only 75 per cent of what I have estimated -- what's my IRR? What if they come in at 50 per cent?'"

The second way is to compute an "expected value," the dollar amount of an outcome multiplied by the probability of its occurring. "If I say I'll get $100,000, and my confidence in getting it is 50 per cent, then that goes into the analysis as $50,000," she says.

Harrah's Entertainment uses three metrics to prioritize IT projects -- primarily NPV and secondarily IRR and Economic Value Added (essentially ROI less the cost of invested capital). Increased sales is usually the key benefit to be measured, but the business sponsor of a project works with IT to measure softer benefits such as increased guest visits at Harrah's hotels and casinos, customer satisfaction and even employee satisfaction.

The use of more than one metric has pros and cons, says Harrah's CIO Tim Stanley. "Using multiple criteria to assess a project provides a robust framework for decisions," he says. "Each tool takes into consideration the investment and the expected business value or return, and we are not limited to a single point of view." But, Stanley says, the prospect of sophisticated financial analyses can inhibit some people from submitting an idea for consideration.

Indeed, the work involved in doing any kind of rigorous financial analysis can be daunting, Gardner says. "It's a blend of corporate strategy and corporate finance and high technology," he says. "Each of those fields is fairly complicated, and when you combine them, you are dealing with some of the most challenging problems a company has.

"And incentives are not always there to come to the right answer," he adds. "The incentives are to get the system built."