Gartner is a prominent research company that offers opinions based on market trends and its own investigations. The bulk of its business, according to vice-president and fellow, Martin Reynolds, is the advice it gives clients as to which vendors and strategies they should consider. The firm's analysts make as many as 500 calls per year to enterprises and vendors, Reynolds said, giving each analyst deep expertise in a particular technology.
Gartner's advice is all about avoiding risk and escaping what Reynolds said was a common complaint among IT executives: "The system did exactly what we asked for, but it wasn't what we wanted."
As beneficial as Gartner, Forrester or AMR Research can be in the decision-making process, however, there is an alternative way to approach the challenge of deciding what technology to deploy. For that side of the story, I spoke with CEO at Nucleus Research, Ian Campbell.
Nucleus takes an ROI case-study approach. Its research is based on its analysts' assessments of recent deployments. "It's not what Nucleus thinks about a product, but what we've found about its use," Campbell said. "Return on ibvestment [ROI] is the unyielding metric."
Campbell's company researches hundreds of ROI case studies in various vertical industries. That's something Campbell claims even highly respected research companies don't always know how to do.
"There is only one correct way to calculate ROI," he said, noting that analyst organisations do it many different ways. For example, there's the concept of cumulative ROI, where you add up the benefits of a project over a 10-year period.
Campbell compared that to putting $100 in the bank and getting 10 per cent interest each year. Is it correct to say that over 10 years you received a 100 per cent return on investment? No. After 10 years, it's still 10 per cent.
"If I have a project that's a dog, all I have to do is go out long enough and it will be positive," Campbell said. He also warned to beware of vendor-initiated case studies. If you heard of some marquee enterprise-level company extolling the benefits of a given solution, Campbell suggested calling the company directly. You might hear quite a different story than what was written in the marketing brochure.
Furthermore, Campbell said to never look at TCO; it was just the cost side of an ROI equation. If you had to choose between two products - one that was a royal pain to deploy and required five people to manage versus one that required only one person - and then found that the pain-in-the-butt solution would bring in hundreds of millions of dollars more, which would you choose?
Finally, Campbell said that when you are looking at ROI, keep in mind that just because one customer achieves a good ROI doesn't mean another will. ROI is unique to a particular company. The only thing it measures is how big of a step a company was able to take: from how bad off it was to how much better off it is now.
Both approaches have value. In fact, Reynolds and Campbell both had good things to say about the other's methodology. If you want to obtain the most complete picture of a product or technology, you just might have to use both.