As a product tester, I always tell people: The product speaks for itself. White papers, customer wins, marketing spin: None of that counts. I don't have to be convinced by a public relations person that the product is good, because good products prove themselves in our lab. In 2004, when I last tested mail security appliances, CipherTrust's IronMail was on our short list as a top finalist. It's a good product, and it proved itself in our labs.
There's something we didn't tell you about that test, though. After the test was underway, CipherTrust engineers logged on to our test bed (I had forgotten to close the hole in the firewall we had opened so that they could help with installation), changed the passwords on the IronMail system and shut it down. We found out about it a few hours later, not because they told us, but because our monitoring systems saw the outage. It was an unprecedented action on the part of a vendor. CipherTrust explained it as a miscommunication - they were monitoring the results and weren't getting the effectiveness they had expected, and someone panicked and ordered the shutdown. After some tense and angry negotiations, we put them back in the test. We didn't write about it because, well, "products speak for themselves."
I had forgotten about it until recently when a consulting customer hired me to help select an e-mail security appliance. CipherTrust was on their short list, and they asked us to do extended and intensive testing of a few products. We decided to buy some products, including the CipherTrust system, to give us freedom in our testing and reporting. That seemed easy enough, except that CipherTrust wouldn't let us buy a box.
The salesperson was ready to give us a local value-added reseller (VAR) so we could buy the US$5,000 unit. But then he passed me over to CipherTrust PR, which passed me over to the vice president of sales, who passed me to a fourth person so we could apply to be a member of their partner program. This was getting ridiculous, so I explained again that I simply wanted to buy a box for my own company to use. This time, silence. No reply.
After waiting a week, I found a VAR and ordered a system. Then the VAR called back: CipherTrust refused to fill the order. Why is CipherTrust unwilling to sell me a box? I don't know; they aren't talking.
More frightening than my experience is the possibility that the company might do this to an existing customer. What good is a security product if the vendor refuses to sell you service on it? Without updates, most of these products are barely useful as doorstops.
In our tests, we look at products, not companies. Things such as training, finances and corporate style don't come into it. But when it comes to buying products, our tests aren't enough. It's important to investigate all those peripheral aspects of the vendor before you sign a purchase order. I was reminded of that the hard way.
Snyder, a Network World Test Alliance partner, is a senior partner at Opus One in Tucson, Arizona. He can be reached at Joel.Snyder@opus1.com.