Microsoft's decision to delay Office 2003 to squash more bugs could produce a more solid release, and may void the old joke about savvy consumers skipping the company's version 1.0 products.
Microsoft said it would "refresh" the current beta test version of Office 2003 before freezing the code, which would delay the suite's release until the third quarter. The upshot? The initial Office 2003 might be as stable as most Microsoft products after the first Service Pack update, a research fellow with Forrester Research, Rob Enderle, said.
"I can speak for myself and most of my clients: We'd rather have something come late and work than come on time and be a pain because it isn't really finished," Enderle said.
Both the analyst - who is among the estimated 600,000 beta testers - and Microsoft attribute the timetable change to the enormous feedback enabled by Microsoft's automated error-reporting system in the Office 2003 beta edition. The utility pops up whenever the software falters, inviting the user to send an automated report online to Microsoft.
The error report merely sends the failed code, and does not identify the user, program manager for Microsoft Office in the US, said. Automated error reporting was also part of the shipping versions of Windows XP and Office XP. Microsoft estimated that about one-third to one-half of the time, users agreed to send the feedback.
This was the first time it has been part of an Office suite beta, he said. And because the users are testing a pre-release product, they were more likely to click "send" to an error-report request.
"We've been trying to make it very, very easy for customers to take one of the most frustrating things that can happen with a computer, when an application fails, to get real-world feedback to us so we can address the problem," LeVine said.
Microsoft has also improved the error-reporting utility: it now permits users to delay sending the report if they're offline when it occurs.
Enderle said he was not aware of any other developers automating the error-reporting process, but expected competitors would adopt similar systems, given Microsoft's success.
"They undoubtedly captured a whole lot of problems they didn't expect," Enderle said.
Traditional beta testing relies on users taking the initiative to report problems. "With the automated method, they're capturing almost everything," he said.
Office entered beta 2 in March. It is a so-called public beta, available for download or on request by any interested parties. Microsoft expected about 500,000 testers.
"This is the largest beta test for Office," LeVine said. "And the larger we can make a beta, the more we can approximate real-world experience, and the more confidence we have that we're seeing the top issues and that the fixes we make really hit this on the head."
LeVine declined to specify the kind of bugs that were getting the most attention, but said they were distributed across the multi-application suite.
He said that "the 20 percent of most common defects accounted for 80 percent of the real-world problems" in the initial use of the automated error checking.
"It showed us that in the past there were defects (in) our traditional ways of doing betas (that) weren't catching all the problems."
Now, he said, Microsoft hoped to fix more bugs before the product ships. However, he also noted that more than half the errors reported from Windows XP involved third-party applications, not the operating system.
With Office 2003's beta test, that includes Microsoft's own applications.
"For InfoPath and OneNote, this is an amazing opportunity," LeVine said of the two new members of the Office line. "These products have never shipped before, and so we're getting feedback - we're getting real-world feedback before it ships."
It's the way things should be, Enderle said.
"This is how it's supposed to work - it's better that they delay and make sure the product works before it ships," he said.