In a postmortem of last month's Windows animated (.ANI) cursor vulnerability, one of Microsoft's security development gurus Friday spelled out how the bug sneaked into Vista.

Michael Howard, an authority on Microsoft's Security Development Lifecycle (SDL) -- a multipart initiative that aims to get developers to design more secure code -- posted an extensive entry on the brand-new SDL blog that outlined lessons learned from the ANI vulnerability. "SDL is not perfect, nor will it ever be perfect," Howard acknowledged Thursday. "We still have work to do, and this bug shows that."

That bug, which first surfaced late last month and posed enough of a threat that Microsoft went out of cycle to patch it, affected all older editions of Windows as well as the newest, and supposedly more secure, Windows Vista. Some security researchers, in fact, took Microsoft and its SDL process to task for not catching the flawed code as Vista was written, debugged, tested and polished.

Some of those same researchers immediately weighed in on the unusual mea culpa by Microsoft. "This is really out of character," said Jonathan Bitle, the manager of the technical accounts team at Qualys. "Microsoft historically has played security issues much closer to the vest."

Oliver Friedrichs, director of Symantec's security response team, was a bit tougher in questioning Microsoft's motives. "They're attempting to be more transparent to explain why this vulnerability was missed. They received a lot of criticism for not catching this earlier and for letting it into Vista, and I think this was one of the only ways for them to explain both to the technical and the management-level communities how they actually missed it."

Specifically, Howard called out flaws that the ANI vulnerability revealed in Vista's security components, as well as in Microsoft's development tools and processes.

The /GS switch, a function of Microsoft Visual Studio's compiler that's designed to protect stack variables from overflows that could result in arbitrary code execution, was one. Some third-party researchers, notably Ollie Whitehouse of Symantec, have criticized Microsoft for not /GS compiling all of Vista's binaries. Turns out, however, that in the ANI case, that wasn't the problem.

"Because there are no candidate buffers on the function's stack, there is no /GS cookie added to the stack, even though the code is compiled with /GS," said Howard. "This is not the first time we've seen code with no cookie, and this has made us rethink the heuristics used by the compiler when it determines whether to place a cookie on the stack or not."

Another Vista security feature, Address Space Layout Randomization (ASLR), which is supposed to randomly assign data to memory to make it tougher for attackers to determine the location of critical OS functions, also didn't have the intended impact on the ANI vulnerability. "If the vulnerable code is wrapped in an exception handler that catches many errors [as was the animated cursor code], a failed attempt will not crash the component and the attacker can try again with a different set of addresses," Howard said. David LeBlanc, also of Microsoft and the co-author with Howard of the just-released book Writing Secure Code for Vista, blogged about the danger of using exception handlers on April 3, the same day that Microsoft patched the ANI bug.

"I've said a lot of times that incorrect use of exception handlers will get you hacked," LeBlanc warned at the time.

Howard backed him up: "[An exception handler] can usually be good for reliability, but it has an interesting security side effect. By itself [its] 'catch everything' construct is not a security bug, but it can aid an attacker if the exception handler wraps vulnerable code."

Some of Microsoft's own development and testing tools also failed to flag the code, which Howard said was taken from Windows 2000, a seven-year, two-month-old operating system.

"Our static analysis tools do not flag this construct as a security bug because it's a very low-priority warning," admitted Howard. Why? "Code that uses calls such as 'memcpy' is hard to flag as vulnerable without generating a great many false positives. This is a research problem that no one has solved, here or elsewhere." Howard said Microsoft will investigate further and may ban calls like memcpy in new code to prevent a recurrence.

Fuzz testing, which drops random data into applications or operating system components to see if -- and where -- breakdowns occur, also missed the bug. "The animated cursor code was fuzz-tested extensively per the SDL requirements," said Howard. "[But] it turns out none of the .ANI fuzz templates had a second 'anih' record. This is now addressed, and we are enhancing our fuzzing tools to make sure they add manipulations that duplicate arbitrary object elements better."

These moves, warned a commenter to Howard's blog, are only cosmetic fixes. "Security has to be written right into the code; you can't add it in later with a bit of compiler magic," said someone identified as Xepol. "Pretending you can 'fix' old code with a few compiler flags is going to result in problems like this repeating endlessly. At some point, you have to stop, go back and fix the foundation, or you might as well be building on quicksand."

Both Bitle and Friedrichs sounded a similar clarion call. "What Microsoft highlighted here is that while the SDL is thorough, reused code is going to be their downfall," said Bitle. "Old code will be the Achilles' heel of SDL."

"This is the gap that we're seeing in the [SDL] process," added Friedrichs, "in that flawed legacy code can make it into a current version of Windows. And this gives them an incentive to analyze that legacy code. I would if I were in their shoes."

"They're obviously recognizing that detecting these vulnerabilities is getting more complicated, and they must get more complex in their response," said Bitle.