Researchers are developing technology they say will use data mining and social networking techniques to spot and stop insider security threats and industrial espionage.

Air Force Institute of Technology researchers have developed software that can spot insider threats using an extended version of automated document indexing known as Probabilistic Latent Semantic Indexing (PLSI). This technology can discern employees' interests from e-mail and create a social network graph showing their various interactions, researchers said.

The technology could help any organization sniff out insider threats by analyzing email activity or find individuals among potentially tens of thousands of employees with latent interests in sensitive topics. The same technology might also be used to spot individuals who feel alienated within the organization as well as unraveling any worrying changes in their social network interactions.The researchers explain that individuals who have shown an interest in a sensitive topic but who have never communicated to others within the organization on this subject are often the most likely to be an insider threat.

The software can reveal those people either with a secret interest in that topic or who may feel alienated from the organization and so communicate their interest in it only to those outside the organization, researchers said. Another important signal of alienation or a potential problem is a shift in the connections between an individual and others within the organization. If an individual suddenly stops communicating or socializing with others with whom they have previously had frequent contact, then the technology could alert investigators to such changes.

The research team tested their approach on the archived body of messages from the liquidated Enron company e-mail system. Their PLSI results unearthed several individuals who represented potential insider threats. However, it should be noted that the individuals under indictment are the bosses of the organization. It was the core of the organization that is responsible for the illegal behavior, researchers said.

The research team points out that while Internet activity was not available for Enron, it is generally available from the same sources that supply e-mail history logs and so could be used to search more widely for insider threats. He adds that by turning the domain 'on its ear' in effect, the identity of the whistleblower could be revealed.

According to the 2007 e-Crime Watch survey, companies said that while hackers and outside threats represented the greatest threat (26 per cent) to networked resources, current employees inside the organization were not far behind (19 per cent). Foreign entities and ex-employees were the next greatest threats, the survey said.

A small percentage of data that leaks from corporate networks (0.5 per cent) is stolen by professionals whose efforts will evade detection by security products touted as data-leakage prevention tools, said Nick Selby, an analyst with 451 Group who spoke at the Security Standard event last year. The products do catch data leaks, 98 per cent of which are linked to an accident or stupidity and 1.5 per cent that are caused by vengeful employees clumsily attempting to steal data, he says.

"Data leakage is an antistupidity issue as much as it is a technology issue," Selby said. "Most data-leakage products can't discover activity by skilled insiders looking to steal."