It's the how the future is meant to be, isn't it? The good guys need to find a bad guy in a crowd of people, so they start scanning the environment with a camera that is equipped with facial recognition technology. Seconds later, they scan a face that's a positive match with an entry in their criminal database and bam, they've smoked him out.
That future, however, is already here. Sort of.
Facial recognition's potential is substantial, even if it isn't fully realized yet; its level of accuracy isn't quite high enough to be used in the aforementioned scenario, at least not with a high rate of success. But it is good enough to already be implemented into a number of different vertical markets including commercial sectors, marketing, healthcare, and hospitality. And in many cases, says Bob Lorenz, executive video specialist at Panasonic, facial recognition is a "force multiplier and an enabler" that's being layered on top of existing systems.
[Related Slideshow: 8 ways physical security has evolved
"For example, in the retail space, shops will already have existing security cameras in place -- to watch people for shoplifting -- but now they're capturing these faces," says Lorenz. "And if they do catch someone, they will store these faces into their databases and next time [that face] is seen, a notification can be sent out to store personnel or security officers. It's a subcomponent added to the infrastructure that's already deployed."
Jay Hauhn, CTO and VP of Industry Relations for Tyco Integrated Security, breaks down the use of facial recognition into two categories: cooperative environments and non-cooperative environments. In the former, the person whose face is going to be scanned is aware of it and is opting into a process where it's serving as their credential; they're going to look straight into a camera with no attempt to obscure their face. Non-cooperative environments, however, are when the subject is not necessarily aware that their face is being scanned and is making no attempt to look directly at the camera.
"In cooperative environments, it works fairly well," says Hauhn. "It works as well as any other biometric, but that's not the 'promise' of facial recognition. [In that scenario], facial recognition is not that difficult to defeat with a picture."
[Related: Facial recognition may need regulating
While Hauhn says that in the security-based, cooperative environment -- where facial recognition is in its most simplistic form -- the technology isn't that difficult to defeat, there's still hope. Though he could not divulge specifics, Hauhn says that Tyco is familiar with a company that is developing other factors to be implemented into facial recognition to develop a form of multi-factor authentication in the cooperative environment. For instance, instead of just recognizing a face, cameras may also soon look for eye blinking, lip movement, facial muscle movement, iris movement, or even analyze of a person's walk.
Non-cooperative environments are where facial recognition isn't as widely implemented -- at least not successfully - -says Hauhn, given its relatively low efficacy.
"The promise of facial recognition is more of the pick-somebody-out-of-the-crowd scenario," he says. "And that's where it starts to not be as effective."
Hauhn used the example of casinos using facial recognition to keep cheats off their premises. Though he admits that he's not well-versed in the casino security methods, he says that they can likely put for the necessary resources to sift through numerous false positives that are generated and look close to identify their targets. But the "holy grail" is when security teams are at the point where they can simply apply any face the camera picks up, at any angle, to identify the possible bad guys.
"We're just not there yet," he says. "The technology is not that good. It's pretty easy to pull a hat down and have your face not looking at the camera at the right angle to get the right data. With [the systems] that are 2D, you have to have a straight-on view for accuracy." There are some companies, however, that are working on developing facial recognition that operates in a 3D environment. With the use of geographic cameras that use multiple markers, these could increase the likelihood of getting a good scan even when the subject isn't looking at the camera head-on.
Still a ways to go
Facial recognition technology is still very much in its infancy, however. Lorenz says that it has only been within the past three or four years that camera technology has improved enough for facial recognition to be successfully employed.
"A lot of times, what you're finding are the pitfalls of facial matching is getting a good read on the face," says Lorenz. "But with the new HD cameras, increased megapixels, and the amount of data that can be streamed into backends, we really reached a tipping point. Now you can get clearer pictures, better defined faces, and detection becomes better in these scenarios."
While Hauhn agrees with the sentiment that better technology has provided better accuracy, he doesn't believe that facial recognition has proven itself yet his so-called non-cooperative environments. If subjects don't know that facial recognition is being used on them, it's a non-cooperative environment and unfortunately, this is the crux of using it for security purposes in many scenarios like scanning airports for known or suspected terrorists.
"After 9/11, facial recognition and video were supposed to solve all of our problems. Well, that hasn't happened," he says. "There were high profile tests about seven or eight years ago that showed that [the accuracy] hasn't gotten any better."
But in the cooperative environments, like booking someone at a police station, Hauhn says that facial recognition works very well.
"Being able to scan someone's face across a whole database of people works pretty well," he says. "If you do fingerprints, they're not nearly as fast in the environment. But again, it's forced cooperation."
Again, though, as Hauhn points out, the promise of facial recognition is being able to work in a non-cooperative environment too. And in that sense, both Hauhn and Lorenz believe that it still has a way to go.
"If you look at where facial recognition is on the scale of accuracy of biometrics, it is one of the less accurate ones compared to, say, fingerprint biometrics, which is way up into the 99 percentile," says Lorenz. "Facial recognition is a little bit less accurate in that regard. It has additional dependences, like being able to capture a good face in low light, or if the face is pointing at the camera at an angle."
Lorenz adds that the efficacy also depends on the technology behind systems, like the type of matching engines that are on the backend side of things. Like Hauhn, Lorenz finds that the typical 2D nature of facial recognition is both limited and easy to fool, making 3D systems more appealing for accuracy and overcoming limitations like the viewing angle.
Ultimately, there's a lot of room for growth. But as facial recognition becomes more advanced and increasingly accurate, we can expect to see it used in more scenarios and for different purposes.
"I think there is a tremendous amount of room for growth in the cooperative environment," says Hauhn. "Like hands-free card access: you could walk up to a door and have it recognize you and open up, or even be able to tell if a second person tailgated you through the door. Or bank transactions. I think a person would want to use their face to make sure it was them accessing their money and not somebody else."
Lorenz shared similar sentiments, saying that as the technology grows, cameras that have already been deployed in major cities will gradually become upgraded and embedded with facial recognition. While Hauhn agrees, he adds that the current (comparative) lack of deployment is also a matter of cost.
Big Brother is watching you
Another major issue with facial recognition beyond accuracy is that of privacy. In order to be effective, the technology usually requires either storing the scanned visage in a backend database, or checking it against a face that is already there.
"You think about cameras that are constantly capturing faces and identifying who those faces are...yeah, there are privacy concerns there," says Hauhn. "But you're going to get into the traditional argument over all privacy issues: what's a reasonable expectation of privacy?"
As he pointed out, there are already plenty of cameras placed around major cities that capture people on a daily basis, and the average person doesn't mind. Once facial recognition enters the equation, however, and the cameras therefore know who you are and where you are at a given point in time, the privacy question changes. Ultimately though, the amount of outcry is probably going to vary from person to person, he says.
"Whether or not there are privacy concerns comes back to what your opinions are on Big Brother," says Hauhn. "You will get as many different answers [about it] as there are people."
And what about how long these facial scans are kept in the backend? Hauhn says that police departments are claiming that they need to keep them for up to 30 days in case they need to do an investigation on where people were on certain days and that "those are the kinds of things that scare privacy guys."
There are, however, measures being gradually implement to regulate how long such private data can be stored, according to Lorenz.
"Yes, there are some privacy concerns. But right now there are a lot of biometric privacy of information acts out there right now," says Lorenz. "They address numerous issues, like, how are you going to protect, use, handle, and store that data? What's the retention around that? Once the retention is satisfied, how do you dispose of that data? Also, encryption: Are you encrypting on the backend? Do you have any tampering and safeguard measures?"
While there are many privacy acts out there, they vary from state to state and are typically centered around biometric privacy on the whole; it may be a while before we see standardized regulation of facial recognition across the board. After all, facial recognition is still early on in its growth process, says Lorenz.
"Collecting this data and analytics around it is very coming-of-age now," he says. "There are a lot of analytics that are being done in cameras today and facial recognition is just one more of those components."