It was created and patented by one of the U.S. government’s most trusted surveillance labs, the nonprofit research center Mitre Corp. The organization is like James Bond’s Q lab but for the whole of the federal government. The Virginia-based Skunk Works has in the past churned out autonomous surveillance drones, smartwatch hacking tech and tools to take fingerprints from social media images. And Forbes has found a previously unreported patent that seeks to boost facial recognition technology by guessing someone’s name by just looking at their face’s characteristics.

 

It might sound like sorcery, but the tech stems from previous research from Hebrew University of Jerusalem that suggested a person’s name may be reflected in his facial appearance, a phenomenon dubbed “The Dorian Gray effect,” so named after Oscar Wilde’s eponymous antihero. In their study, they found that people could often guess the name of a person when presented with five different options. Participants accurately picked the right name in 28.21% of the cases, higher than the expected 20%. When a computer, trained on a data set of 100,000 faces, was given two different names and a face, it was right 59% of the time, higher than the 50% one would expect from random guesses. These findings, said the researchers, indicated that both humans and computers were able look at a face and have a better chance of matching the correct name to it than the wrong one. They suggested that this could be down to the way a name affects a person’s life: “We propose that one’s given name may have a Dorian Gray effect on one’s 

face. Our given name is our very first social tagging. Each name has associated characteristics, behaviors and a look, and as such, it has a meaning and a shared schema within a society. These name stereotypes include a prototypical facial appearance such that we have a shared representation for the ‘right’ look associated with each name. Over time, these stereotypical expectations of how we should look may eventually manifest in our facial appearance.”

 

In Mitre’s patent, filed in 2018 and published in 2020, it said it wanted to improve facial recognition based on the Hebrew University’s findings. “Current facial recognition systems may be further improved by matching faces with names using the ‘Dorian Gray’ effect, which does not require that the person in a face image be known to the system before selecting a likely name for that person. The 59% accuracy achieved by the researchers, however, is far too low to be useful and needs to be improved.” In experiments, Mitre used the “Labeled Faces in the Wild” database (hosted by the University of Massachusetts at Amherst) that contains more than 13,000 images of faces collected from the Web, each labeled with the name of the person pictured. Mitre claims that its tech was able to get that accuracy up to between 72% and 80.5% when its system was presented with a face and two names to choose from, which “vastly exceeds the expected accuracy of 50% were the system to randomly select one of two names.”

 

Mitre suggests its tech can work both ways. Either a face can be used and analyzed to guess a name, or a name can be taken and a face selected by the technology as the best possible match.

 

Whether the U.S. government has access to the tool, or is using it in some capacity, is unknown. Mitre’s only customer, though, is the federal government, from which it has received between $1 billion and $2 billion of taxpayer money over recent years. And it has previously supplied facial recognition services to the FBI.

 

Much of Mitre’s advanced national security-focused research is shrouded in secrecy and it declined to comment on this patent, and requests for interviews over the last year. The organization has been more forthcoming about its work on fixing America’s broken Covid-19 contact track-and-trace system. 

 

The face-to-name tech creators didn’t reach out to the University of Hebrew researchers either, who are only now learning that their work has been adopted by a government contractor as a potential way to improve surveillance tools.  “We are not familiar with this patent and no one approached us,” said one of the researchers Jacob Goldenberg. Asked if he thought that creating a tool that could match names to faces was feasible, he added: “I am pretty sure it can work.”

 

“But this shouldn't be a big a surprise—there are so many projects on topics like facial recognition and what can be learned from faces, both at academic and at industry levels.” He pointed to other tools that can detect whether a pupil is paying attention in class, or Face2Gene, which claims to be able to help doctors detect syndromes based on a computer’s analysis of the face.

 

 

Source: Forbes