Photo: 123rf
A facial recognition expert is rejecting an official test that cleared a government system of racial bias.
Internal Affairs got independent tests done using selfies of about 150 New Zealanders, of the facial recognition technology within its new online Identity Check tool.
But AI, data and emerging tech ethicist Dr Karaitiana Taiuru said the tests were too limited.
"I'm not convinced at the amount of people that were used, I'm not convinced that they only used Department of Internal Affairs staff who volunteered to participate in it.
"No Māori with moko were volunteered in this test."
The government aims to roll out Identity Check widely to provide online access to public services.
The testing by an accredited Australian lab should have used 245 people at least, to be 95 percent accurate.
While it only had 148 people in it, the test report said "in the context of this evaluation looking at fairness for different demographic groups, the sample size was sufficient to undertake a comparative bias analysis".
Of those tested, 61 percent were of European or Asian ethnicity, and 39 percent were Māori or Pasifika.
The system has to be able to catch impostors trying to use it for false verifications. Tests of this returned higher error rates for Māori and Pasifika which the report said might be due to blur, or lighting, or "shadows".
"If there's shade on a face which can create an issue recognising a face, what happens when a person has a moko on their face?" Taiuru said.
"It's likely to also contribute to false negative matches" - that is, the system treating a genuine selfie as if it were from an impostor.
"It's not clinical.
"I'm quite certain if I submitted those numbers to a university or a government research fund, I'd be declined because it wouldn't be scientific.
"I think what should happen, is we should be using a trial of people from the community, we need to recognise that not all Māori look the same."
Internal Affairs should be further removed from the testing process, due to its incentive to get Identity Check spread around, while it might also be a problem that an international testing lab might not appreciate the local cultural nuances, Taiuru said.
Internal Affairs said the tests were "just one step the department is taking to ensure that our technology works fairly for everyone".
"DIA is committed to engaging and collaborating further with Māori and the wider community and intends to conduct further studies."