Deepfake porn is traumatic for those targeted, experts say, and is increasing, but remains a legal grey area. Photo: 123RF
An increasing number of New Zealand students are being victimised by deepfake pornography, and legal grey area means help can be hard to find
Artificial Intelligence has gone bad in our classrooms.
A new trend is emerging called pornographic deepfakes, created by students, of students. It's increasing. And it's devastating.
"They look real... whilst the individual knows they are not... they are pretty accurate, and this brings a sense of shame and embarrassment to the victim, and they don't want to reconnect back in school because they think everyone is talking about them, so it's incredibly challenging for students from a mental health perspective," Secondary Principals Association President Vaughan Couillault tells The Detail.
Put simply, pornographic deepfakes are a form of online bullying using A.I.
How they work is simple: deepfakes create a sexual image of a person - in this case a student, normally female - doing or saying things they have not said or done.
"It's gone from zero to many, very quickly, so it is getting worse. Is it an epidemic right now? No. But is it getting easier for someone who is tech savvy to do it? Then yes," Couillault says.
"There have already been hundreds of people affected by deepfake imagery that isn't them, and they're hurt and broken and may not have the chance to get the support they need to get back to fully functioning. It can have life-long implications for the person targeted."
Sex therapist and parent consultant Jo Robertson from Makes Sense, an organisation advocating for a safer digital space for children, tells The Detail it is mainly young females who are targeted, and many do not want to report it to the police.
"We have got image-based sexual abuse of women and girls, and it is in the classroom. They will most likely keep it a secret. They don't like going to the police about it or making more of a deal of it or gaining more attention.
"But it is exactly the same [as] if they received in-person sexual abuse. In society we tend to put it at a lower grade... but there is no hierarchy in trauma... and that's absolutely the case here. Just because it's online doesn't mean it's less real or less painful. And it hangs over them, sometimes for much longer than in-person abuse."
When it comes to legal protection, the area is grey, she says.
"It's not really clear in the Harmful Digital Communications Act if those images need to be realistic or if they can be created, so it's ambiguous. And when it's not clear, people can fall between the cracks."
The act regulates the occurrences and impacts of issues such as online bullying, harassment, revenge porn, and other forms of digital abuse and intimidation. In 2022, the act was amended, adding "intimate visual recording" being posted without consent as a form of harmful communication.
Check out how to listen to and follow The Detail here.
You can also stay up-to-date by liking us on Facebook or following us on Twitter.
Where to get help:
Need to Talk? Free call or text 1737 any time to speak to a trained counsellor, for any reason.
Lifeline: 0800 543 354 or text HELP to 4357
Suicide Crisis Helpline: 0508 828 865 / 0508 TAUTOKO (24/7). This is a service for people who may be thinking about suicide, or those who are concerned about family or friends.
Depression Helpline: 0800 111 757 (24/7) or text 4202
Samaritans: 0800 726 666 (24/7)
Youthline: 0800 376 633 (24/7) or free text 234 (8am-12am), or email talk@youthline.co.nz
What's Up: free counselling for 5 to 19 years old, online chat 11am-10.30pm 7days/week or free phone 0800 WHATSUP / 0800 9428 787 11am-11pm
Asian Family Services: 0800 862 342 Monday to Friday 9am to 8pm or text 832 Monday to Friday 9am - 5pm. Languages spoken: Mandarin, Cantonese, Korean, Vietnamese, Thai, Japanese, Hindi, Gujarati, Marathi and English.
Rural Support Trust Helpline: 0800 787 254
Healthline: 0800 611 116
Rainbow Youth: (09) 376 4155
OUTLine: 0800 688 5463 (6pm-9pm)
If it is an emergency and you feel like you or someone else is at risk, call 111.