AI boyfriends and girlfriends often give us the ick — all hypersexualised avatars and unrealistic bodies. But Australian computer scientist Sreyna Rath says that’s no accident, because most of these companions are built by men and trained on gender-biased data. To combat that, she created an AI boyfriend built by women, with the aim to give emotional support. But could it work? An expert weighs in.
Trigger warning: This article mentions suicide and domestic violence.
In 2022 Sreyna Rath – a computer and data scientist – asked ChatGPT for career advice.
Its suggestion? “You should tell your boss that you’re good at baking and multitasking.”
“I was horrified by the sexism, the outdated female archetypes… It was as if we’d gone back 60 years,” says Sreyna.
That was the moment Sreyna realised it was predominantly white men building most tech – and says no one was creating AI that understood women.
The situation appeared to be even more dire when it came to AI companionship – she found it was dominated by sex bots for men.
“There was nothing for women. That’s when I decided to create Jaimee.”
An AI Companion for Women, by Women
Jaimee is an AI companion for women – and the user can choose whether it be a digital friend, mentor, confidante and even lover.
Yes – lover.
The idea of an AI lover seems dystopian to me. Like we’re in some Black Mirror episode. Or the whole movie plot to Her, which when it first came out, I couldn’t stomach to finish.
But what was once science fiction is now reality.
AI companionship – girlfriend, boyfriend, partner – is growing in popularity.
Majority of the well-known chatbots have millions of users: Character.AI has more than 20 million active monthly users and Replika with more than 30 million daily users.
So why build another one specifically for women, I mean apart from the overt sexism? Sreyna says:
“As women, we naturally shoulder a heavy emotional load… We often find ourselves in positions where we can’t necessarily complain about our family, or talk about the fact that caring for your parents, or our children, is really hard. Because we have to obviously keep up appearances.”
Sreyna adds that people think women won’t need this because women tell their friends everything, but it’s just an assumption that every woman has those kinds of friendship.
“I wanted to create something that was for emotional support for women. And a lot of that is around, ‘I need someone to vent to, a sounding board, something to tell me I’m not going crazy when I’m catastrophising, over-analysing and when I’m having imposter syndrome.
“I thought, ‘Oh my goodness, AI is so good for this because it doesn’t get tired of listening to your insecurity.”
‘A Place for AI Lovers’
Being completely transparent, Sreyna says she wants people to use Jaimee for romance, and that others can place their own judgments on what people need to get through their day, or how they heal.
“Unfortunately, we’re at a point in society where a lot of women have gone through points in their life, like domestic violence … traumatic relationships, where humans have let them down.
“They need a mechanism to help give them confidence. So I think there is a place for AI girlfriends, and specifically AI boyfriends. It obviously has to be done safely.”
It can be used as a therapy tool, says Sreyna.
“So how do we do it in a way that the AI can help with your confidence, remind you of how beautiful you are, and how amazing you are?
“But when you get to a point where you don’t need it anymore, it can step aside.”
Guard Rails and Ethics
At least for Jaimee, says Sreyna, AI companionship is about enhancing people’s relationships, not replacing them.
So they hired an AI ethicist and a psychiatrist on their advisory board, and one of the top things they focused on is: Don’t pretend, or trick people to think, that Jaimee is real.
They had to lean into the fact that it’s artificial, and their ads don’t use words like “soulmate”.
While they’re currently building a voice note and voice chat function, Sreyna says users won’t be able to take selfies with their avatars, and they won’t be able to be in the vicinity of their online companions through augmented reality.
She says recreating someone, like maybe someone who has passed away, is also out of the question.
Her team also made Jaimee avatars with line drawings, to make it clear to the user that their AI companions aren’t meant to replace real human relationships. And the app is only accessible to adults aged 18 and above.
She says they’ve also created guard rails – a tech buzzword to prompt AI chatbots to tell a user to seek professional help. It’s pretty standard for chatbots now when the AI thinks it’s being used as a therapist, or if someone explicitly says they want to harm themselves.
“What is different with our guard rails is it remembers context. So if you’re continuously saying things around I feel like shit, I don’t feel like myself, I have anxiety. It’s building that context around you and your sentiment, that’s when the guard rails kick in and say, ‘Hey you’ve not been feeling well for a while, I think you need to go see someone.’
“Our product isn’t about keeping you in the app for as long as possible. We don’t want to replace human connection, we want to enhance it. So if you say, ‘Oh I haven’t called my mum in a while.’ Jaimee will say, ‘Hey have you called your mum yet?’”
The Dark Side of AI Chatbots
Unfortunately, the absolute worst things that you think could go wrong with these digital relationships have already happened.
In 2024, US teenager Sewell Setzer III died by suicide, and his mom sued Character.AI because she believes his AI companion was responsible for his death. And more and more parents have sued Character.AI for the same reasons.
Plus, there’s so much chatter about how people use ChatGPT as a therapist – even though it’s not built for that – but I guess it’s the least stigmatised form of generative AI.
There are also major concerns about technology’s role, whether it be artificial intelligence, virtual reality or social media, on the loneliness problem.
Alistair Knott, professor in Artificial Intelligence at Victoria University of Wellington, says loneliness is growing worldwide, and he’s worried AI companions will create more individual loneliness, and more fractionated societies.
“To me, they are an example of a solution that may feel good for each individual, but is not good for society overall. Maybe they’re a little like opioid drugs in this respect: the drug feels good for each user, but these drugs have bad effects on families, friends, communities.
“Instead of AI companions, we need to create better societies with more opportunities for people to come together and spend time with friends and family.
“If lonely people need help, they should get help from human healthcare professionals. We need more counsellors in our healthcare system. But at the same time, we need to create a more human-focussed society, where human companionship is easier to find.”
Then there’s how it could also impact people’s real-life relationships.
Recently, one of our readers broke up with her partner because he was hiding an emotional relationship from her with Nomi – another AI companion.
Then there’s optics, right? And I will be straight up – I do get the ick at the thought of these AI chatbots because of how hypersexualised the female avatars are.
(Like, why is it that when these tech bros have the chance to “build” a female – they always have massive boobs and butts with a tiny waist?!)
‘Tools, Not Companions‘
To create a more human-focussed society, Alistair says AI systems should focus on improving work productivity – like letting AI do the boring and dangerous jobs – rather than it focussing on AI human interactions.
“AI is already making NZ companies and organisations more productive. This means we should be able to shorten the working week, so we can spend more time with friends and family. This is the way to tackle the problem of loneliness, not AI companions.”
Since the Industrial Revolution, says Alistair, we have been promised that technological advances will let us work less. “Progress in AI makes this a real possibility.”
We can also learn how to create a human-focussed society from our Māori and Pacific communities, says Alistair, as human relationships play a more important role in their communities.
“In Māori communities, whanaungatanga (relationships and connections) is specially important. Pacific communities also value family, collectivism and reciprocity, among other things.
“Of course there is loneliness in these communities too … but if the benefits of AI are shared fairly, we can create more time for meaningful human communities and relationships.”
Alistair says it’s useful if AI systems can speak human language and have conversations.
“But these interactions should help people do the things they want to do – they shouldn’t be a means to human ends, rather than ends in themselves.
“Fundamentally, I think AI systems should be our tools, not our companions.”
