AI-generated personas becoming increasingly common online
ABC investigated the rise of AI personas where people are forging bonds with influencers generated by artificial intelligence.
Transcript
00:00Well, it actually began when I met a woman at a party who had created one of her own.
00:06This character was called Margot Monet.
00:09She was entirely fictitious, generated using AI image generation software.
00:15And the goal was, she said, to monetize Margot, which was primarily through brand partnerships
00:21but also selling subscriptions to people who would then get access to exclusive not-safe-for-work
00:27content.
00:29She said she'd been actually inspired by this character, Aitana, whose creators I spoke
00:35to and who said she could earn sometimes $80,000 a month.
00:40And once I started looking at these two, my Instagram algorithm obviously caught on and
00:46suddenly I was inundated with these AI influences.
00:50So it really kind of piqued my interest from there.
00:53I have to admit, I've been following Lil Miquela for a number of years now and watching her
00:58progress with interest, she's become more and more realistic as time goes on.
01:03Just how common are these AI influences?
01:06Well, as I say, once you start looking, they really come out of the woodwork.
01:13It's hard to establish.
01:14There are definitely hundreds, probably thousands, but what really kind of struck me as I researched
01:19more and more was that there was kind of these two different types of AI influences.
01:24There was the quite basic image generation with a human behind the keyboard kind of orchestrating,
01:29but then there were these AI chatbot companions, which were chatbots like ChatGPT, but where
01:35ChatGPT responds with knowledge and to search requests, these AI chatbots offer companionship
01:42and conversation support.
01:46When you factor this other group in, the largest provider of these chatbot companions is a
01:52company called Replica, and they say that they've got 10 million customers, all of whom
01:56are paying a subscription.
01:58They pay the subscription to design their friend, but usually a girlfriend or a boyfriend,
02:03virtual of course, which then support them.
02:07Wow, that's incredible.
02:09Some of these AI influencers having millions of followers, clearly it may have a good financial
02:15impact on the people operating those accounts, but what about on the flip side?
02:18What is the impact on the people who are interacting with these influencers as though
02:23they are real people?
02:26The first one that probably will occur to most people is body image issues.
02:33For decades, we've kind of spoken about, particularly young women, but increasingly young men who
02:36kind of get continually confronted with images of models and fitness influencers whose full-time
02:43jobs are to look good and stay fit, and it kind of creates unrealistic expectations and
02:47a lot of pressure on young people.
02:49Now, they are obviously being confronted with these entirely fake creations who can be designed
02:56in any way the designer wants in pursuit of whatever they consider to be the most desirable
03:01features.
03:03Less obvious is this risk of a thing called limerence, which is, I spoke to a psychiatrist
03:08who has already seen this between his clients, or his patients rather, and real influencers,
03:14which is the projection of a real connection when none exists.
03:19This psychiatrist, Rahul Kahana, actually had a patient who had done this with an older
03:26female influencer who provided the likes of cooking tutorials, and in the patient's mind,
03:31this connection had taken on a real-world aspect, which was obviously highly distressing
03:36when he kind of came to realize in time that there was no real connection there.
03:40There's also this risk that comes with these kind of parasocial relationships, which is
03:44a complete imbalance of power between the user and the creator.
03:48In a normal human relationship, a healthy one, there's an equal distribution of power.
03:54In these cases, all of the power rests with the creator.
03:58They can delete an account, they can block a user, or they can even change a bit of code,
04:02so one of these chatbot companions can suddenly start acting completely differently.
04:06We actually saw this with Replica.
04:07They were threatened with shutdown by Italian regulators, and they removed an erotic functionality
04:16from these characters, and the community that used them went on to Reddit with an outpouring
04:22of grief, talking about the lobotomy, they called it, which was essentially thousands
04:25and thousands of people seemingly to have lost their boyfriends or girlfriends.
04:30So when you start kind of getting that real-world connection kind of forged, it can be very
04:36damaging to the user.
04:38And the final thing, and possibly the darkest thing, is the implications for AI pornography.
04:43Margot Monet, like many of these other influencers, sell essentially not-safe-for-work content.
04:51The potential there for what one psychiatrist described to me as opening Pandora's box and
04:57creating a really dangerous spiral for people who haven't had normal sexual relationships
05:02and engage with AI pornography, they get unrealistic expectations, it can kind of
05:07feed the darkest kind of desires, I suppose, and that can have a kind of a really damaging
05:12effect on what that person's perception of what a healthy, happy sexual relationship
05:17could be.
05:18God, just an absolutely fascinating space this is, and will only become more and more,
05:25I guess, complex as we move forward into the brave new world of AI.
05:29Gerard and Kan, thank you so much for talking us through it.