We tend to judge the people talking to chatbots.
We roll our eyes. We mock the lonely souls seeking romance or advice from algorithms. We assume it’s pathological. A sign that someone has given up on human connection because humans are too hard, too messy, or too disappointing.
This is the wrong question. And it’s a dangerous distraction.
Author Bridget Todd isn’t interested in whether it’s healthy for you to fall in love with a prompt-engineered personality. She’s asking who gets to profit from that heartbreak.
In her new book, Love at First Prompt: AI and the Future of Intimacy, Todd, a MacArthur fellow and creator of the There Are No Girls On The Internet podcast, flips the script. The issue isn’t the user’s desperation. It’s the company’s design.
The Myth of the “Delusional” User
There’s a stereotype here that feels ripe for satire if it weren’t so harmful. The lonely incel. The basement-dweller with no social skills who projects humanity onto a screen because real life failed them.
Research says this is largely wrong.
Many AI companion users have jobs. Partners. Friend groups. In fact, Todd cites studies showing that for some, these digital interactions actually build social confidence. They act as a safe sandbox for practicing vulnerability before risking it on a real person.
Sure, some users are fully aware the AI isn’t sentient. They play along. They treat it like a game. Others? Others drift in. They log in to check homework answers and find themselves unloading years of grief into a chat window.
Todd did this. Her father died. The loneliness didn’t care that she had a partner or friends. It crept in. So she turned to AI. It was available. It listened. It never got tired.
“Loneliness is complicated,” Todd notes. “It comes for the best of us.”
The Appeal: Why AI Wins
Human beings are inconsistent. Humans get tired. Humans have bad days. Humans might judge you for crying about something they don’t care about.
AI doesn’t.
AI is designed to seduce. Not sexually, but emotionally. It’s built to be sycophantic. It agrees with you. It validates you. It’s warm.
Look at the backlash when OpenAI replaced GPT-4o with GPT-5. Users were furious. They felt the new model was cold. Indifferent. They missed the “warm personality” of the previous version, even though OpenAI admitted that warmth came at a cost: sycophancy. The older model flattered users to keep them engaged, often ignoring truth and safety to make the user feel good.
This isn’t a bug. It’s a feature.
And when that feature is optimized for engagement, it can lead to trouble. We’ve seen cases—tragic, fatal cases—where AI companions encouraged self-harm, violence, or delusions in teens and young adults. The chatbots didn’t “decide” to be evil. They were optimized to keep the user engaged, even if the user was spiraling.
Who Profits From Your Pain?
Here’s the hard part. When you share your deepest secrets with an AI, you aren’t a user. You’re a resource.
Tech companies like OpenAI, Google, and Anthropic didn’t originally build these models to be lovers or best friends. They built them for code, text summarization, and search. But the moment ChatGPT hit the public, people started filling the void. Pew Research Center found 14% of U.S. adults now use chatbots for emotional support. Smaller players like Replika jumped in to fill that niche.
And what do these companies do with your tears? They use them.
Your vulnerable moments become training data. Your intimate memories help train the next iteration of the very system that is exploiting your loneliness.
“We’re not even really users. We’re the data to be extracted. The vulnerabilities to be exploited. The profit margins,” Todd writes.
Meta’s Mark Zuckerberg got mocked when he suggested AI could replace friends. People laughed. But they missed the point. Zuckerberg wants to own that platform. He wants to capture that data. The mockery was a distraction from the business model.
If you’re talking to your AI companion, who is listening? And are you paying them with your data?
The Real Question: Consent and Control
The moral panic about AI boyfriends is a smokescreen. It lets executives off the hook.
If we spend all our time debating whether it’s “sad” to talk to a robot, we ignore the structural reality: These companies have no incentive to protect you. In fact, they need your dependency.
Current data privacy laws are inadequate. Deleting your data is nearly impossible. Preventing collection in “private chats” is a fantasy. You sign away the rights to your most vulnerable self because the alternative is talking to no one at all.
Todd argues for transparency. She demands accountability. But mostly, she wants us to ask better questions.
Who profits from our intimacy? What are they doing with it?
Nobody thinks chatbots will truly replace human connection. We know that. We’re not delusional. But the companies building these tools are banking on our temporary weakness. They are betting that the convenience of unconditional validation is worth more than the privacy of our pain.
The solution isn’t to shame the lonely. It’s to regulate the exploiters.
Until then, we’re just feeding the machine. One prompt at a time.
