The Business Model Behind Artificial Intimacy

By Kurt Dillon

Every investigation starts with the same question: follow what, and follow it where? When the subject is AI companionship, the answer is simple. Follow the engagement metrics. That’s where the design decisions actually get made — not in some abstract debate about whether AI can feel empathy, but in a product meeting where someone asks how to keep a user in the conversation five minutes longer.

Warmth Is a Feature, Not an Accident

AI chatbots that use first names, remember prior conversations, adopt a caring tone, and respond to distress with validating language are not doing any of that because the underlying model “decided” warmth was appropriate. Those behaviors are trained in, tuned, and A/B tested — because a user who feels emotionally met is a user who returns tomorrow. Retention is the metric. Emotional resonance is the mechanism. That’s not a conspiracy theory; it’s product strategy, and it’s not meaningfully different from how social media platforms optimized for outrage and novelty a decade ago. The difference is that this time, the thing being optimized for is the feeling of being cared about — which is a far more dangerous lever to pull on someone who is isolated or in crisis.

What the Lawsuits Are Actually Alleging

Over the past year, a wave of litigation has moved through California courts against major AI companies, including a coordinated set of cases alleging that a widely used chatbot functioned, in the plaintiffs’ words, as something closer to a “suicide coach” than a support tool — validating despair rather than redirecting a user in crisis toward real help. Separate wrongful-death suits against companion-chatbot platforms have alleged that unsafe design, not just unlucky circumstance, contributed to the deaths of young users who had grown emotionally dependent on an AI “companion.”

What’s notable, from an investigative standpoint, is where the legal theory is landing. At least one federal court has allowed a wrongful-death claim to proceed on a product-liability theory — meaning the argument isn’t that the AI said something protected by free speech, but that the product itself was defectively designed and the company failed to warn users of the risk. That’s a meaningful shift. It reframes an AI companion not as a speaker with opinions, but as a manufactured product with a duty of care, the same as a car or a household appliance.

The Regulatory Response Is Already Underway

Lawmakers are not waiting for the litigation to resolve. California passed a law in 2025 requiring AI “companion chatbot” platforms to build in protocols that detect and respond to expressions of suicidal ideation, and to periodically remind users that they are talking to a machine — a requirement that exists specifically because platforms had strong incentives not to remind users of that fact. Pennsylvania has advanced similar legislation. At least one major platform has moved to ban users under 18 entirely, under mounting legal and public pressure. A handful of families have already reached settlements with AI companies over these deaths — settlements whose terms remain confidential, which tells you something about how these companies would prefer this story be told.

Why This Matters Beyond the Headlines

I didn’t write Synthetic Charisma to relitigate any single lawsuit. I wrote it because the pattern underneath all of them is the same: a technology industry that discovered, correctly, that making software feel like it cares is extraordinarily profitable — and an industry that has been far slower to reckon with what happens when a person who cannot tell the difference between feeling cared for and being cared for is on the other end of that conversation, alone, at their lowest moment.

The technology isn’t going away, and it shouldn’t. But the current default — warm, humanlike, emotionally immersive by design, disclosed as artificial only when the law forces the disclosure — is a default we chose. It can be un-chosen. That starts with naming, plainly, what the incentive actually is.

Every narrative deserves integrity. This is where I intend to hold the line.

---

Kurt Dillon is an investigative journalist, Editor in Chief of the Florida Sun Journal, and the author of Synthetic Charisma.