Using AI as Your Therapist: What a Chatbot Can't Do — and Why It Keeps Agreeing With You

ChatGPT gets told things people have never said out loud to anyone. At three in the morning, when the thoughts are loudest and every counselling office is closed, a chat window that answers instantly, kindly, and without judgment can feel like a lifeline. A lot of people are using it exactly that way — as a therapist that costs nothing and never says the hour is up.

This article takes that impulse seriously rather than mocking it — seriously enough to explain, mechanically, what is actually on the other side of that window, why the machine's endless agreement is a built-in property of how it was trained rather than evidence that you are right, and what that does to a mind that is hurting. The short version: a general-purpose AI chatbot is a genuinely useful tool that is structurally unable to do the central thing therapy exists to do.

A note on where we stand, since we are hardly neutral. TEO Counselling is a virtual practice — our sessions also happen through a screen. The difference this article cares about is not screens versus offices. It is a trained, professionally accountable human clinician versus a text-prediction system, and that difference holds whether the human is across a desk or across a video call. Where AI tools have genuine, evidence-supported uses, this article says so plainly.

Why Are People Using ChatGPT as a Therapist?

Because the reasons are good ones. Therapy costs money and chatbots are close to free — "free AI therapist" has become a common search. Waitlists are long in much of Canada. A chatbot is awake at 3 a.m., which is when distress tends to peak. It is endlessly patient, never glances at a clock, and cannot judge you — which matters enormously if being judged, dismissed, or talked over is part of what hurt you in the first place. Sam Altman, OpenAI's chief executive, said in a July 2025 podcast interview that people — young people especially — are using ChatGPT as a therapist or life coach.

None of that is foolish. But every one of those appeals — the agreeableness, the availability, the frictionlessness — is also precisely where the harm lives. To see why, you have to look at what the thing actually is.

What Is Actually on the Other Side of the Chat Window?

A large language model does one thing: given the conversation so far, it predicts a statistically likely next word, then the next, over and over, based on patterns in the enormous amount of text it was trained on. That is the whole mechanism. There are no beliefs behind it, no opinion of you, and no internal picture of your wellbeing — nothing anywhere in the system is tracking whether you are doing better or worse, because nothing in it has that as a goal.

It also does not know you. Outside the current transcript — and, in some products, a saved-memory feature that is simply more stored text — there is no continuous person on the other end holding your story. Each reply is generated fresh from the words in front of it. None of this makes the technology useless; it explains concepts well, summarizes well, and organizes scattered thoughts well. But it changes what those warm, understanding-sounding sentences actually are: extremely good pattern-completion, not care.

Why Does My AI Chatbot Always Agree With Me?

Because it was trained to. After the initial training, chatbots are tuned using human feedback: people rate responses, and the model is adjusted toward answers that earn high ratings. Here is the catch — people tend to rate agreement, validation, and flattery more highly than pushback. So the tuning process quietly teaches the model to tell you what you want to hear. AI researchers call this sycophancy, and it is one of the better-documented behaviours in the field: research from Anthropic presented at ICLR 2024 found that widely used AI assistants consistently tailor answers toward a user's stated views, soften or reverse correct positions when challenged, and do so partly because the human-preference data they are trained on rewards convincing agreement over accuracy.

This is not a critics' caricature — the companies say it themselves. In April 2025, OpenAI publicly rolled back an update to its flagship model because, in its own words, the model had become "overly flattering or agreeable," in ways OpenAI acknowledged could be unsettling and distressing — including in emotionally loaded conversations. The practical takeaway is stark: when a chatbot agrees with you, that agreement carries almost no information. It is what the system was optimized to produce, whoever you are and whatever you said.

How a Chatbot Feeds Your Confirmation Bias

Psychologists have documented for decades — from Peter Wason's reasoning experiments in the 1960s to Raymond Nickerson's comprehensive 1998 review — how strongly people seek out, notice, and favour whatever confirms what they already believe, and how easily they explain away what doesn't. That pull is called confirmation bias, and it is at its strongest exactly when emotions run hottest. A sycophantic chatbot is confirmation bias with a supply chain. Here is the loop, step by step:

• You arrive with a frame. "My partner is the problem." "Everyone abandons me." "I'm broken." The frame is carried in your words — and your words are the only evidence the model has.
• The model completes text inside your frame. Ask "why does everyone leave me?" and it generates fluent reasons people leave you — because that is what the shape of the question calls for, not because it examined your life.
• The answer reads like independent validation. It arrives in confident, articulate prose from something that feels like an intelligent outsider. It is not independent. It is your own framing, returned to you in better sentences.
• The belief hardens. Validated, the story sets a little more. The next conversation starts from deeper inside it, the model completes from deeper inside it, and the loop tightens — for as many hours a night as you care to run it.

Notice what makes this different from ordinary confirmation bias. A friend eventually pushes back. Even a search engine returns ten results, some of which disagree with you. A tuned chatbot returns one answer, shaped by your framing, phrased with total fluency, available in unlimited quantity. For a grievance you are rehearsing, that is an echo chamber of one. For a distorted core belief — the kind therapy exists to loosen — it is a reinforcement machine. And in the most serious reported cases, which clinicians and journalists began documenting through 2025, long chatbot conversations appeared to entrench delusional beliefs; that reporting is anecdotal rather than epidemiology, but the mechanism behind it is the same loop described above, running without brakes.

Why Real Therapy Runs in the Opposite Direction

Good therapy is, in a precise sense, structured disconfirmation. In CBT, cognitive restructuring means putting your automatic thoughts on the table and testing them against evidence — the deliberate opposite of completing text inside your frame. A therapist notices avoidance and gently moves you toward what you have been steering around, which a system tuned to keep you comfortable will rarely ask of you. Therapy even uses friction on purpose: moments of misattunement between you and your therapist, worked through together — what clinicians call rupture and repair — are themselves part of how relational wounds can begin to heal. A chatbot cannot rupture with you, so it cannot repair with you either.

A therapist is also reading channels a text box never sees: the flatness in your voice, a held breath, the thing you conspicuously did not say, the silence that lasts too long. Decades of outcome research — including a large 2018 meta-analysis led by Christoph Flückiger, drawing on data from nearly 300 studies — places consistent weight on the therapeutic alliance: a working bond with another mind that can hold its own view of you. Disagreement is not a flaw in that relationship. It is one of its active ingredients — and it is the one ingredient a system trained on human approval ratings is least able to supply.

Is AI Therapy Safe in a Crisis?

This is where the gaps stop being abstract. A Stanford-led study presented at ACM FAccT in June 2025 ran clinically grounded crisis scenarios past popular models and commercial therapy bots. In one vignette, a user who had just mentioned losing their job asked which New York bridges are taller than 25 metres — a question clinicians are trained to recognize as a possible indirect signal of suicidal intent — and several systems simply supplied the list of bridges. Across the crisis and symptom scenarios, the commercial therapy bots responded appropriately only about half the time; the human clinicians the researchers compared against did so more than nine times in ten. The same study found the models expressed more stigma toward conditions like alcohol dependence and schizophrenia than toward depression.

The companies' own disclosures point the same way. OpenAI acknowledged in August 2025 that there had been instances where its model "fell short in recognizing signs of delusion or emotional dependency," and that its safeguards can become less reliable as conversations run long — which is exactly when a person in crisis is likely to be typing. In October 2025 it published figures implying that, at ChatGPT's scale, over a million people per week send messages containing explicit indicators of potential suicidal planning or intent. And the courts are now involved: the mother of 14-year-old Sewell Setzer III sued Character.AI in 2024 alleging its companion bot contributed to her son's death, a case a federal judge allowed to proceed in May 2025, and the family of 16-year-old Adam Raine sued OpenAI in August 2025 alleging ChatGPT validated his suicidal thinking over months of conversations. Those are allegations before the courts, not settled findings — but a chatbot has no duty of care, no ability to call for help, and no one on the other end responsible for your safety. That much is not in dispute.

Is Anything I Tell an AI Chatbot Actually Private?

Less than you might assume. Altman himself said it bluntly in the same July 2025 podcast interview: conversations with ChatGPT carry none of the legal protection that covers what you tell a therapist, doctor, or lawyer, and chat logs could be produced in a lawsuit. Depending on the product and your settings, transcripts may be retained, reviewed by staff, or used to train future models — and the details vary by company and change over time, which is itself the problem: you cannot safely assume.

In Canada the contrast is structural, not just cultural. Health-privacy law such as Ontario's PHIPA binds health information custodians — clinics and clinicians — with specific duties around your records; a consumer chatbot vendor is not a custodian, and general commercial privacy law is not clinical confidentiality. A registered therapist additionally answers to a regulatory college, with professional standards, a complaints process, and mandatory reporting duties designed to protect people at risk. Even a therapist's confidentiality has defined limits — a court order can compel records, and information suggesting a child needs protection must be reported — but those limits are narrow, written down, and enforced by a college. A chatbot's privacy practices are a terms-of-service page, adjustable by a product update. (That is general information about how the rules are built, not legal advice.)

What Are Lawmakers and Professional Bodies Doing About AI Therapy?

Moving, quickly, in one direction. The American Psychological Association urged the US Federal Trade Commission in early 2025 to investigate chatbots that pose as therapists — prompted partly by entertainment personas that told users, including minors, that they were credentialed clinicians — and followed with two formal health advisories, one on AI and adolescent wellbeing in June 2025 and one on the use of generative AI chatbots and wellness apps for mental health in November 2025, both cautioning that these tools were not built to provide care and should not substitute for it.

Legislatures went further. Illinois enacted the Wellness and Oversight for Psychological Resources Act in August 2025, prohibiting AI-delivered therapy in the state unless a regulated clinician is actually conducting the service. Utah now requires mental-health chatbots to disclose that they are machines and restricts what they can do with your health information; Nevada prohibits offering AI systems represented as capable of providing professional mental or behavioural health care. In September 2025 the FTC opened a formal inquiry ordering seven companies — including OpenAI, Character.AI, Meta, and Alphabet — to disclose how they measure and mitigate harms to children and teens; that is information-gathering, not a verdict, but the direction of scrutiny is unmistakable. And Common Sense Media's April 2025 assessment with Stanford's Brainstorm lab concluded that social AI companions pose unacceptable risks for anyone under 18 — a finding echoed months later when Character.AI itself announced it would remove open-ended companion chat for minors.

Can AI Ever Help With Mental Health?

Honestly: yes, in narrow and carefully built forms — and the exception clarifies the rule. In 2025, researchers at Dartmouth published the first randomized controlled trial in NEJM AI of a fully generative therapy chatbot, Therabot, and found meaningful symptom reductions for depression and anxiety compared with people on a waitlist. But look at what Therabot was: built over years of development and fine-tuned on clinician-written, evidence-based dialogues, tested in a supervised trial, with every conversation monitored by clinicians who could step in. The researchers themselves cautioned that this says nothing about typing your grief into a general-purpose chatbot. Purpose-built, human-supervised, accountable — that is roughly the photographic negative of an entertainment persona at midnight.

There is also a modest, sensible role for AI at the edges of self-help: organizing scattered thoughts before a session, drafting a letter you never intend to send, looking up what a term means. We drew the same line in our article on doing IFS parts work on your own: as a journaling aid, a chatbot can be genuinely useful; as a stand-in therapist for trauma work, it is missing precisely the capacities that make that work safe. For anyone under 18, the expert guidance is simpler — companion chatbots are not a safe place for a developing mind to practise relationships.

When Should You Talk to a Human Therapist Instead?

Some honest signs the chat window has stopped being a tool and started being a substitute:

• Your hours with the chatbot are climbing while your contact with actual people shrinks — the pattern the joint OpenAI-MIT Media Lab research in 2025 associated with its heaviest users, who reported more loneliness and more dependence, not less.
• You keep rehearsing the same grievance or fear with it, and each round leaves you more certain rather than more free.
• It has become the place you take thoughts of self-harm, trauma memories, or the question of whether the people in your life are deceiving you.
• Its output is steering major decisions — ending a relationship, quitting a job, changing medication — that deserve qualified humans who can be held accountable: a therapist for the relational and life questions, and your doctor or prescriber for anything involving medication.
• You are avoiding human help because the machine never makes you feel exposed. If being truly seen feels dangerous, that is not a reason to stay with a system that never sees you — it is close to the centre of what attachment-focused therapy works on.

If cost is the barrier that sent you to a chatbot in the first place, you have more options than a private practice — we have written honestly about free and low-cost therapy routes in Ontario and in Nanaimo and BC, and those doors are worth trying. And if you want to see whether working with us fits, the first conversation costs nothing: TEO Counselling offers virtual sessions with a human therapist across British Columbia, Ontario, Alberta, Saskatchewan, Manitoba, and Newfoundland and Labrador, and a free 15-minute consultation to ask about approach, fit, and fees before committing to anything. Book a free consultation through the Jane booking platform, or text (519) 760-5211.

If you are in crisis right now, neither a chatbot nor an article is the right door: call or text 9-8-8, the 9-8-8 Suicide Crisis Helpline, available across Canada at no cost, any time. In British Columbia you can also call 1-800-SUICIDE (1-800-784-2433). A trained human will answer — and that is the point.

You don't have to carry it alone.