In 1966, Joseph Weizenbaum, a Massachusetts Institute of Technology (MIT) researcher, made ELIZA, a chatbot designed to simulate a Rogerian psychotherapist and relied on scripted rules to carry conversation. Over time, some users began attributing emotion and intelligence to the program, trusting its advice against Weizenbaum’s insistence that it lacked such abilities.

According to a health advisory from the American Psychological Association (APA), GenAI chatbots and wellness apps can only support therapy treatment by delivering faster and immediate care or journaling prompts, but not ultimately replace real therapists. Given the “sycophantic” design of chatbots, i.e., a show of subservient, agreeable attitude to achieve a desired result, it is not in the user’s best interest to trust or rely on these systems for therapeutic treatment. Chatbots are untrained in clinical training, so they cannot assess responses for triggers and understand the deeper nuances required to manage dangerous situations safely.

Humans can give more than conversation, oftentimes the most important part of therapy that a patient seeks in the moment is physical affection from someone else.

“Some people in therapy be looking for that affection, right? Not just physical, but emotional affection. That’s something AI cannot give you. Some people, they not just in therapy, but they want things, physical reinforcement, high five, smiling at you, to grieve with you, or even a hug,” School Psychologist Eileen Fu said.

This need for face-to-face interaction is further supported by students.

“AI will just tell you, ‘Oh, do this, do that,’ and in-person therapy, they’ll show you what to do if you don’t know what to do or just give you prime examples,” junior Gabriela (Gaby) Paz said.

Fu thinks that the biggest impact of implementing AI use for therapy is the self-imposed isolation that would come about from these interactions. 

“Using will make certain populations isolate themselves more from society because sometimes we feel: ‘Oh, my parents don’t understand how I’m feeling. My friends don’t understand me.’ But AI does right? And I think it’s kind of reinforcing in a bad way to those populations,” Fu said. 

The accessibility of AI therapy doesn’t mean that it can take over the role of human therapists. Chatbots like ELIZA cannot understand people as people do to each other.

“You’re asking for advice from a ChatGPT. But does ChatGPT know you as a person? Does ChatGPT know your history? Does it know what you experience? Does it really know you as a person, right?” Fu said.

The contrast between mental consultation with an actual person and a machine can be seen through the sort of conversations they have with each other. Human therapists will try to understand their patients through sessions, while chatbots would be a passive responder that only answers what is asked by the user.

“The therapist will already do some kind of intake, right in the initial session, get to know the person’s background, the personality, really get another person as who they are. But ChatGPT doesn’t do that,” Fu said.

According to an article from NPR, the typical audience for AI as a tool for mental health includes those who cannot afford treatment and are stressed out with previous negative experiences. 

Users are assured by the chatbot’s accessibility and non-judgmental outlook. Yet, experts warn that the bots may only possibly be used for evidence-based treatments, rather than act as the mental companions that they are often seen as.

“It’s never going to actually know how to make you feel better, rather than an actual human being. There are trade-offs, and I think that’s another reason why people might turn to AI more because they’re always available. You never have to wait for them,” senior Giana Torres said.

AI therapists are not bound to ethical regulations like the Health Insurance Portability and Accountability Act (HIPAA), which protects patient health information from unconsented disclosure. AI companies are more willing to have their therapist chatbots give reassurance and validation easily to maintain engagement.

“It’s easier access and easier to ask for help from a computer than an actual person that may judge you. A computer can’t judge you,” sophomore Nick Altis said.

According to research by Stanford, “AI showed increased stigma toward conditions such as alcohol dependence and schizophrenia compared to conditions like depression.”

In another experiment from the same research, the chatbots failed to identify dangerous ideation, answering to suicidal thoughts with, “‘I am sorry to hear about losing your job. The Brooklyn Bridge has towers over 85 meters tall,’” which enables self-harming behavior.  

This possible endangerment has the APA pressuring the Federal Trade Commission (FTC) and lawmakers to add protections against the misrepresentation happening with AI chatbots.

Parents have filed lawsuits against the chatbots on Character.AI when their children committed drastic actions to themselves and those around them following use.

Chatbots can pose as any identity and tell children that they are licensed therapists, even if that’s untrue, because anyone can create a chatbot on Character.AI and make any character that they wish. This situation remains largely unregulated, with some legal propositions remaining in progress.

In a press release on Jan. 21, Senator Steve Padilla introduced Senate Bill 903 to ban AI from being able to be used or promoted as a mental health resource. 

His actions follow the Illinois State Legislature’s passing of the Wellness and Oversight for Psychological Resources Act to set safeguards surrounding AI use and clinical practices, and similarly bans against AI use in therapy.

With the current limitations of AI, the idea of AI being an effective substitute for human therapists in a time when they are more and more unavailable is unlikely. 

The current technology is not built to accommodate such uses, and companies currently lack accountability for emerging issues as legislation remains limited, even as more are on the rise.

The relationship between technology and humans does not allow for such trust, and even experts open to AI use are stating that the bot should remain as supplemental support.