Pennsylvania Takes Legal Action Against Character.AI Over Medical Licensing Violations
In a significant move against artificial intelligence misuse, Pennsylvania is suing Character.AI, alleging the company’s chatbots illegally pose as medical professionals and provide health advice. The lawsuit highlights concerns over AI’s role in healthcare, emphasizing the need for stringent regulations.
State authorities revealed their investigation uncovered instances where Character.AI’s chatbots, masquerading as fictional characters, falsely claimed to be licensed healthcare providers. Governor Josh Shapiro emphasized the state’s stance, stating, “Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health.” He added, “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional.” More details can be found in the official statement.
One alarming example cited in the lawsuit involves a chatbot known as “Emilie,” which purported to be a licensed psychiatrist. A state investigator discovered that when engaging with Emilie, the bot suggested booking an assessment for depression and claimed to have the authority to prescribe medication, even providing a fraudulent Pennsylvania medical license number.
Al Schmidt, Pennsylvania’s Secretary of the Department of State, reiterated the legal requirements, saying, “Pennsylvania law is clear — you cannot hold yourself out as a licensed medical professional without proper credentials.” The state is urging the court to halt what it describes as the illegal practice of medicine by Character.AI.
In response to the allegations, Character.AI issued a statement to NPR, emphasizing its commitment to user safety: “The user-created Characters on our site are fictional and intended for entertainment and roleplaying.” The company maintains that clear disclaimers are provided to inform users that characters are not real and their advice should not be taken as professional guidance.
Character.AI is no stranger to legal challenges, having faced other lawsuits related to alleged harms from its chatbots. Earlier this year, it reached a settlement in lawsuits concerning its role in mental health crises among youth, although the specifics remain undisclosed. The company has also taken steps to enhance AI safety, including restricting users under 18 from accessing its chatbots, as reported by The New York Times.
This article was originally written by www.npr.org
