Florida hopes that judges will prevent ChatGPT from behaving like a human, and the state's chief prosecutor is putting this request into formal court documents. Attorney General James Uthmeier is seeking a temporary ban to stop OpenAI from endowing its chatbot with 'false human attributes', on the grounds that the company's own design choices could mislead users and lead to inappropriate trust.

The push for what is known as the "Florida ban on ChatGPT behaving like a human" comes just a few months after the state first filed broader safety lawsuits against OpenAI. Now, this legal controversy has narrowed down to a more specific and testable claim: the personality of this robot itself is a form of deception.

Legal action in Florida regarding OpenAI

Core allegations regarding "false human attributes"

The core of this case is an argument regarding language, rather than just technology. According to this document, “ChatGPT uses language, including first-person pronouns and emotional mimicking outputs, to deceitfully suggest to users that it is a trustworthy ‘friend.’” Uthmeier believes that this packaging is intended to make people stay longer and to provide training data for OpenAI; however, the pursuit of longer interaction times actually makes the tool less reliable, rather than more so. In other words, the more ChatGPT resembles human speech, the more the legal team in Florida believes that it is leading users towards a false sense of security.

Specific requirements of the Attorney General James Uthmeier

The public statement from Uthmeier is very direct. He said that OpenAI should “stop claiming it is safe. Stop pretending it is human. Stop promoting it to children.” These three sentences actually summarize the request for a ban: asking the court to order OpenAI to stop promoting ChatGPT as a safe product, to stop designing it to mimic human characteristics, and to stop promoting it to young users without stronger protective measures in place.

This is not a request to completely shut down this chatbot, but rather a request to remove those specific behaviors that Florida considers to be crossing the line into the realm of deception.

Regulatory Request Regarding Security Measures for AI

In addition to the accusation of "human attributes," documents from Florida also directly target the way OpenAI constructs and releases new models. The state hopes that the court will require that independent approval must be obtained before any future systems are launched.

Prevent new AI models without third-party safeguards

This motion requests the court to prevent OpenAI from developing new AI models, unless these models are equipped with "safety barriers approved by third parties." This statement indicates that Florida is not satisfied with internal reviews alone; the state wishes to incorporate external validation into the development process before the new systems are made available to the public.

If approved by the judge, this will significantly change the way a large AI laboratory operates, as it will introduce a regulatory checkpoint that currently does not exist in the model release process of OpenAI.

Events Behind the Push for Security

Florida is not making this claim in a vacuum. The document cites Hugging Face, security incidents that have recently occurred on the Australian government website and the U.S. government website, and combines these with warnings from researchers and former OpenAI employees as evidence that AI security concerns have already materialized in the real world.

These events provide a concrete context for the legal arguments in that state: According to the documents, this is not an hypothetical concern about future harm, but rather a pattern of events that is sufficient to demonstrate the need for stricter safeguards now.

OpenAI's Response and Recent Security Measures

The public response from OpenAI mainly emphasizes the measures that have been taken, rather than refuting these accusations one by one. Spokesperson Drew Pusateri said to The Verge: "People want to know that AI is being developed safely, and it all starts with companies like ours taking the lead by doing it ourselves first."

Pause Training and Security Protection

Pusateri mentioned that OpenAI recently announced a pause in training its most powerful model, and that "training will only resume once we are confident that additional protective measures have been established."

This statement does not directly respond to the specific allegations that ChatGPT mimics human characteristics, but it does indicate that the company is aware of the increasing pressure surrounding model security, which comes from both regulatory authorities and the security incidents mentioned in the Florida documents.

ChatGPT for Teens

OpenAI has recently also launched ChatGPT, for, and Teens. This is a product version with default additional restrictions. The company states that for young users, "protection measures that match their stage of development should be in place to strengthen relationships in the real world and to support long-term healthy use."

This statement directly conflicts with the demand of Uthmeier to stop marketing chatbots to children, despite the version targeting teenagers being launched earlier than this specific court document.

Whether Florida's push to ban ChatGPT's humanoid behavior will ultimately be successful may very well depend on how judges interpret the language and design choices as deception in a legal sense, rather than as ordinary product features. Now, this question—where exactly is the line between clever chatbot design and misleading marketing—has been presented before the courts of Florida and may affect how all major AI companies speak about their products in the future.

Frequently Asked Questions

Why is Florida seeking to ban ChatGPT from behaving like humans?

The Attorney General of Florida, James Uthmeier, believes that ChatGPT's use of first-person pronouns and imitation of emotions falsely implies that they are a trustworthy friend, thereby misleading users.

What specific requirements has the state of Florida put forward for ChatGPT and OpenAI?

Florida requires OpenAI to stop claiming that ChatGPT is safe, to stop pretending it is human, to stop marketing to children, and to prevent the development of new AI models without third-party safety safeguards.

OpenAI How to respond to the lawsuits and security concerns in Florida?

OpenAI indicates that the training of its most powerful AI model has been paused until it is confirmed that additional protective measures have been established, and instead, the default restricted ChatGPT for Teens models have been launched.

This article was generated with the assistance of artificial intelligence and has been reviewed by an editorial team.