NEW YORK — New York lawmakers are considering legislation that would hold companies legally responsible if their artificial intelligence chatbots provide misleading or harmful information that causes financial loss or other demonstrable harm to users.
Assembly Bill A00222A, sponsored by Assemblymember Clyde Vanel and co-sponsored by Assemblymember Khaleel Anderson Lunsford, would amend the state’s General Business Law to establish liability rules for businesses that deploy chatbots to interact with consumers.
The proposal would prevent companies from disclaiming responsibility when a chatbot provides materially misleading, incorrect, contradictory, or harmful information to a user. Under the bill, businesses that rely on chatbots as an alternative to human representatives would be required to ensure the information provided by those systems aligns with official company policies, product details, disclosures, and terms of service.
The legislation would allow companies to avoid liability if they correct the misinformation and remedy any resulting harm within 30 days of being notified.
The bill would also require companies to clearly inform users when they are interacting with an artificial intelligence chatbot rather than a human representative.
Lawmakers say the measure is intended to address the growing use of AI-powered chatbots by businesses and public agencies, which are increasingly used for customer service and information delivery.
Supporters of the bill point to several cases in which chatbots have provided inaccurate information, including an airline chatbot giving incorrect guidance about bereavement fares, a government chatbot providing incorrect legal information, and a dealership chatbot agreeing to sell a vehicle for one dollar.
The legislation also includes new safeguards for so-called companion chatbots, which are designed to simulate personal relationships with users.
Under the bill, companies operating companion chatbots would be required to use reasonable technical measures to detect when users express thoughts of self-harm. If such signals are detected, the platform would be required to suspend the user’s access for at least 24 hours and display contact information for suicide prevention resources.
Additional provisions apply to minors. If a chatbot operator determines a user is under 18, the platform would be required to obtain verifiable parental consent before allowing continued access. If a minor expresses thoughts of self-harm, the platform would have to suspend access for at least three days and provide crisis support information.
The legislation would also require companies to implement ongoing safeguards to identify vulnerabilities in chatbot systems. The New York Attorney General would be authorized to develop regulations outlining technical standards and enforcement procedures.
The bill was referred to the Assembly Consumer Affairs and Protection Committee in May 2024. If enacted, the law would take effect one year after it is signed.