Character.AI to shut down teens from accessing its chatbot amid rising concerns over teen safety

AI startup Character.AI has said that it will ban users under 18 from having conversations with its chatbot. The comapny has announced that users under 18 will immediately get to have only two hours of ‘open ended chats’ with its AI and it will turn into a complete ban by 25th November. The decision by the company as there is mounting pressure on AI companies about the potentially harmful impact of their products on the mental health of teens.

Until the complete ban comes into effect, the AI startup says it will work to build an under-18 experience which gives teens ways to be creative like creating videos, stories and streams with characters.

Character.AI says it has built an in-house model to determine user age and will also be using tools from age verification startup Persona to ‘to help ensure users receive the right experience for their age.’

Why is Character.AI banning minors?

Explaining the reason behind the changes, Character.AI after receiving questions from regulators ‘about the content teens may encounter when chatting with AI and about how open-ended AI chat in general might affect teens’

“We do not take this step of removing open-ended Character chat lightly – but we do think that it’s the right thing to do given the questions that have been raised about how teens do, and should, interact with this new technology.” the company said in a blogpost.

In recent days there have been a number of lawsuits coming against AI companies with parents alleging that their teenager died by or attempted suicide after interacting with a chatbot.

In one case filed in Florida against Character.ai, the complainants alleged that a 14 year boy died by suicide after getting attached to a chatbot impersonating Game of Thrones character Daenerys Targaryen.

In another case, a family has alleged that Character.AI chatbot encouraged their 17 year old son towards self-harm and suggested that murdering his parents would be a ‘reasonable’ solution.

Just last month, US Federal Trade Commission (FTC) also launched an inquiry into seven AI comapnies including Character.AI, Meta, OpenAI, Google (Alphabet), Snap, and xAI to seek information on how they evaluate, test and monitor the potential negative effcts of their chatbots on teens. A new law in California also restricts how AI chatbots can react to users. Meanwhile, a new bill was also recently proposed in the US Senate which seeks to ban all ‘companion’ AI chatbots while ensuring stringent measures to ensure that all users are above 18.