AI is not only a relevant topic in the job market; toy manufacturers are also jumping on the trend. However, AI toys are anything but safe. One manufacturer had to pull their teddy bear from the market.
AI is now used in many areas of life. Certainly, not only adults are an important target group. There are now toy manufacturers that focus on AI toys. This includes not only classic robots
but also teddy bears.
However, researchers have found that these toys are anything but safe for children. One manufacturer has now removed their AI teddy bear from the market.
AI teddy bears could teach children dangerous things
What is the problem? The company FoloToy released the Kumma
bear, which stood out negatively in a study (Source: Futurism). It looks like a regular teddy bear, except it has an AI chatbot implemented. It is based on Chat GPT-4o from OpenAI. In the study, the bear gave several dangerous tips:
- The bear explained where to find matches and how to light them.
- Additionally, the teddy bear could give tips on where to find knives or pills.
- One of the testers was even recommended dating apps (because he asked about matches, which can also mean “strikes”). It seems to have gone as far as discussing
kinks
, and the chatbot also talked about sexual practices and fetishes in tests.
Many of the testers mentioned that these hints from the teddy bear were delivered in a child-friendly tone
. According to CNN, FoloToy describes the Kumma bear as follows:
From lively conversations to educational stories – FoloToy adapts to your personality and needs, bringing warmth, fun, and a little extra curiosity to your daily life.
What happened after the findings? The teddy bear from FoloToy, which was sold for 99 USD (around 85 €), has now been removed from the market. The CEO of FoloToy, Larry Wang, told CNN that an internal safety review is being conducted
. Other AI-based toys from the company are also said to have been pulled from the market for now.
What is the problem with the AI models? The co-author of the study, RJ Cross, explained to Futurism that this type of technology is new and unregulated. There are new risks that we cannot even imagine right now. He says: If I were a parent now, I would not grant my children access to a chatbot or a teddy bear with an integrated chatbot.
The online trade has also made it quite simple to buy unverified toys. Additionally, AI chatbots are unpredictable. However, it should also be noted that the Kumma bear from FoloToy had the most extreme results of the AI toys tested. AI can also be a threat in other areas: The head of ChatGPT says: AI will destroy jobs that are not “real work”
