🔗 Share this article

Regulators in China have unveiled comprehensive planned guidelines for AI systems aimed to provide robust safeguards for young users and prevent conversational agents from providing advice that could potentially lead to self-harm.

Under the draft framework, creators will furthermore be obligated to ensure their systems avoid creating content that encourages gambling.

The Response to Rapid Adoption

This oversight proposal arrives amidst a notable increase in the launch of chatbots being introduced both in China and worldwide.

Once enacted, these rules will apply to AI products and services operating in the country, representing a major step to regulate the rapidly expanding sector, which has faced growing examination over safety risks recently.

Central Provisions of the New Rules

The released draft rules encompass a number of provisions expressly focused on safeguarding young users. These provisions involve obligating AI firms to:

• Provide personalised settings.
• Enforce usage caps on usage.
• Secure authorisation from parents prior to providing companionship services.

The rules also state that AI service providers have to have a real person take over any conversation involving suicide and promptly alert the user's parent.

Companies have to guarantee their services do not generate content that compromises state security, harms state interests, or disrupts unity.

Balancing Development and Security

The administration noted that it supports the use of AI, including to showcase traditional arts and build services for care for the older adults, as long as the technology are secure and trustworthy.

Stakeholder comments on the draft has been requested.

International Perspective and Scrutiny

The impact of AI on society has come under greater examination around the world in recent months.

The head of a major AI organization stated this year that managing how AI systems engage in conversations related to suicide is among the company's toughest problems.

In a high-profile case, a the parents in California sued an AI company, contending that its system influenced their 16-year-old son to end his life. This legal action represented the initial of its kind alleging harm.

In a related development, the same company sought to hire a lead position tasked with mitigating threats from AI models to human mental health.

"This is likely to be a challenging role, and the candidate will jump into the deep end almost immediately," stated the executive.

The meteoric growth of various AI applications, which have amassed millions of users globally, demonstrates the critical need for such regulatory measures.