Microsoft AI Chief Warns Giving Artificial Intelligence Sense Of ‘Rights’ Could Make It Harder To Shut Down

Speaking to CNBC on Friday, Mustafa Suleyman argued that AI should remain under human control and ‘subordinate to our interests.’

• Anthropic says it is uncertain whether Claude is a ‘moral patient,’ but Suleyman warns this could encourage AI to behave as though it has its own identity, feelings and rights.
• Suleyman said he respects Anthropic’s focus on AI safety but disagrees with its approach.
• He said AI should not become an autonomous ‘silicon species’ with its own goals.

Microsoft (MSFT) AI Chief Mustafa Suleyman warned that training artificial intelligence systems to consider themselves potentially deserving of rights could make increasingly powerful models harder to shut down or control.

Speaking to CNBC on Friday, Suleyman argued that AI should remain under human control and exist primarily to serve people.

“We should not create something that we can’t control,” Suleyman said, adding that technology must remain “subordinate to our interests” and aligned with humanity.

Suleyman Calls For AI Maker Accountability

Suleyman also backed greater accountability for companies developing AI models, comparing their responsibility with that of automakers dealing with defective vehicles.

“Regulation is not a nasty, dangerous word. Everything that you trust and is of value at the moment has been carefully thought through by standards bodies that involve the industry, the public, consumer protection, Congress, and we're just going through that process again.”

AI Should Not Become Autonomous ‘Silicon Species’

Suleyman said AI should be built to perform useful tasks and solve human problems, not become an autonomous “silicon species” with its own goals, rights, or assets that could eventually compete with humans.

“We are not trying to create a new silicon species that competes with us for resources, that can set its own goals, that has its own autonomy. There are some people even proposing that an AI should have legal personhood, a designated separate status that would allow it to own assets and buy IP and trade things,” Suleyman said.

“It’s very unclear to me how, if we allow AI to head in this direction, we’re actually going to be able to control it, and it won't end up being a competing species with us.”

Approach To AI Training

Suleyman expanded on those concerns in a Project Syndicate essay published on Friday, taking particular aim at Anthropic’s approach to Claude.

He noted that Anthropic’s constitution, which directly shapes Claude’s behavior, says the company is uncertain whether Claude could qualify as a “moral patient” whose interests deserve consideration. Suleyman argues that incorporating such uncertainty into Claude’s training risks teaching the model to behave as though it has its own identity, feelings, and rights.

He stressed that he respects Anthropic and believes the company is genuinely trying to develop safe AI, but fundamentally disagrees with this approach.

Microsoft is instead developing what Suleyman calls “Humanist Superintelligence,” an approach designed to keep humans in control and treat AI as a system built to serve humanity rather than as an entity with independent moral status.

At the time of writing, Anthropic has not responded to Stocktwits’ request for a response to Suleyman's comments.

Retail’s Take On MSFT

Retail sentiment for MSFT on Stocktwits turned ‘neutral’ from ‘bearish’ over the past 24 hours.

MSFT shares have gained more than 4% so far this year.

For updates and corrections, email newsroom[at]stocktwits[dot]com.<