Call of Duty Takes Aim at Hate Speech with AI Voice Moderator

Activision’s popular shooter Call of Duty is arming its moderation team with an AI voice chat moderator in a bid to target hate speech during online matches.  

The AI-powered moderator, dubbed ToxMod, will be powered by Modulate and will scan conversations for spoken violations of the game’s code of conduct. 

“Call of Duty’s new voice chat moderation system utilizes ToxMod, the AI-Powered voice chat moderation technology from Modulate, to identify in real-time and enforce against toxic speech—including hate speech, discriminatory language, harassment and more,” Activision wrote in a blog post. 

“This new development will bolster the ongoing moderation systems led by the Call of Duty anti-toxicity team, which includes text-based filtering across 14 languages for in-game text as well as a robust in-game player reporting system.”

Modulate hasn’t given many details about how exactly ToxMod will work. Its website states that the tool “triages voice chat to flag bad behaviour, analyses the nuances of each conversation to determine toxicity, and enables moderators to quickly respond to each incident by supplying relevant and accurate context.” 

The company’s CEO said in a recent interview that the tool aims to go beyond mere transcription, considering factors like a player’s emotions and volume in order to differentiate harmful statements from playful ones.

What we do know is that the tool will not actually take action against players based on its data but instead submit reports to Activision’s human moderators. 

Human involvement, for now, remains a crucial element for AI voice moderation tools, as research has shown that speech recognition systems can display bias in the way they respond to users with different racial identities and accents. 

A hotbed for hate speech

First-person shooters like Call of Duty have become somewhat infamous for the high levels of hate speech and inappropriate behaviour across their lobbies and voice chats, with women and minorities among those particularly targeted. 

The issue is exacerbated in popular multiplayer games like Call of Duty due to the large number of players, with around 90 million people playing Call Of Duty each month.

Surveys have determined the franchise’s fan base the most negative in all of gaming, with one feud between two players even resulting in the false summoning of a SWAT team.

Activision has been trying to down on this behaviour for years, and the company says ToxMod may be part of the solution to its problem. 

In examining the data focused on previously announced enforcement, 20% of players did not re-offend after receiving a first warning. Those who did re-offend were met with account penalties, which include but are not limited to feature restrictions (such as voice and text chat bans) and temporary account restrictions

This positive impact aligns with our strategy to work with players in providing clear feedback for their behaviour, Activision said in their blog post.

Activision's chief technology officer Michael Vance said it would help make the game "a fun, fair and welcoming experience for all players".

The new moderation system began a beta test starting Wednesday, covering North America initially and focusing on the existing games Call of Duty: Modern Warfare II and Call of Duty: Warzone.

The full rollout of the moderation technology is planned to coincide with the launch of Modern Warfare III, beginning in English, with other languages added over time.

For 12 years, BIG DATA & AI PARIS has provided a unique ecosystem where leading French data and AI experts meet to discuss new and exciting trends in the big data and AI space.

Taking place on September 25-26 2023, the exciting event offers a unique opportunity to learn about the latest market trends and network with all data and artificial intelligence professionals from France and around the world!

Attendees can obtain a complete vision of AI with three complementary conference courses (strategy, feedback, technique), find the answer to their business problems by talking to exhibitors on-site and discover the latest technologies and solutions on the market through our workshops and demos.

REGISTER NOW!