Twitter Turns to AI to Tackle Surge in Hate Speech

Twitter plans to employ more AI systems than human moderators in its attempt to combat the increasingly prevalent abusive content on the site, according to reports. 

Researchers from the Centre for Countering Digital (CCDH) reported that hate speech and racism have increased exponentially on the platform since Elon Musk’s takeover in October.  

Ella Irwin, Twitter's new Head of Trust and Safety, said that Twitter will moderate content through the automated restriction of words and phrases related to hate speech rather than relying on the manual moderation of abusive content.  

She told Reuters “The biggest thing that’s changed is that the team is fully empowered to move fast and be as aggressive as possible” in its response to hateful content.  

But as Twitter struggles to moderate content on its platform following the layoff of roughly 11,000 staff, experts warn that Twitter’s AI-wired approach may not be enough the combat the surge in abusive content  plaguing the site. 

Musk’s Twitter: “a safe space for hate”  

When Elon Musk took over Twitter in October, the rate of violent and abusive content increased at such a rate that organisations and senior officials have decided to depart ways with the platform. 

Musk introduced new regulations on the platform that transformed its verification and moderation systems in order to favour free speech over censuring, which in turn led to around 60,000 previously banned accounts being reinstated on the platform.

According to several organisations monitoring cyber-social threats online, these regulations have opened the floodgates to a tsunami of racism, antisemitism and homophobia. 

Vicious hate is spiralling out of control on Elon Musk’s Twitter.

Our research on Twitter’s hate speech epidemic in @itvnews. pic.twitter.com/VRVrJ6ChFr

— Center for Countering Digital Hate (@CCDHate) December 3, 2022

“From racial slurs tripling to a shocking increase in antisemitic and misogynistic tweets, Mr Musk’s Twitter has become a safe space for hate,” CCDH said on Friday, adding that misinformation had also risen since the billionaire's takeover of the platform. 

Another research organisation, the Centre for Countering Digital Hate, found that the use of the N-word on Twitter increased by nearly 500 per cent in the 12 hours after Mr Musk’s takeover of the platform. 

It also found more than 50,000 posts including homophobic and transphobic slurs, up by 53 per cent and 39 respectively from last year. 

Despite claims that the site had reduced the prevalence of hate speech on its search and trending pages from Twitter’s former head of Trust and Safety, “the actual volume of hateful tweets has spiked,” according to the centre.  

Is AI-wired moderation enough? 

In response to these concerns, Ms Irwin announced on Thursday that Twitter would “aggressively” restrict abusive hashtags and search results relative to abusive content such as child exploitation. 

But even with these integrations, reports suggest that the site is still struggling to contain the overflow of abusive content due to its human rights and machine learning ethics team being reduced to no or little staff following the company’s large-scale layoffs.

At the end of November, several videos of a white supremacist who murdered 51 Muslim worshippers in 2019 – footage that is illegal to share in New Zealand – were not caught by the platform’s AI moderation tools at the end of last month. 

The clips were only removed after the country’s government told Twitter about the content’s presence on the microblogging platform. 

Meanwhile, last weekend, the site’s automated moderating systems were unable to pick up a bombardment of huge adult spam content that researchers said was an attempt to obscure news about widespread protests across China.

“This is a known problem that our team was dealing with manually, aside from automation we put in place,” an ex-Twitter staff member told The Washington Post.

Regardless, Musk and Twitter executives stand firm on their decision to reduce content moderation on the platform. 

Ms Irwin said on Thursday that the reduction in staff has affected the effectiveness of Twitter’s moderation teams, adding that the site needs to begin relying on trusted figures with a track record of correctly flagging content to report content when they see it.

To read more about AI, visit our dedicated AI in the Enterprise Page