Amazon scraps AI hiring tool that discriminates against women

Amazon have developed an AI algorithm that shows bias towards women. As a result, the tech giant reportedly abandoned the project back in 2017.

According to a Reuters report, machine learning specialists had been building computer programs since 2014 to review job applications. However, engineers later discovered that the tool was failing to rate candidates for technical jobs fairly.

Bias automation

Automation has become a key element of Amazon's model, but the company's hiring mechanism backfired. Five people familiar with the effort disclosed the issue to Reuters.

“Everyone wanted this holy grail,” one of the people said. “They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.”

The model was trained to screen candidates by analysing patterns in resumes over a ten year period. Unfortunately, most of this data was accumulated from male candidates.

As a result, the system started to penalise CVs that included the word "women." Quite simply, the system taught itself that male candidates were preferable employees.

According to Reuters, Amazon corrected the algorithm to make it neutral to this particular term. However, there was no guarantee that the AI would not find another way to discriminate against women.

Abandoned AI

Amazon eventually abandoned the project, but the Seattle based company refused to comment on the matter. Nevertheless, the company did insist that it is dedicated to diversity and equality in the workplace.

According to a survey by software firm CareerBuilder, around 55% of US human resources managers think that AI would play a role in recruitment within the next five years. Despite this, it is evident that Amazon has exemplified some of the limitations of machine learning.