Skip to main site content.
This illustration picture taken on April 29, 2018, shows the logo of Amazon displayed on a screen and reflected on a tablet in Paris. (LIONEL BONAVENTURE/AFP/Getty Images)

Systemic bias

Amazon’s secret AI project to evaluate job candidates discriminated against women, sources say

October 12, 2018

Online retail giant Amazon was reportedly forced to abandon a secret project to automate its recruitment process after it was discovered that the artificial intelligence powering its learning algorithm was systematically devaluing the job applications of women. Speaking under condition of anonymity, five sources familiar with the project told Reuters that in 2014 Amazon had assembled a team to build computer programs that would automatically review lists of resumes and output the best candidates from the bunch. Unfortunately, the learning algorithm was trained on the patterns of resumes submitted to the company over the past decade — which, in a heavily male-dominated sector, almost all came from men.

The resulting program learned to penalize resumes that contained the word ‘women’ or the names of at least two-all women’s colleges, and to favor candidates who used words more typically found on male engineers’ resumes. Developers attempted to modify the computer model to make it neutral to those particular terms, but it was impossible to tell if the algorithm wouldn’t find other ways to discriminate against women candidates. Other problems within the model that led to unqualified candidates being repeatedly recommended for jobs forced executives to scrap the recruitment automation effort last year, sources said. Prior to the end of the project, Amazon recruiters allegedly did examine the results generated by the tool when evaluating new hires, but did not make final hiring decisions solely on that basis.

Amazon officials have refused to comment on the problems with the ill-fated AI. According to Reuters, the company did not deny that recruiters had looked at the computer models, but did say the algorithm “was never used by Amazon recruiters to evaluate candidates.”

Read the full story at Reuters.


Ex-Google engineer sues company for discriminating against white, male conservatives

Uber’s chief brand officer says white men should bring diversity to tech industry

Major tech conference under fire for naming 21 men keynote speakers — and just 1 woman