Online retail giant Amazon was reportedly forced to abandon a secret project to automate its recruitment process after it was discovered that the artificial intelligence powering its learning algorithm was systematically devaluing the job applications of women. Speaking under condition of anonymity, five sources familiar with the project told Reuters that in 2014 Amazon had assembled a team to build computer programs that would automatically review lists of resumes and output the best candidates from the bunch. Unfortunately, the learning algorithm was trained on the patterns of resumes submitted to the company over the past decade — which, in a heavily male-dominated sector, almost all came from men.
The resulting program learned to penalize resumes that contained the word ‘women’ or the names of at least two-all women’s colleges, and to favor candidates who used words more typically found on male engineers’ resumes. Developers attempted to modify the computer model to make it neutral to those particular terms, but it was impossible to tell if the algorithm wouldn’t find other ways to discriminate against women candidates. Other problems within the model that led to unqualified candidates being repeatedly recommended for jobs forced executives to scrap the recruitment automation effort last year, sources said. Prior to the end of the project, Amazon recruiters allegedly did examine the results generated by the tool when evaluating new hires, but did not make final hiring decisions solely on that basis.
Amazon officials have refused to comment on the problems with the ill-fated AI. According to Reuters, the company did not deny that recruiters had looked at the computer models, but did say the algorithm “was never used by Amazon recruiters to evaluate candidates.”
Read the full story at Reuters.