Study finds racial bias in AI hiring

A new study has found that artificial intelligence tools used to screen job applicants in the United States exhibit significant racial bias against Black and Asian candidates, raising concerns about algorithmic discrimination at a moment when the US labor market is already under strain.

Researchers from Stanford University, Chapman University and Northeastern University tracked 3.4 million people who submitted 4 million job applications to 1,700 job postings across 150 employers and 11 industry sectors. Every application in the dataset was assessed by an AI hiring tool built by a single third-party vendor.

The authors describe their paper as the first to offer large-scale empirical evidence of racial disparities in high-stakes hiring decisions made by algorithmic systems.

"Our new paper offers a rare look inside the 'black box' of algorithmic hiring, showing that these tools increase racial bias and shut the same people out of jobs everywhere they apply," they wrote in an article published on the website of the Stanford Institute for Human-Centered Artificial Intelligence.

The researchers explain the hiring AI pipeline as follows: job seekers submit applications, which are routed to a hiring AI vendor; the vendor's machine learning models generate predictions; and the resulting labels of "recommend" or "do not recommend" are sent back to the employer to guide decisions.

The study, titled Algorithmic Monocultures in Hiring, found that 26 percent of Black applicants and 15 percent of Asian applicants submitted applications to positions where the AI system discriminated against their racial group, as measured by US employment discrimination standards.

If Black and Asian candidates had been recommended at the same rate as the most-favored group, typically white applicants, 40,000 more of their applications would have advanced to the next round, according to the study.

Asian applicants experienced the largest shortfall. If Asian candidates had been selected at the same rate as the most-selected racial group for each position, 29,000 additional applications from Asian job seekers would have been forwarded to employers, the study found.

The researchers stressed that the method used to measure adverse impact matters, because broad averages can mask discrimination that is occurring at the level of individual job postings. For example, if an AI tool that frequently recommends Black applicants for warehouse positions but rarely recommends them for finance roles, it would seem like there is no discrimination on average, but it still produces discriminatory outcomes position by position.

The US labor market has seen hiring slow, while AI tools have made it easier than ever for applicants to submit large numbers of applications. The combination has driven the volume of applications for entry-level positions to nearly three times the levels seen in 2022, according to the paper.

The concentration of the AI hiring industry has caused another problem. They found that more than 60 percent of Fortune 100 companies rely on algorithms built by HireVue. When a significant share of employers across an industry use the same vendor's system, the researchers warned, a single algorithmic bias can ripple across the entire job market, creating what they call systemic rejection.

Among applicants who submitted four applications to positions screened by the same algorithmic vendor, 10 percent were rejected from every single one. As one vendor comes to dominate screening for an industry, the researchers said, candidates become increasingly likely to be shut out entirely rather than simply passed over for a particular role.

The study does not identify what caused the racial biases observed in the hiring algorithms. The researchers acknowledged that much of the technology's impact remains poorly understood by the public.

The researchers are calling on regulators and auditors to measure adverse impact at the individual job-posting level rather than in aggregate, and they are urging agencies to strengthen monitoring mechanisms to prevent systemic rejection in algorithmic hiring.

liazhu@chinadailyusa.com