AI-powered hiring systems are increasingly being used by employers to streamline recruitment, but a new study led by researchers at Stanford University reveals that these tools can perpetuate racial bias on a massive scale. The research examined the Pymetrics platform, which uses online games to screen candidates, analyzing a dataset from December 2018 to December 2022 that covered 4 million applications across 156 employers. The findings show that about one in 10 positions had an adverse impact for Black applicants, and one in 20 for Asian applicants. While these numbers represent a minority of job postings, the high volume of those roles meant that 26 percent of Black candidates were affected, indicating that bias is concentrated in positions that attract large numbers of applicants or specifically more candidates of color.
The term "adverse impact" is used by the U.S. federal government to denote when a race, sex, or ethnic group is selected at a rate less than four-fifths (80 percent) of the rate of the most-selected group. This legal standard is often applied in employment discrimination cases. The Stanford study found that the Pymetrics platform generated adverse impact patterns that mirror human biases, suggesting that the AI models are learning from historical data that reflects systemic inequalities in hiring. The researchers note that many employers using Pymetrics rely on the same algorithmic setup—42 models were shared across the 156 employers—so a candidate rejected by one company using the algorithm is more likely to be rejected by another, creating a cascading effect of exclusion.
This study comes amid an explosion in AI usage for hiring. According to the Society for Human Resource Management (SHRM), AI adoption in HR departments increased from 26 percent of organizations in 2024 to 43 percent in 2025. The rapid proliferation of these tools raises concerns about fairness, especially since many job seekers are subjected to lengthy automated testing processes without interacting with a human recruiter until late stages. The Stanford research is not an isolated case. A study by the University of Illinois and Ahmedabad University found that AI hiring recommendations favored men over women, with women often steered toward lower-wage positions. Workday, one of the largest HR software providers, is currently facing a lawsuit over claims that its AI software unfairly screens out candidates, further fueling debates about the reliability of these systems.
The broader context includes regulatory efforts to rein in AI bias. The European Union's AI Act identifies hiring as a high-risk area, requiring AI systems to have robust safeguards, technical documentation, transparency, and human oversight. This law targets both the developers and deployers of AI hiring tools. In the United States, New York, Colorado, and Illinois have enacted laws that force employers to audit their AI systems or face penalties. California and other states are integrating AI hiring practices into existing anti-discrimination laws. The fundamental question is no longer whether AI can speed up hiring, but whether employers can prove these systems are fair, explainable, and compliant before they filter out thousands of candidates based on biased criteria.
The Stanford study's focus on Pymetrics is notable because the platform uses gamified assessments—tests that measure cognitive and emotional traits rather than traditional resume screening. This approach was intended to reduce bias, but the data shows that the algorithms still learn patterns that correlate with race, possibly due to differences in how demographic groups perform on these games based on cultural exposure or socioeconomic factors. The study's authors emphasize that bias often emerges not from malicious intent but from the data used to train the models, which may reflect historic hiring disparities. For example, if a company historically hired fewer Black employees for a certain role, the AI will learn to deprioritize those candidates, even if the game scores are race-neutral.
To understand the scale of the problem, consider the dataset: 4 million applications over four years. This volume means that even small biases can affect hundreds of thousands of people. The researchers call for independent audits of hiring algorithms and greater transparency from vendors like Pymetrics. They also urge employers to customize models rather than relying on shared templates, as that can reduce the compounding effect of bias across multiple companies. In addition to the legal risks, there are reputational and ethical risks. Candidates who feel unfairly treated may share their experiences online, and class-action lawsuits are a growing threat.
The Stanford study adds to a growing body of evidence that AI is not the panacea it was once thought to be for eliminating human bias. Instead, it can amplify it if not carefully designed, monitored, and regulated. The technology is still evolving, and the hope is that with proper safeguards—like diverse training data, regular testing for adverse impact, and human oversight—AI can eventually become a tool for fairer hiring. But for now, the data shows that millions of applicants are being screened out by systems that replicate the very biases they were meant to overcome. As the regulatory landscape tightens, companies will need to invest in more rigorous auditing and transparency to avoid legal action and to build trust with a diverse workforce.
Source: eWeek News