Artificial intelligence and hiring bias raise real challenges

Few developments have generated as much simultaneous excitement and concern as the growing use of artificial intelligence in the American workplace and the introduction of various forms of Diversity, Equity, and Inclusion (DEI) programs. AI tools can perform many tasks with greater efficiency than human workers, at a much lower cost. For these reasons, many fear that AI could eventually replace and displace workers at all levels. Companies are already using AI to screen job applicants, saving recruiters considerable time and effort.

But efficiency, if care is not taken, can come at the expense of equal opportunity. When AI is used to review job applicants or candidates for promotion, it can tilt the scale on the side of qualities or descriptors that favor certain categories of workers. It can screen out promising candidates who have no idea that computer code denied them a job opportunity. Without intending to, companies that use AI tools for recruiting may be discriminating against entire groups of applicants. While intended for “good,” some DEI programs have created backlash, and combined with careless use of AI, can create exactly the kinds of problems they are intended to solve.

Algorithmic bias

That AI is fraught with peril should not be a surprise. After all, AI programs use algorithms created by human beings. And just like the humans who create them, algorithms may be biased and fallible. In February, Google issued an apology after discovering that a model used by its Gemini AI program generated non-white images of historical figures, including US Founding Fathers, a female Pope, and Nazi-era German soldiers, as well as Google’s own founders.

Google’s apology, asserting that the AI had been trained to ensure that a range of people were included in its results, was disingenuous, to say the least. Based on how the AI was programmed, it could not show “white” results and in fact showed only historically false representations. By seeking to counter the tendency of massive test datasets to amplify existing biases, Google essentially created brand new biases.

If they are intended to mimic human decision-making, AI tools will thus show the same biases and preconceptions as their human creators. When reviewing job applications, they may – unconsciously – favor certain applicants while dismissing others. An applicant may never know that an algorithm was the reason for her rejection. The Google misadventure reveals the danger in trying to “counter discrimination with discrimination” by, for example, creating an algorithm intentionally created to advantage certain groups that are perceived as disadvantaged.

It is a problem that has generated consternation among lawmakers. In October 2022, the White House Office of Science and Technology Policy, concerned about the impact of AI on the hiring of individuals with disabilities, released its Blueprint for an AI Bill of Rights. The document was designed to foster more equitable and inclusive digital hiring of workers with disabilities, as well as other underserved communities. An Equal Employment Opportunity Commission hearing in January 2023 focused on preventing unlawful bias in the use of AI; an EEOC website on the “Artificial Intelligence and Algorithmic Fairness Initiative” reviewed fairness in the AI-utilized workplace. In April 2023, the Department of Labor hosted an online “think tank” to examine the use of AI tools in hiring.

AI legislation

Congress has twice attempted to address the issue of AI use in the workplace. The most recent bills, S. 2892/H.R. 5628, also known as the Algorithmic Accountability Act of 2023, would have required the Federal Trade Commission to establish rules for companies to conduct assessments of their AI systems and identify go/no-go points within those systems and their real-world consequences. The bills stalled in committee.

The first – and only – major legislation to address AI use by employers is New York City’s Local Law 144, governing employers’ use of “automated employment decision tools” or AEDTs. AEDTs are defined in the law as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence” that is used to “substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.”

Companies in New York City are barred from using AI to screen candidates for hiring or promotion unless, within a year of using the AEDT, they conduct an audit, through an independent auditor, to determine whether there is bias in the tool and post the audit results on their websites. The audit should evaluate an AEDT’s potential disparate impact on a group of job applicants or employees based on demographic categories that mirror EEO-1 reportable data such as sex, race, and ethnicity. Other states continue to consider such legislation. While California’s AB 331 was not passed by the Legislature, employers and employees continue to watch Sacramento and other state capitals for new developments.

New DEI considerations

Rules seeking to increase workplace diversity, equity, and opportunity face new challenges in the wake of the Supreme Court’s affirmative action decision in Students for Fair Admissions, Inc. v President and Fellows of Harvard College, as well as a federal district court’s March 6, 2024, decision in Nuziard v. Minority Bus. Dev. Agency finding it unconstitutional for the U.S. Department of Commerce’s Minority Business Development Agency to exclude White-owned businesses from its loan program. AI algorithms cannot be written to purposely exclude job applicants based on race, age, gender, disability, and other protected categories. But can such algorithms be written in a way that favors minorities or other underrepresented categories?

For companies seeking to broaden their workplace demographics, an AI tool could easily be programmed to help identify these candidates, but it should not be used to select some and exclude others. Just as the Google Gemini tool went far beyond injecting diversity into historical figures and excluded vast segments of the population from its results, so too might AI be used to highlight or amplify qualities or characteristics that signify diversity but end up screening out or excluding others. Even when companies are not using numerical quotas or other unacceptable criteria, they must still ensure that their efforts to diversify applicant pools do not lead to claims of discrimination.

Is it legal to build an AI algorithm in a way that promotes diversity? Given the spate of recent lawsuits challenging DEI programs at a range of companies, including law firms, it may be just a matter of time before AI tools become the focus of such actions. At that point, the principal question may be whether a decision to hire or reject an applicant was dictated by an algorithm. Did human decision makers play a role in the company’s decision? And was the final decision fair?

Conclusion

The advent of AI has rewritten the rule books for just about every part of society. Combined with DEI initiatives, AI holds the possibility of expanding opportunities for all. But we are just beginning to see how AI impacts the workplace. AI has the power to change the way we work, and the way we make decisions about work, including establishing DEI-related goals. Depending on how carefully they are crafted and implemented, both AI and DEI programs can be significantly beneficial for businesses and their workers, or quite problematic.