NYC creates US' 1st law to govern AI job hiring tools

New York City has enacted the first law in the United States to regulate the use of artificial intelligence and automated tools in the job hiring process, as civil liberties groups warn that nothing can stop computerized devices from bias against marginalized groups.

Local Law 144 will take effect on July 5. It was first passed by the New York City Council in 2021 and was delayed after the Department of Consumer and Worker Protection, the body in charge of enforcing it, worked on how it should be used.

The DCWP asked for public comments before its final draft, and several companies replied. They all aimed to define what automated employment decision tools, or AEDTs, should be included in the law and how a yearly audit by employers can weed out bias.

Sona Rai, director of communications for nonprofit research institute Data & Society, told China Daily its team had raised several points in October about how AEDTs are prone to bias.

Important implications

"Local Law 144 is one of the first laws in the world to mandate an independent audit of any algorithmic systems for bias, and therefore this law and rule-making process has important implications beyond the jurisdiction of the DCWP," Data& Society wrote in an October letter to the department.

"As is well-documented, the use of machine learning to train these computational systems is prone to bias against vulnerable and historically disadvantaged groups of people."

Many employers are increasingly using AEDTs in candidate searches as AI-based devices can quickly sift through hundreds of resumes to find the best candidate with little human interaction.

Local Law 144 stipulates that employers must carry out an audit — by an independent body — every year to see if their AEDT is prone to bias against candidates based on gender, sex or race, but not disability.

Local Law 144 will target companies that use AEDTs solely or in most of their decisions over that of a human. This makes the law hard to follow, as some critics said if it is not deemed to be used enough, or used in conjunction with other tools, an employer may believe it does not apply to them.

Civil liberties groups suggest that despite best efforts to implement a bias audit, the automated job search tools will likely still be prone to discriminate.

"At their core, these systems learn to replicate the past decisions and behaviors recorded in their training data — these systems predict how we would have acted in similar contexts, leaving little room for adjusting how we should have acted," Data& Society said.

Daniel Schwarz, senior privacy and technology strategist for the New York Civil Liberties Union, also warned of the dangers of AEDTs in testimony to the DCWP in October.