FTC Says Racist Algorithms Could Get You in a Lot of Trouble

{snip}The FTC has declared that it is serious about racist algorithms, and it will hold businesses legally accountable for using them. In a friendly-reminder type announcement today, it said that businesses selling and/or using racist algorithms could feel the full force of their legal might.

“Fortunately, while the sophisticated technology may be new, the FTC’s attention to automated decision making is not,” FTC staff attorney Elisa Jillson wrote in a statement on Tuesday, adding that the agency “has decades of experience” enforcing laws that racist algorithms violate. They write that selling and/or using racially biased algorithms could qualify as unfair or deceptive practices under the FTC Act. They also remind businesses that racial discrimination (by algorithm or human) could violate the Fair Credit Reporting Act and the Equal Credit Opportunity Act.

{snip} Racist algorithms have been shown to disproportionately deny Black people recommendations for specialized healthcare programs. They have priced out higher interest rates on mortgages for Black and Latinx people than whites with the same credit scores. {snip} They have encouraged police to target locations and arrest records which perpetuate further disproportionate arrests in Black communities. {snip}

Government use of racist algorithms makes the “selling” part especially important. The FTC can’t try the cops, but it might be able to go after a company that misrepresented its tool as race-neutral.

Given the endless churn of stories about the racist results of facial recognition, it could seem that the FTC is equipping itself to practically annihilate the technology. In an email to Gizmodo, an FTC spokesperson said that a seller could be guilty of “deceptive” practices if it “misleads consumers (whether they are businesses or individuals) about (for example) what an algorithm can do, the data it is built from, or the results it can deliver, the FTC may challenge that as a deceptive practice.”

{snip}