What is Amazon's responsibility over its facial recognition tech?

US Rep. Jimmy Gomez was not terribly shocked to learn he was among 28 members of Congress matched to...

Posted: Jul 27, 2018 9:30 AM
Updated: Jul 27, 2018 9:31 AM

US Rep. Jimmy Gomez was not terribly shocked to learn he was among 28 members of Congress matched to criminal mugshots in an experiment by the ACLU of Northern California. In fact, the California Democrat had a pretty good idea what many of the people on that list shared in common.

"I told my staff that I wouldn't be surprised if it was mostly people of color or minorities," he told CNNMoney.

He was right.

On Thursday, the ACLU released the results of its test of Amazon's controversial Rekognition software, which uses artificial intelligence to, among other thing, recognize people.

After building a database of 25,000 publicly available arrest photos and comparing it to all 535 members of Congress, the ACLU found that Rekognition, when left to its default settings, identified lawmakers like Luis Guitierrez of Illinois, John Lewis of Georgia, and Norma Torres of California, as criminals.

Of the 28 lawmakers Rekognition misidentified, about 40% were people of color, a result that led the ACLU to call for a government moratorium on the use of facial recognition software -- a call that gained traction Thursday among lawmakers.

"This is a big deal," Gomez told CNN. "The bias that exists will be digitized and used against people who already have a lot of obstacles and struggles."

Although inaccuracy in facial recognition among women and people of color is a known issue, experts say it underscores the need for broader conversations about the ethics of such technology and the responsibility of companies creating it to ensure it is used fairly.

To that end, Gomez and Lewis sent Amazon CEO Jeff Bezos a letter requesting "an immediate meeting to discuss how to address the defects of this technology in order to prevent inaccurate outcomes."

In a separate letter, senators Edward Markey, Mark DeSaulnier, and Luis Gutierrez asked Bezos to provide details on any internal accuracy or bias assessments Amazon has conducted on Rekognition. They also want to know which law enforcement or intelligence agencies use Rekognition, and whether Amazon audits their use of it.

Amazon pushed back against the ACLU's results by arguing, in effect, that it didn't use the tool properly.

Related: Amazon asked to stop selling facial recognition tech

"We think that the results could probably be improved by following best practices around setting the confidence thresholds (this is the percentage likelihood that Rekognition found a match) used in the test," an Amazon spokesperson said in a statement. "While 80% confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases, it wouldn't be appropriate for identifying individuals with a reasonable level of certainty. When using facial recognition for law enforcement activities, we guide customers to set a threshold of at least 95% or higher."

Suresh Venkatasubramanian, a computer scientist at the University of Utah, said that argument lets Amazon off the hook too easily. The company has an obligation to make sure people know how to use its tools properly, he told CNNMoney.

"You had the choice to not put the tech out if you felt it was sensitive," he said. "We don't let people randomly prescribe medicines to themselves."

Related: Microsoft wants regulation of facial recognition tech

Tech companies are coming around to this view. In a blog post published earlier this month, Microsoft President Brad Smith suggested regulating facial recognition technology given its "broad societal ramifications and potential for abuse."

Bezos has not addressed the issue publicly despite recent calls from shareholders and civil rights groups to stop selling the technology to the government. In the statement to CNNMoney, Amazon called Rekognition "a driver for good in the world."

Perhaps, or perhaps not, said Woodrow Hartzog, who teaches law and computer science at Northeastern University. "The idea that this is simply neutral technology that can be used for good or evil and Amazon shouldn't be responsible, I think is purely wrong," he said.

"It's not unreasonable to say if you build a product that is capable of harm than you should be responsible for the design choices you make for enabling the harm," he said, "and when you release it out into the world, you're doing so in a safe and sustainable way."

Correction: A previous version of this story misstated the number of members of Congress.

Indiana Coronavirus Cases

Data is updated nightly.

Confirmed Cases: 51079

Reported Deaths: 2756
CountyConfirmedDeaths
Marion12019693
Lake5588248
Elkhart353959
Allen2939134
St. Joseph210669
Hamilton1691101
Cass16449
Hendricks1454100
Johnson1340118
Porter82638
Tippecanoe7709
Vanderburgh7276
Clark69544
Madison67464
LaPorte61628
Howard59858
Bartholomew59745
Kosciusko5754
Marshall5449
Noble51328
LaGrange4849
Boone48244
Jackson4783
Delaware47152
Hancock46736
Shelby45425
Floyd40644
Morgan34231
Monroe34028
Grant31826
Dubois3046
Henry30018
Montgomery29720
Clinton2903
White27410
Dearborn25823
Decatur25632
Lawrence25225
Vigo2528
Warrick25029
Harrison21722
Greene19432
Miami1932
Jennings17912
Putnam1738
DeKalb1694
Scott1649
Wayne1546
Daviess15017
Perry14710
Orange13723
Steuben1362
Jasper1352
Ripley1307
Franklin1278
Gibson1202
Wabash1162
Carroll1142
Fayette1067
Whitley1066
Starke1043
Newton10010
Huntington942
Jefferson862
Wells821
Randolph794
Fulton731
Knox710
Jay700
Washington681
Pulaski661
Clay645
Rush613
Posey570
Spencer541
Owen521
Benton510
Sullivan501
Adams491
Brown431
Blackford402
Fountain352
Crawford330
Switzerland320
Tipton321
Parke270
Martin260
Ohio230
Vermillion200
Warren151
Union140
Pike110
Unassigned0193

Ohio Coronavirus Cases

Data is updated nightly.

Confirmed Cases: 64214

Reported Deaths: 3036
CountyConfirmedDeaths
Franklin11724445
Cuyahoga8979393
Hamilton6781207
Lucas2952305
Marion274539
Montgomery244035
Summit2327209
Pickaway222241
Mahoning1928239
Butler182147
Columbiana137860
Stark1214114
Lorain112069
Trumbull104578
Warren97725
Clark80010
Delaware69815
Fairfield66717
Tuscarawas60410
Lake58923
Medina58232
Belmont56922
Licking56812
Miami50631
Portage49259
Wood48851
Clermont4737
Ashtabula44744
Geauga42543
Wayne37253
Richland3715
Allen35641
Greene3439
Mercer29910
Erie27122
Holmes2595
Darke25626
Huron2402
Madison2169
Ottawa17324
Sandusky16015
Washington14620
Ross1443
Coshocton1423
Athens1391
Crawford1385
Putnam13715
Hardin12312
Morrow1201
Auglaize1094
Jefferson1092
Muskingum1001
Union931
Preble901
Monroe8917
Hancock861
Lawrence830
Guernsey823
Clinton811
Hocking809
Williams762
Shelby744
Logan711
Ashland672
Carroll673
Fulton670
Scioto670
Wyandot635
Brown611
Fayette550
Defiance533
Knox531
Champaign511
Highland501
Van Wert471
Perry441
Seneca412
Henry330
Paulding300
Jackson280
Pike280
Adams261
Vinton232
Gallia201
Noble140
Harrison131
Meigs130
Morgan110
Unassigned00
Fort Wayne
Clear
72° wxIcon
Hi: 81° Lo: 60°
Feels Like: 72°
Angola
Scattered Clouds
73° wxIcon
Hi: 80° Lo: 58°
Feels Like: 73°
Huntington
Broken Clouds
75° wxIcon
Hi: 79° Lo: 58°
Feels Like: 75°
Decatur
Broken Clouds
73° wxIcon
Hi: 80° Lo: 60°
Feels Like: 73°
Van Wert
Broken Clouds
73° wxIcon
Hi: 81° Lo: 61°
Feels Like: 73°
Sunday starts wet, ends dry
WFFT Radar
WFFT Temperatures
WFFT National

Community Events