Originally Posted by rowan

It's quite common for AI models to show racism because those models are usually created with data supplied by (or generated by) humans, which may also show a bias.

For example, if Police patrol a black neighborhood more aggressively, they'll make more arrests there, even if the crime level is the same as everywhere else. That skews the data. The AI model then concludes that people living in the black neighborhood are more likely to offend.