I’m always amazed how often the “surprise” that digital technology is in fact not unbiased and robotic in nature keeps getting written about. Algorithms that optimize for click through rate for ads and search relevancy will inevitably be a reflection of ourselves and all of the shitty things various people believe.
>> What we know about Google’s responses to racial stereotyping in its products is that it typically denies responsibility or intent to harm, but then it is able to “tweak” or “fix” these aberrations or “glitches” in its systems.
Only Google knows the fate of the code base that has workarounds for all of these edge cases. I think the author is wrong that all of these edge cases could’ve been caught beforehand. Some sure, but I think others would always slip through.
It shouldn’t be surprising. Engineers routinely talk about how it would be great if we removed the human element and replaced it with algorithmic decision making. But see how sentencing algorithms give longer prison terms to black people. Yes, the technical explanation is that it’s just reflecting society’s racism. But the point is that, contrary to the bill of sale, using algorithms doesn’t actually remove the human element. Indeed, there is a good argument (as the author of this piece makes elsewhere) that it ossifies the human biases that exist at one moment in time.
> I’m always amazed how often the “surprise” that digital technology is in fact not unbiased and robotic in nature keeps getting written about. Algorithms that optimize for click through rate for ads and search relevancy will inevitably be a reflection of ourselves and all of the shitty things various people believe.
What you just observed and described is that technology _is_ unbiased and robotic. The bias is in the humans. You can't remove it with a software bandage; the software is working as intended.
Indeed so. I'm always reminded of Charles Babbage's self-reported interactions on the subject of his Difference Engine:
'On two occasions I have been asked, — "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" In one case a member of the Upper, and in the other a member of the Lower, House put this question. I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.'
But I think the problem is more subtle than you give it credit for. The software may be working as written, but 'intention' rarely maps cleanly to execution. The behavior of a complex system is a multidimensional shape; stretching it one way or another along some axis according to your intention for its behavior can easily have unanticipated, and often unobservable effects along other axes that you were not minding. So it may be an intention that the system optimize for click-through rates for ads, while at the same time not consciously intended to be biased against, for example, race, but also not deliberately crafted to avoid this behavior. This may functionally be the same thing as intending it to be racist, but it has different implications for who's guilty and how things get fixed.
The quip by Babbage is clever, but Babbage was too early and didn't know what we know. He could not anticipate the digital abstraction, error-correcting codes, compressed sensing, and machine learning.
Modern computing systems have components that produce output that's "better" than their input, at every level. Because we have information theory now, we know it is possible for a machine to produce good output even when some of its input is bad.
Google, incidentally, has aimed to do that for its entire existence. From the first time the Google search engine was deployed, its job was to take in the messy Web, full of misdirection and spam, and show you the useful pages you were looking for. And we can keep hoping for Google to keep doing this as the threats to information become more insidious.
Though some people might argue that the biases software reproduces will in turn reinforce biases held by the users at large. Conversely, you may reduce human bias if you algorithmically reduce the bias in the data (which is possible). Not saying whether Google should, that's a very delicate debate, but it's thinkable.
I wouldn't want to be in the place of the person at Google trying to make guidelines for that, but it's definitely doable. For instance, you could probably easily argue that the phrase "three black teenagers" (an example from the article) is pretty neutral in itself and definitely shouldn't be interpreted as a call to produce arrest pictures. In general the term "three [ethnicity] teenagers" could probably be normalized with regards to the setting the pictures show, for instance.
For example, a search algorithm with an override that removes all links of sexual nature for certain hardcoded queries (say, "black girls") could be an example of a biased technology.
If I'd have to make a definition, then it would probably be a technology that somehow encodes an assumption that doesn't match reality. In this context, saying that "black girls" is an innocent search term and users searching for it don't expect to get sexual results is such an assumption - it would be polite, politically correct and possibly socially desirable; but it seems likely this is simply not true in the reality we live in.
I had a person on my team preparing a presentation and for whatever (reasonable) reason was looking for a stock image of a rabbit in winter with frozen precipitation around.
Just as our (very straight laced) CIO walked up behind, employee hit image search on "snow bunny". Hilarity ensued...
>> What we know about Google’s responses to racial stereotyping in its products is that it typically denies responsibility or intent to harm, but then it is able to “tweak” or “fix” these aberrations or “glitches” in its systems.
Only Google knows the fate of the code base that has workarounds for all of these edge cases. I think the author is wrong that all of these edge cases could’ve been caught beforehand. Some sure, but I think others would always slip through.