Algorithmic Effects

A drawing of various colored circles of all sizes and hues connected by lines of other colors over a white background.

Image copyright Adobe Stock

Critiquing algorithms

How do algorithms shape the information we have access to? What ethical considerations are given when designing search engines, since they are overwhelmingly managed by private, for-profit enterprises?

These questions are at the centre of many research projects and debates in media studies, communications, political science, and related fields. Although algorithms are most neutrally defined as a step-by-step procedure for solving a problem, in the context of digital information, algorithms seem to have taken on deeply political roles. They shape the knowledge we can access most easily through online searches. They also sometimes take decisions for public institutions as to who can access resources and aid, or which neighborhoods get policed more heavily, and so on (as discussed by scholars including Safiya Umoja Noble, Virginia Eubanks, Cathy O’Neil, and Ruha Benjamin).

Black boxes

There is a bi-directional relationship between algorithms and the people who interact with them. Algorithms, in the context of predictive technologies, are trained on data inputs. While some algorithms may get a fixed set of inputs when they are first created, many are designed to continue receiving new inputs (e.g., people’s personal information, searches, online behaviors), which then further influence the algorithms. Often algorithms are black boxes, as we have limited knowledge of their inputs and inner workings.

Paragraph adapted from the Unpacking Algorithmic Bias workshop by Andrea Baer, CC BY-NC-SA 4.0 International License.

Diagram of a black box labelled algorithm finds patterns. An arrow labelled information points into the box, and an arrow pointing to decisions made leads out of the box.

Algorithms find patterns in data and use them to make decisions or take actions. Algorithms are called black boxes because the process through which they arrive at a decision cannot be explained.

Some computer algorithms are black boxes to users because of a commercial strategy to protect the intellectual property of the researchers and companies marketing software products. In other cases, the application of machine learning, which gives computers rules to write their own software, means that some algorithms are black boxes even to their makers because those rules have resulted in systems that are difficult for humans to reverse engineer.

The developers behind these algorithms often create systems that deal with large amounts of information in unsupervised ways that are difficult to decode for humans. In that sense, the patterns that machine learning systems detect and the predictions they offer are not always easy to justify, especially to users.

In the case of search engines, the political power of the underlying algorithms comes from two sources: first, the fact that private companies do not have to tell other people how their search results are assembled, and second, the fact that these companies are not rewarded for offering the most educational, accurate results, but rather by selling advertisements delivered in the process of offering search results to users.

Activity

In this activity, we'll consider how the black boxes of predictive and generative algorithms deserve scrutiny, whether they’re in a search engine, in a traffic light controller, or in a police patrol route generator.