Instructor Notes
Below are two activities that will help cultivate an ability to think about data critically. It will be important to alert students to the fact that the material included in the FBI hate crimes dataset addresses deeply disturbing forms of identity-based violence. Instructors should be alert to the possibility that some students may have had experience with hate crimes and might be impacted by the exercise.
This discussion exercise takes the FBI’s 2021 Hate Crimes Statistics as a case study of the power and limits of data analysis. The discussion activities below build on other resources in the Data Advocacy Toolkit, and thus provide an opportunity to reinforce the lessons of critical data analysis.
The full package of FBI hate crimes data for 2021 can be downloaded from this website (scroll down to the Hate Crimes header). The downloaded package will include a significant amount of supporting documentation and data tables. The discussion exercise below refers to table 1, “Incidents, Offenses, Victims, and Known Offenders by Bias Motivation, 2021”)
Activity one assumes that students have read Douglas Shafer and Zhiyi Zhang, “Basic Definitions and Concepts” and “Overview” from Introductory Statistics (also included in the Data Advocacy Toolkit).
Activity two requires students to read two items: Catherine D’Ignazio and Lauren Klein, “What Gets Counted Counts,” chapter four of Data Feminism (also included in the Data Advocacy Toolkit). Ken Schwenke, “Why America Fails at Gathering Hate Crime Statistics,” Pro Publica, December 4, 2017.
Examining a Dataset (25 minutes)
This exercise provides an occasion to see in action the key concepts developed by Shafer and Zhou in the first chapter of their Introductory Statistics textbook. Ask students to read over and analyze carefully, noticing as many things as they can, about the summary table of the 2021 FBI hate crimes dataset for 2021, table 1, “Incidents, Offenses, Victims, and Known Offenders by Bias Motivation, 2021”.
Here are a few key questions to ask students to investigate:
- What information would be useful to have about this data’s biography, that is, about how the dataset was assembled, by whom the data were collected, and for what purposes?
- Is the dataset a sample of the entire population or does it represent the entire population?
- Why is it important to know this information? Why does it matter whether a dataset captures a population or depicts a sample?
- What are the quantitative variables that make up the dataset?
- What are the qualitative variables that make up the dataset?
- What kinds of questions does this dataset allow you to ask?
- What kinds of questions does the dataset not address?
Discussion: What Counts as Hate Crime? (25 minutes)
This discussion exercise builds on Catherine D’Ignazio and Lauren Klein’s chapter from Data Feminism, “What Get’s Counted Counts.” D’Ignazio and Klein examine a number of ways that the categories used to organize data are always partial and embed assumptions and bias about what the world looks like. Also implicit in their argument is that the way data is collected and counted also shapes our perception of reality.
Question 1: Bringing these insights to the FBI hate crimes dataset, invite students to think about the categories utilized in that dataset. What kinds of questions about these categories might D’Ignazio and Klein ask? How might the categories used in the dataset bias inquiry? How might the procedures for collecting data bias the inquiry?
Question 2: The FBI hate crimes dataset has been criticized for its methods and accuracy. The article by Ken Schwenke (“Why America Fails at Gathering Hate Crimes Statistics”) describes the limits of data gathered through the FBI’s voluntary reporting system. The 2021 data was especially concerning and widely criticized as incomplete. As Cynthia Miller-Idriss reported in the online journal Lawfare in December of 2022:
“The FBI released its 2021 hate crime report this week amid widespread criticism that its analysis rests on incomplete data and was hindered by a significant drop in local agency reporting. At first glance, the report suggests that there has been a decline in hate crimes, as reported crimes dropped to 7,262 crimes from last year’s 12-year high of 8,263. But nearly 40 percent of agencies across the country failed to report any data at all for 2021—only 11,883 of 18,812 agencies reported. In 2020, FBI hate crime statistics for the nation included data received from 15,138 of 18,625 agencies. The 2021 data reflects a reduction of about 20 percent from the previous year’s submissions.” (“The FBI’s 2021 Hate Crime Data Is Worse Than Meaningless,” Lawless, Friday, December 16, 2022) The FBI subsequently updated the statistics for 2021 in a supplemental report, and the new data indicates that there had been a significant increase in hate crimes, rather than a significant drop.
What concerns does this episode raise? How does the method of gathering, classifying and reporting hate crimes impact our understanding of the extent of hate crime in the United States? What kinds of bias might enter into the voluntary act of reporting such crimes? How do police officers, or for that matter, victims, decide whether or not to report a hate crime? How do these limits impact the usefulness of the data?