The 20-Year-Old Point-of-View: Weapons of Math Destruction by Cathy O’Neil
By Whom: Cathy O’Neil, PhD Mathematics, Harvard, Columnist for Bloomberg View.
For Whom: Data Scientists, Computer Scientists, Mathematicians, Political Scientists, Sociologists, and Anyone With a Social Media Account
Weapons of Math Destruction is the first time I’ve ever read a book that I’ve been assigned a excerpt from in class. In Intellectual Foundations of Informatics at the University of Washington, the introduction to O’Neil’s book is found at the beginning of the chapters on Data Science. Therefore, when I was asked to give a lecture on Artificial Intelligence and Machine Learning, this book immediately came to mind as a starting point for my research.
O’Neil’s book can, in the very most abstract of terms, be described as a warning. Throughout the book, she combines her experience with both Data Science and Journalism to recount several fascinating stories of how our models of the world cause unjust harm to individuals. Among these are examples of teachers being fired based on seemingly random rating systems, denying low-income and disadvantaged minorities access to health insurance based on racial and geographic data, and even swaying of political support based on Facebook timelines. Every story in the book is a relatable, seemingly dystopian, but hauntingly familiar painting of a world ran by numbers and machines with little regard for humanity.
Essentially, my greatest fear.
O’Neil makes the argument that these WMDs (Weapons of Math Destruction) are often the byproduct of greed, borne of the free market, and require federal regulation similar to acts like HIPAA and FCRA to protect individuals from harm. I disagree that such regulations will be sufficient to prevent use of WMDs for nefarious purposes — HIPAA and FCRA protect certain kinds of information that are largely secluded and bound to one area of specialty, whereas WMDs and Big Data are things that quite literally try to wiggle their way into every facet of our daily lives. If all a teacher has to do is sign a waiver allowing a tech startup to track their teaching habits and rank them nationally, the damage will still be done on the entire system. I believe we should take a stance higher on the chain: educating engineers, designers, and leaders on the potential harm in confiding too closely with models, and becoming more comfortable with uncertainty. After all, the world will never be modeled perfectly, so let’s not fool ourselves into thinking that it can be.
5/5. A book that may become reality.
