"A former Wall Street quantitative analyst sounds an alarm on mathematical modeling, a pervasive new force in society that threatens to undermine democracy and widen inequality, "--NoveList.
Longlisted for the National Book Award New York Times Bestseller A former Wall Street quant sounds an alarm on the mathematical models that pervade modern life — and threaten to rip apart our social fabric We live in the age of the algorithm. Increasingly, the decisions that affect our lives—where we go to school, whether we get a car loan, how much we pay for health insurance—are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: Everyone is judged according to the same rules, and bias is eliminated. But as Cathy O’Neil reveals in this urgent and necessary book, the opposite is true. The models being used today are opaque, unregulated, and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination: If a poor student can’t get a loan because a lending model deems him too risky (by virtue of his zip code), he’s then cut off from the kind of education that could pull him out of poverty, and a vicious spiral ensues. Models are propping up the lucky and punishing the downtrodden, creating a “toxic cocktail for democracy.” Welcome to the dark side of Big Data. Tracing the arc of a person’s life, O’Neil exposes the black box models that shape our future, both as individuals and as a society. These “weapons of math destruction” score teachers and students, sort résumés, grant (or deny) loans, evaluate workers, target voters, set parole, and monitor our health. O’Neil calls on modelers to take more responsibility for their algorithms and on policy makers to regulate their use. But in the end, it’s up to us to become more savvy about the models that govern our lives. This important book empowers us to ask the tough questions, uncover the truth, and demand change. — Longlist for National Book Award (Non-Fiction) — Goodreads, semi-finalist for the 2016 Goodreads Choice Awards (Science and Technology) — Kirkus, Best Books of 2016 — New York Times, 100 Notable Books of 2016 (Non-Fiction) — The Guardian, Best Books of 2016 — WBUR's "On Point," Best Books of 2016: Staff Picks — Boston Globe, Best Books of 2016, Non-Fiction
A former Wall Street quant sounds an alarm on the mathematical models that pervade modern life - and threaten to rip apart our social fabric We live in the age of the algorithm. Increasingly, the decisions that affect our lives - where we go to school, whether we get a loan, how much we pay for insurance - are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: everyone is judged according to the same rules, and bias is eliminated. And yet, as Cathy O'Neil reveals in this urgent and necessary book, the opposite is true. The models being used today are opaque, unregulated, and incontestable, even when they're wrong. Most troubling, they reinforce discrimination. Tracing the arc of a person's life, O'Neil exposes the black box models that shape our future, both as individuals and as a society. These "weapons of math destruction" score teachers and students, sort CVs, grant or deny loans, evaluate workers, target voters, and monitor our health. O'Neil calls on modellers to take more responsibility for their algorithms and on policy makers to regulate their use. But in the end, it's up to us to become more savvy about the models that govern our lives. This important book empowers us to ask the tough questions, uncover the truth, and demand change.
Doing Data Science
Author: Cathy O'Neil, Rachel Schutt
Publisher: "O'Reilly Media, Inc."
Now that people are aware that data can make the difference in an election or a business model, data science as an occupation is gaining ground. But how can you get started working in a wide-ranging, interdisciplinary field that’s so clouded in hype? This insightful book, based on Columbia University’s Introduction to Data Science class, tells you what you need to know. In many of these chapter-long lectures, data scientists from companies such as Google, Microsoft, and eBay share new algorithms, methods, and models by presenting case studies and the code they use. If you’re familiar with linear algebra, probability, and statistics, and have programming experience, this book is an ideal introduction to data science. Topics include: Statistical inference, exploratory data analysis, and the data science process Algorithms Spam filters, Naive Bayes, and data wrangling Logistic regression Financial modeling Recommendation engines and causality Data visualization Social networks and data journalism Data engineering, MapReduce, Pregel, and Hadoop Doing Data Science is collaboration between course instructor Rachel Schutt, Senior VP of Data Science at News Corp, and data science consultant Cathy O’Neil, a senior data scientist at Johnson Research Labs, who attended and blogged about the course.
On Being a Data Skeptic
Author: Cathy O'Neil
Publisher: "O'Reilly Media, Inc."
"Data is here, it's growing, and it's powerful." Author Cathy O'Neil argues that the right approach to data is skeptical, not cynical––it understands that, while powerful, data science tools often fail. Data is nuanced, and "a really excellent skeptic puts the term 'science' into 'data science.'" The big data revolution shouldn't be dismissed as hype, but current data science tools and models shouldn't be hailed as the end-all-be-all, either.
A revealing look at how negative biases against women of color are embedded in search engine results and algorithms Run a Google search for “black girls”—what will you find? “Big Booty” and other sexually explicit terms are likely to come up as top search terms. But, if you type in “white girls,” the results are radically different. The suggested porn sites and un-moderated discussions about “why black women are so sassy” or “why black women are so angry” presents a disturbing portrait of black womanhood in modern society. In Algorithms of Oppression, Safiya Umoja Noble challenges the idea that search engines like Google offer an equal playing field for all forms of ideas, identities, and activities. Data discrimination is a real social problem; Noble argues that the combination of private interests in promoting certain sites, along with the monopoly status of a relatively small number of Internet search engines, leads to a biased set of search algorithms that privilege whiteness and discriminate against people of color, specifically women of color. Through an analysis of textual and media searches as well as extensive research on paid online advertising, Noble exposes a culture of racism and sexism in the way discoverability is created online. As search engines and their related companies grow in importance—operating as a source for email, a major vehicle for primary and secondary school learning, and beyond—understanding and reversing these disquieting trends and discriminatory practices is of utmost importance. An original, surprising and, at times, disturbing account of bias on the internet, Algorithms of Oppression contributes to our understanding of how racism is created, maintained, and disseminated in the 21st century.
A revealing look at how tech industry bias and blind spots get baked into digital products—and harm us all. Buying groceries, tracking our health, finding a date: whatever we want to do, odds are that we can now do it online. But few of us ask why all these digital products are designed the way they are. It’s time we change that. Many of the services we rely on are full of oversights, biases, and downright ethical nightmares: Chatbots that harass women. Signup forms that fail anyone who’s not straight. Social media sites that send peppy messages about dead relatives. Algorithms that put more black people behind bars. Sara Wachter-Boettcher takes an unflinching look at the values, processes, and assumptions that lead to these and other problems. Technically Wrong demystifies the tech industry, leaving those of us on the other side of the screen better prepared to make informed choices about the services we use—and demand more from the companies behind them.
What Algorithms Want
Author: Ed Finn
Publisher: MIT Press
The gap between theoretical ideas and messy reality, as seen in Neal Stephenson, Adam Smith, and Star Trek. We depend on—we believe in—algorithms to help us get a ride, choose which book to buy, execute a mathematical proof. It's as if we think of code as a magic spell, an incantation to reveal what we need to know and even what we want. Humans have always believed that certain invocations—the marriage vow, the shaman's curse—do not merely describe the world but make it. Computation casts a cultural shadow that is shaped by this long tradition of magical thinking. In this book, Ed Finn considers how the algorithm—in practical terms, “a method for solving a problem”—has its roots not only in mathematical logic but also in cybernetics, philosophy, and magical thinking. Finn argues that the algorithm deploys concepts from the idealized space of computation in a messy reality, with unpredictable and sometimes fascinating results. Drawing on sources that range from Neal Stephenson's Snow Crash to Diderot's Encyclopédie, from Adam Smith to the Star Trek computer, Finn explores the gap between theoretical ideas and pragmatic instructions. He examines the development of intelligent assistants like Siri, the rise of algorithmic aesthetics at Netflix, Ian Bogost's satiric Facebook game Cow Clicker, and the revolutionary economics of Bitcoin. He describes Google's goal of anticipating our questions, Uber's cartoon maps and black box accounting, and what Facebook tells us about programmable value, among other things. If we want to understand the gap between abstraction and messy reality, Finn argues, we need to build a model of “algorithmic reading” and scholarship that attends to process, spearheading a new experimental humanities.
Author: Seth Stephens-Davidowitz
Foreword by Steven Pinker Blending the informed analysis of The Signal and the Noise with the instructive iconoclasm of Think Like a Freak, a fascinating, illuminating, and witty look at what the vast amounts of information now instantly available to us reveals about ourselves and our world—provided we ask the right questions. By the end of an average day in the early twenty-first century, human beings searching the internet will amass eight trillion gigabytes of data. This staggering amount of information—unprecedented in history—can tell us a great deal about who we are—the fears, desires, and behaviors that drive us, and the conscious and unconscious decisions we make. From the profound to the mundane, we can gain astonishing knowledge about the human psyche that less than twenty years ago, seemed unfathomable. Everybody Lies offers fascinating, surprising, and sometimes laugh-out-loud insights into everything from economics to ethics to sports to race to sex, gender and more, all drawn from the world of big data. What percentage of white voters didn’t vote for Barack Obama because he’s black? Does where you go to school effect how successful you are in life? Do parents secretly favor boy children over girls? Do violent films affect the crime rate? Can you beat the stock market? How regularly do we lie about our sex lives and who’s more self-conscious about sex, men or women? Investigating these questions and a host of others, Seth Stephens-Davidowitz offers revelations that can help us understand ourselves and our lives better. Drawing on studies and experiments on how we really live and think, he demonstrates in fascinating and often funny ways the extent to which all the world is indeed a lab. With conclusions ranging from strange-but-true to thought-provoking to disturbing, he explores the power of this digital truth serum and its deeper potential—revealing biases deeply embedded within us, information we can use to change our culture, and the questions we’re afraid to ask that might be essential to our health—both emotional and physical. All of us are touched by big data everyday, and its influence is multiplying. Everybody Lies challenges us to think differently about how we see it and the world.
Author: Virginia Eubanks
Publisher: St. Martin's Press
The New York Times Book Review: "Riveting." Naomi Klein: "This book is downright scary." Ethan Zuckerman, MIT: "Should be required reading." Dorothy Roberts, author of Killing the Black Body: "A must-read." Astra Taylor, author of The People's Platform: "The single most important book about technology you will read this year." Cory Doctorow: "Indispensable." A powerful investigative look at data-based discrimination—and how technology affects civil and human rights and economic equity The State of Indiana denies one million applications for healthcare, foodstamps and cash benefits in three years—because a new computer system interprets any mistake as “failure to cooperate.” In Los Angeles, an algorithm calculates the comparative vulnerability of tens of thousands of homeless people in order to prioritize them for an inadequate pool of housing resources. In Pittsburgh, a child welfare agency uses a statistical model to try to predict which children might be future victims of abuse or neglect. Since the dawn of the digital age, decision-making in finance, employment, politics, health and human services has undergone revolutionary change. Today, automated systems—rather than humans—control which neighborhoods get policed, which families attain needed resources, and who is investigated for fraud. While we all live under this new regime of data, the most invasive and punitive systems are aimed at the poor. In Automating Inequality, Virginia Eubanks systematically investigates the impacts of data mining, policy algorithms, and predictive risk models on poor and working-class people in America. The book is full of heart-wrenching and eye-opening stories, from a woman in Indiana whose benefits are literally cut off as she lays dying to a family in Pennsylvania in daily fear of losing their daughter because they fit a certain statistical profile. The U.S. has always used its most cutting-edge science and technology to contain, investigate, discipline and punish the destitute. Like the county poorhouse and scientific charity before them, digital tracking and automated decision-making hide poverty from the middle-class public and give the nation the ethical distance it needs to make inhumane choices: which families get food and which starve, who has housing and who remains homeless, and which families are broken up by the state. In the process, they weaken democracy and betray our most cherished national values. This deeply researched and passionate book could not be more timely.
The seventeen thought-provoking and engaging essays in this collection present readers with a wide range of diverse perspectives on the ontology of mathematics. The essays address such questions as: What kind of things are mathematical objects? What kinds of assertions do mathematical statements make? How do people think and speak about mathematics? How does society use mathematics? How have our answers to these questions changed over the last two millennia, and how might they change again in the future? The authors include mathematicians, philosophers, computer scientists, cognitive psychologists, sociologists, educators and mathematical historians; each brings their own expertise and insights to the discussion. Contributors to this volume: Jeremy Avigad Jody Azzouni David H. Bailey David Berlinski Jonathan M. Borwein Ernest Davis Philip J. Davis Donald Gillies Jeremy Gray Jesper Lützen Ursula Martin Kay O’Halloran Alison Pease Steven Piantadosi Lance Rips Micah T. Ross Nathalie Sinclair John Stillwell Hellen Verran
The powerful, unforgettable graphic memoir from Jarrett Krosoczka, about growing up with a drug-addicted mother, a missing father, and two unforgettably opinionated grandparents.
How to make simple sense of complex statistics--from the author of Numbers Rule Your World We live in a world of Big Data--and it's getting bigger every day. Virtually every choice we make hinges on how someone generates data . . . and how someone else interprets it--whether we realize it or not. Where do you send your child for the best education? Big Data. Which airline should you choose to ensure a timely arrival? Big Data. Who will you vote for in the next election? Big Data. The problem is, the more data we have, the more difficult it is to interpret it. From world leaders to average citizens, everyone is prone to making critical decisions based on poor data interpretations. In Numbersense, expert statistician Kaiser Fung explains when you should accept the conclusions of the Big Data "experts"--and when you should say, "Wait . . . what?" He delves deeply into a wide range of topics, offering the answers to important questions, such as: How does the college ranking system really work? Can an obesity measure solve America's biggest healthcare crisis? Should you trust current unemployment data issued by the government? How do you improve your fantasy sports team? Should you worry about businesses that track your data? Don't take for granted statements made in the media, by our leaders, or even by your best friend. We're on information overload today, and there's a lot of bad information out there. Numbersense gives you the insight into how Big Data interpretation works--and how it too often doesn't work. You won't come away with the skills of a professional statistician. But you will have a keen understanding of the data traps even the best statisticians can fall into, and you'll trust the mental alarm that goes off in your head when something just doesn't seem to add up. Praise for Numbersense "Numbersense correctly puts the emphasis not on the size of big data, but on the analysis of it. Lots of fun stories, plenty of lessons learned—in short, a great way to acquire your own sense of numbers!" Thomas H. Davenport, coauthor of Competing on Analytics and President’s Distinguished Professor of IT and Management, Babson College "Kaiser’s accessible business book will blow your mind like no other. You’ll be smarter, and you won’t even realize it. Buy. It. Now." Avinash Kaushik, Digital Marketing Evangelist, Google, and author, Web Analytics 2.0 "Each story in Numbersense goes deep into what you have to think about before you trust the numbers. Kaiser Fung ably demonstrates that it takes skill and resourcefulness to make the numbers confess their meaning." John Sall, Executive Vice President, SAS Institute "Kaiser Fung breaks the bad news—a ton more data is no panacea—but then has got your back, revealing the pitfalls of analysis with stimulating stories from the front lines of business, politics, health care, government, and education. The remedy isn’t an advanced degree, nor is it common sense. You need Numbersense." Eric Siegel, founder, Predictive Analytics World, and author, Predictive Analytics "I laughed my way through this superb-useful-fun book and learned and relearned a lot. Highly recommended!" Tom Peters, author of In Search of Excellence
Hand to Mouth
Author: Linda Tirado
Originally published in hardcover in 2014 by G.P. Putnam's Sons.