Failed Machine Learning (FML)

High-profile real-world examples of failed machine learning projects

---

“Success is not final, failure is not fatal. It is the courage to continue that counts.” - Winston Churchill

If you are looking for examples of how ML can fail despite all its incredible potential, you have come to the right place. Beyond the wonderful success stories of applied machine learning, here is a list of failed projects which we can learn a lot from.

---

Contents

1. Classic Machine Learning
2. Computer Vision
3. Forecasting
4. Image Generation
5. Natural Language Processing
6. Recommendation Systems

---

Classic Machine Learning

Title	Description
Amazon AI Recruitment System	AI-powered automated recruitment system canceled after evidence of discrimination against female candidates
Genderify - Gender identification tool	AI-powered tool designed to identify gender based on fields like name and email address was shut down due to built-in biases and inaccuracies
Leakage and the Reproducibility Crisis in ML-based Science	A team at Princeton University found 20 reviews across 17 scientific fields that discovered significant errors (e.g., data leakage, no train-test split) in 329 papers that use ML-based science
COVID-19 Diagnosis and Triage Models	Hundreds of predictive models were developed to diagnose or triage COVID-19 patients faster, but ultimately none of them were fit for clinical use, and some were potentially harmful
COMPAS Recidivism Algorithm	Florida’s recidivism risk system found evidence of racial bias
Pennsylvania Child Welfare Screening Tool	The predictive algorithm (which helps identify which families are to be investigated by social workers for child abuse and neglect) flagged a disproportionate number of Black children for 'mandatory' neglect investigations.
Oregon Child Welfare Screening Tool	A similar predictive tool to the one in Pennsylvania, the AI algorithm for child welfare in Oregon was also stopped a month after the Pennsylvania report
U.S. Healthcare System Health Risk Prediction	A widely used algorithm to predict healthcare needs exhibited racial bias where for a given risk score, black patients are considerably sicker than white patients
Apple Card Credit Card	Apple’s new credit card (created in partnership with Goldman Sachs) is being investigated by financial regulators after customers complained that the card’s lending algorithms discriminated against women, where the credit line offered by a male customer's Apple Card was 20 times higher than that offered to his spouse

---

Computer Vision

Title	Description
[Inverness Automated Football Camera System](https://www.theverge.com/tldr/2020/11/3/21547392