AI systems are enabling new experiences and abilities for people around the globe. Beyond recommending books and television shows, AI systems can be used for more critical tasks, such as predicting the presence and severity of a medical condition, matching people to jobs and partners, or identifying if a person is crossing the street. Such computerized assistive or decision-making systems have the potential to be fairer and more inclusive at a broader scale than decision-making processes based on ad hoc rules or human judgments. The risk is that any unfairness in such systems can also have a wide-scale impact. Thus, as the impact of AI increases across sectors and societies, it is critical to work towards systems that are fair and inclusive for all.
This is a hard task. First, ML models learn from existing data collected from the real world, and so an accurate model may learn or even amplify problematic pre-existing biases in the data based on race, gender, religion or other characteristics. For example, a job-matching system might learn to favor male candidates for CEO interviews, or assume female pronouns when translating words like “nurse” or “babysitter” into Spanish, because that matches historical data.
Second, even with the most rigorous and cross-functional training and testing, it is a challenge to ensure that a system will be fair across all situations. For example, a speech recognition system that was trained on US adults may be fair and inclusive in that context. When used by teenagers, however, the system may fail to recognize evolving slang words or phrases. If the system is deployed in the United Kingdom, it may have a harder time with certain regional British accents than others. And even when the system is applied to US adults, we might discover unexpected segments of the population whose speech it handles poorly, for example people speaking with a stutter. Use of the system after launch can reveal unintentional, unfair blind spots that were difficult to predict.
Third, there is no standard definition of fairness, whether decisions are made by humans or machines. Identifying appropriate fairness criteria for a system requires accounting for user experience, cultural, social, historical, political, legal, and ethical considerations, several of which may have tradeoffs. Is it more fair to give loans at the same rate to two different groups, even if they have different rates of payback, or is it more fair to give loans proportional to each group’s payback rates? Is neither of these the most fair approach? At what level of granularity should groups be defined, and how should the boundaries between groups be decided? When is it fair to define a group at all versus better factoring on individual differences? Even for situations that seem simple, people may disagree about what is fair, and it may be unclear what point of view should dictate policy, especially in a global setting.
Addressing fairness and inclusion in AI is an active area of research, from fostering an inclusive workforce that embodies critical and diverse knowledge, to assessing training datasets for potential sources of bias, to developing unbiased models, to evaluating machine learning models for disparities in performance, to continued testing of final systems for unfair outcomes. In fact, ML models can even be used to identify some of the conscious and unconscious human biases and barriers to inclusion that have developed and perpetuated throughout history, bringing about positive change. Far from a solved problem, fairness in AI presents both an opportunity and a challenge. Google is committed to making progress in all of these areas, and to creating tools, datasets, and other resources for the larger community. Our current thinking at Google is outlined below.