Label-free Bias Mitigation For Fair Generative Modeling
Large-scale generative models, such as GPT, for both language and vision domains are trained on a variety of data sources scraped from the internet. Unsurprisingly, these data sources are often biased with respect to key demographic factors such as gender and race. Due to the latent nature of the underlying factors, detecting and mitigating bias is especially challenging for unsupervised machine learning. In this talk, I will present a model-agnostic and label-free approach for mitigating the bias of deep generative models based on importance weighting. Empirically, we demonstrate the efficacy of our approach which reduces bias with respect to latent factors by an average of up to 34.6% over baselines for comparable image generation using generative adversarial networks.
Key Takeaways: - Generative models can be trained on large, Internet-scale datasets. Many compelling examples show that in doing so, they can amplify harmful biases in the dataset. - Fixing these biases is hard because the datasets used for training are unlabelled. - Fortunately, we do not need expensive labels for fixing the dataset bias and can instead rely on weaker forms of supervision.
Aditya Grover is a research scientist at Facebook AI Research, a visiting postdoctoral researcher at UC Berkeley, and an incoming assistant professor of computer science at UCLA (starting Fall 2021). His research focuses on probabilistic modeling for representation learning and reasoning in high dimensions, and is grounded in applications in science and sustainability, such as weather forecasting and electric batteries. Aditya’s research has been published in top machine learning and scientific venues including Nature, covered by various media outlets, included in widely-used open source software, and deployed into production at major technology companies. He has won several awards, including a best paper award (StarAI), a best undergraduate thesis award, a Stanford Centennial Teaching Award, a Stanford Data Science Scholarship, a Lieberman Fellowship, and a Microsoft Research Ph.D. Fellowship. Aditya received his Ph.D. and masters from Stanford University in 2020 and bachelors from IIT Delhi in 2015, all in computer science.