Operationalizing Responsible AI in Large-Scale Organizations
Most large-scale organizations face challenges while scaling their infrastructure to support multiple teams across multiple product domains. More often than not, individual teams build systems and models to power their specific product areas, but because of the innate differences in the products and infrastructure support, the broad use of Responsible AI techniques poses a serious challenge for organizations.
Each product can potentially have a different definition of “fairness” across different dimensions and hence require very different measurement and mitigation solutions. In this talk, we will focus on how we are building a scalable system on our machine learning platform that can not only measure but also mitigate unintended consequences of AI models across most products at LinkedIn.
We will discuss how this system aims to seamlessly integrate into each and every AI pipeline and measures unfairness across different protected attributes. The system is flexible to incorporate different definitions of fairness as required by the product. Moreover, if and when algorithmic bias is detected we also have a system to remove such bias through state-of-the-art AI algorithms across different notions of fairness. That being said, we are just starting and there is much more work to be done and we don’t have all the answers yet.
Finally, all of the above point to having a good intent towards ethical practices. But the real win comes from the actual member impact after launching such bias mitigated models in production. We will also discuss how we A/B test our models and systems once they are launched in production and incorporate those learnings to improve the overall member experience. Thus, connecting the overall intent and impact cycle.
Kinjal is currently a Sr. Staff Software Engineer and the tech lead for Responsible AI at LinkedIn, focusing on challenging problems in fairness, explainability, and privacy. He leads several initiatives across different product applications towards making LinkedIn a responsible and equitable platform. He received his Ph.D. in Statistics from Stanford University, with a best thesis award and has several published papers in many top journals and conferences. He has been serving as a reviewer and program committee member in multiple top venues such as NeurIPS, ICML, KDD, FAccT, WWW, etc