Classifier Fairness and How to Measure It

Speaker:  Dr. Sivan Sabato | Associate Professor
Department of Computing and Software, McMaster University
Canada CIFAR AI Chair, Vector Institute

Date: April 24, 2025
Time: 4:00 pm
Room: LH 3058 (Math Boardroom) & Hybrid

Summary:  Discrimination by AI is widespread, resulting in some groups being treated unfairly by systems that incorporate AI. The field of algorithmic fairness studies methods for combating algorithmic discrimination. An essential step is formalizing a notion of fairness for classifiers. In this talk, I will demonstrate why this is more challenging than one might initially assume, and discuss types of formal fairness notions. I will then consider the challenges of measuring unfairness and auditing the fairness of classifiers that are not directly accessible, such as proprietary classifiers used by private companies. I will present a principled approach for quantifying unfairness, and methods for drawing conclusions on the unfairness of classifiers from limited aggregate statistics.

This talk is based on joint work with Eran Treister (Ben-Gurion University) and Elad Yom-Tov (Bar-Ilan University).

Biography:  Dr. Sivan Sabato is an Associate Professor at the Department of Computing and Software at McMaster University, a Canada CIFAR AI Chair and a Vector Institute faculty member. Sivan serves as an Action Editor for the Journal of Machine Learning Research and for Transactions of Machine Learning Research and is a frequently holds senior organization positions at machine learning conferences such as ICML, COLT and ALT. Sivan's research interests include machine learning theory, interactive learning algorithms, and algorithmic fairness.