Automatic Causal Fairness Analysis with LLM-Generated Reporting

arXiv:2604.27011v2 Announce Type: replace-cross
Abstract: AutoML, intended as the process of automating the application of machine learning to real-world problems, is a key step for AI popularisation. Most AutoML frameworks are not accounting for the potential lack of fairness in the training data and in the corresponding predictions. We introduce textscFairMind, a software prototype aiming to automatise fairness analysis at the dataset level. We achieve that by resorting to the assumptions of the emphstandard fairness model, recently proposed by Plevcko and Bareinboim. This allows for a sound fairness evaluation in terms of causal effects, based on emphcounterfactual queries involving the target, possibly confounders and mediators, and the different values of an input feature we regard as emphprotected. After the necessary data preprocessing, the tool implements a closed-form computation of the effects. LLMs are consequently exploited to generate accurate reports on the fairness levels detected in the training dataset. We achieve that in a zero-shot setup and show by examples the expected advantages with respect to a direct analysis performed by the LLM. To favour applications, extensions to ordinal protected variable and continuous targets and novel decomposition results are also discussed.