Machine learning and prejudice: building theory with algorithm-supported abduction

Abstract

Machine learning is a powerful analytical tool that can identify robust and replicable patterns in complex datasets, and create models with high predictive power. With interpretable machine learning, these models can be queried to identify the most important predictors of an outcome variable from hundreds of potential predictors. I propose that these machine learning capabilities can be used to engage in abductive reasoning, that is, identifying the most likely explanations of important phenomena in an empirical manner. I will first review past research in management using machine learning, and then describe two empirical projects in which I used machine learning to generate novel hypotheses about antecedents of sexism and racism in the US context. I verified these hypotheses using conventional research methods and identified the underlying mechanisms. The findings suggest that machine models can help expand the scope of researchers’ explanatory frameworks, and thereby identify neglected directions that can benefit from further theorizing.