Exploring the Gap Between Privacy and Utility in Automated Decision-Making

The rapid rise of automated decision-making systems has left a gap in researchers’ understanding of how developers and consumers balance concerns about the privacy and accuracy of such systems against their utility.  With our goal to cover a broad spectrum of concerns from various angles, we initiated two experiments on the perceived benefit and detriment of interacting with automated decision-making systems. We refer to these two experiments as the Patch Wave study and Automated Driving study. This work approaches the study of automated decision making at different perspectives to help address the gap in empirical data on consumer and developer concerns. In our Patch Wave study, we focus on developers’ interactions with automated pull requests that patch widespread vulnerabilities on GitHub. The Automated Driving study explores older adults’ perceptions of data privacy in highly automated vehicles. We find quantitative and qualitative differences in the way that our target populations view automated decision-making systems compared to human decision-making. In this work, we detail our methodology for these studies, experimental results, and recommendations for addressing consumer and developer concerns.