Raising the Stakes: Assessing the Influence of Stakes on User Reliance Behavior in Human-AI Decision-Making

Human-AI collaboration is often proposed to improve highstakes decisionmaking, yet the influence of increased stakes and imperfect AI on decisionmaking strategies is not fully understood. Studying such behavior in realistic settings is challenging, as applicationgrounded evaluations are costly, rely on experts, or lack meaningful consequences for decision errors. To address this, we introduce Blockies, a parametric dataset generator for visual diagnostic tasks, and conduct an empirical study examining how perceived stakes influence reliance calibration and behavior. Results show that raised stakes lead to longer deliberation, but less calibrated reliance, with participants increasingly deferring to incorrect AI advice as decision time increased. These findings highlight that increased effort under higher stakes does not necessarily improve reliance calibration and show the importance of accounting for stakes when evaluating human-AI decision-making.