Examining the Impact of a Reasoning Aid to Help People Evaluate the Evidentiary Weight of Consensus

Abstract

Social media is a vortex of information and people may see distorted views of consensus, where the independence of information and sources is unclear. A tool that summarises consensus information might help people to navigate these important cues. This study examined whether a reasoning aid (in the form of a diagram) visually illustrating both the number of independent people supporting/disagreeing with a claim and the diversity of arguments would persuade people to change their original beliefs. Participants (n=605) were recruited through Amazon’s Mechanical Turk to evaluate 24 claims on a mock Twitter interface. Participants were randomly assigned to conditions with either tweets only, diagram only or tweets with a diagram. Participants rated their initial agreement level (0-100) with each claim and then saw the diagram and/or set of tweets, then were able to update their agreement level if their original opinion had now changed. The findings of this study show that without assistance, people mostly rely on cues of argument quantity, such as the number of tweets for a given stance. However, when presented with a diagram, people were able to utilise cues of argument quality, such as when there were different sources providing the information and when multiple arguments were used.