Title of talk: Extremity Bias in Online Reviews: A Field Experiment
Abstract: In a range of studies across many platforms, submitted online ratings have been shown to be characterized by a distribution with disproportionately-heavy tails. These have been referred to as “j-shaped” or “extreme” distributions. Our focus in this paper is on understanding the underlying process that yields such a distribution. We develop a simple analytical model to capture the most-common explanations: differences in utility associated with posting extreme vs moderate reviews, and differences in base rates associated with extreme vs moderate reviews. We compare the predictions of these explanations with those of an alternative memory-based explanation based on customers forgetting about writing a review over time. The forgetting rate, by assumption, is higher for moderate reviews. The three models yield stark differences in the predicted dynamics of extremity bias. To test our predictions, we use data from a large-scale field experiment with an online travel platform. In this experiment, we varied the time at which the firm sent out a review solicitation email. Specifically, the time of review solicitation ranged between one and nine days after the end of one’s vacation. This manipulation allows us to observe the extremity dynamics over an extended period both before and after the firm’s solicitation email. Our results clearly support the predictions from the memory-based explanation, but are inconsistent with those from the utility-based and base-rate explanations.