Monte Carlo examination of common approaches for detecting response styles

Abstract

Response styles (RS) are a respondent’s tendency to respond to questions in certain ways regardless of the content. RS received increasing interest, as it exerted negative impact on assessment of true scores by inflating or deflating observed values. A variety of approaches were proposed to identify various RS, including: Count Procedure, Representative Indicators Response Style (RIRS), Representative Indicators Response Style Means and Covariance Structure (RIRSMACS), and Multidimensional Nominal Response Model (MNRM), etc. However, there is lack of systematic simulation studies to examine performance of Count Procedure, RIRS, and RIRSMACS; examination of MNRM in prior simulation studies is not without limitations. Therefore, the present study explored performance of RS approaches under various simulation conditions, including different sample size (300 vs. 800), test length (10 vs. 20), and RS (No RS, Acquiescent RS, Extreme RS, vs. Mid-point RS). Results suggested that across 4 approaches explored in the present study, it was easier to detect MRS than ARS and ERS. Among 4 RS approaches, MNRM performed better than other approaches in picking out the correct RS. Results of simulation studies were discussed and recommendations of employing these approaches to identify RS were provided.