Survey researchers have reported differing results on frequency distributions when the same item is delivered via an interactive voice response (IVR) system versus the web. The current paper expands such research into the organizational research field and evaluated the hypothesis that respondent motivation affects the occurrence of mode differences. In this study, personnel selection instruments using five‐point Likert scales were administered to job applicants and job incumbents. Data were collected via IVR or via the web. With job incumbents, the mode effect observed was similar in magnitude to that observed in the survey research literature. However, with job applicants the mode effect was smaller.