New Personality Test is Faster – and Tougher to Trick

Psychology researchers have developed a new personality test that is both faster to take and much harder to manipulate by those attempting to control the outcome.

“The test draws on the ‘Big Five’ approach to personality assessment, so it’s a new method for personality testing, rather than a new paradigm,” says Adam Meade, a professor of psychology at North Carolina State University and lead author of a paper on the work.

“But whether the testing is being done as part of workplace training or for research purposes, there is real value in being able to conduct the test quickly and in making it difficult for users to game the system.”

The Big Five theory of personality focuses on five broad factors: extraversion, conscientiousness, openness, neuroticism and agreeableness.

“We used this paradigm because it is supported by the broadest body of research and holds up across cultures and demographic groups,” Meade says.

The new test is an online program that relies on a technique developed by Meade called rapid response measurement (RRM), which displays a series of adjectives one after another. Users have a few seconds to click whether each adjective is “like me” or “not like me.”

“People take about one second, on average, to make a decision for each word,” Meade says. “This allows us to collect a significant amount of data in a very short period of time – orders of magnitude faster than other tests. The rapid response rate also makes it difficult to manipulate the outcome, and our software incorporates response time into its analysis of responses. For example, if users take too long to respond to an adjective, that response does not have as much impact on scores as adjectives with faster responses.”

Altogether, the new paper includes the results from four studies. In three of the studies, involving a total of 425 participants, researchers evaluated the accuracy of the RRM Big Five test when compared to well-established Big Five testing protocols. All three studies found that the results were comparable, even though the RRM technique took far less time.

In the third study, involving 205 people, researchers also ran an iteration of the test asking study participants to manipulate the outcome. They found it was four times harder for participants to alter the outcome of the RRM test, as compared to the conventional test.

And the RRM protocol can be used to assess things other than the Big Five.

The paper’s fourth study focused on workplace personality testing and involved 228 business managers from across the United States. For this study, researchers compared the results of the RRM test to an established workplace personality test – and to assessments from the study participants’ real-world supervisors and peers. The RRM test results tracked more closely with the actual workplace assessments than the traditional tests did.

“We’re optimistic that our approach can streamline the testing process and provide more accurate information for use in training, hiring and research,” Meade says.

Meade has launched a company called PerSight Assessments to market RRM testing for use by employers, and also plans to make a free version available for the research community.

“We’re now working on adapting RRM to measure attitudes toward workplace satisfaction, workplace culture and consumer feedback,” Meade says. “It’s a flexible tool, and there are likely applications we haven’t even thought of yet.”

The paper, “Development of a Faking-Resistant Assessment Method for Personality,” is published in the journal Organizational Research Methods. The paper was co-authored by Gabriel Pappalardo, a former Ph.D. student at NC State; and Phillip Braddy and John Fleenor of the Center for Creative Leadership.

-shipman-

Note to Editors: The study abstract follows.

“Development of a Faking-Resistant Assessment Method for Personality”

Authors: Adam W. Meade and Gabriel Pappalardo, North Carolina State University; Phillip Braddy and John Fleenor, Center for Creative Leadership

Published: Aug. 29, Organizational Research Methods

DOI: 10.1177/1094428118795295

Abstract: While rating-scale-based assessments have been shown to be useful for measuring a variety of workplace-relevant constructs, assessment length and response distortion present practical limitations on their use. We describe a new type of measurement method termed rapid response measurement (RRM) in which stimuli are presented on a computer screen one at a time in rapid succession and respondents are asked to quickly provide a dichotomous response. Two personality assessments using RRM were developed and reliability and validity evidence across four independent samples were evaluated. Both RRM assessments showed adequate reliability, even at short test lengths, with acceptable levels of convergent and discriminant validity with traditional survey-based measures. Analyses based on a within-participants design indicated that the RRM was significantly more difficult to fake when instructed than was a survey-based measure of personality. The second RRM was related to several aspects of job performance. While initial results show promise, further research is needed to establish the validity and viability of the RRM for organizational and psychological measurement.

This post was originally published in NC State News.