How does the PI Behavioral Assessment Compare to the Five Factor Model?

At PI, we are frequently asked how the PI Behavioral Assessment compares to Five Factor Assessments. Although this is a great question, there is not a simple answer. In this article, we provide some information to help you understand the differences. The main distinctions are:

• The PI Behavioral Assessment was built around a theory, while the Five Factor Model (FFM) was derived empirically from observations. This isn’t a big deal, but it is worth noting.
• The PI Behavioral Assessment measures four Primary Factors and one Secondary Factor. The Five Factor Model measures five broad traits.
• The Five Factor Model is generally a broader model of personality measurement than the PI Behavioral Assessment. The PI Behavioral Assessment was designed to measure specific workplace-relevant personality as opposed to being an all-encompassing model of general personality.
• Many experts agree that broad isn’t always better when working with a specific application. For example, the FFM’s Conscientiousness trait is typically the only relevant measure when one is interested in workplace performance. However, with the PI Behavioral Assessment, each Factor can be very important, depending on the job in question.
• In most cases, both the PI Behavioral Assessment and the FFM predict job performance at a similar level; however, the PI Behavioral Assessment is much more efficient. It attains similar validity and reliability as the FFM, but in only six minutes, on average.
• The PI Behavioral Assessment has only a weak relationship with the FFM trait of Neuroticism. This may be a good thing, since using Neuroticism in hiring may select out people with psychological challenges, including some covered by the Americans with Disabilities Act (ADA).
• The PI Behavioral Assessment doesn’t have any relationship to the FFM trait of Openness to Experience, but does not take away from its usability. The PI Behavioral Assessment is a workplace assessment, and research shows that Openness to Experience is not a good predictor of job performance.

The Five Factor Model: Five Broad Factors, But Not Theory Based

The Five Factor Model (also called the “FFM” or “Big Five” or “OCEAN” model) has become widely accepted by the scientific community as the†model of normal personality. The model itself is not a theoretical one; instead, it is based on research going back to the 1960’s. Tupes and Christal (1961) evaluated all of the personality descriptors (e.g., curious, dutiful) in human language and used a statistical categorization tool called factor†analysis†to conclude that there were five broad personality factors underlying human behavior. In other words, the Five Factor Model is an organizing structure based on the notion that every existing personality characteristic is influenced by the following five broad traits:

1. Openness to Experience (intellectually curious and inquisitive)
2. Conscientiousness (rule, detail, and achievement-orientation)
3. Extraversion (outgoing, sociable, and talkative)
4. Agreeableness (friendly, sensitive, and interpersonally aware)
5. Neuroticism (worrisome, anxious, self-critical, impacted by stress)

For research purposes, the FFM has been beneficial because it has allowed psychologists and other interested scientists to evaluate the relationship between personality and meaningful outcomes (e.g., work performance) using a simple, common language. Prior to the FFM language, research on the power of personality assessments was often inconclusive because the findings were based on specific traits which did not always generalize beyond the context of the studies. With a unified language, key meta-analytic research in by Barrick and Mount (1991) and Tett, Rothstein and Jackson (1991) provided the initial evidence that personality was a generalizable predictor of work performance, and in particular, that Conscientiousness was the strongest and most consistent of the five factors.

The PI Behavioral Assessment: Four Primary Theory-Based Factors

In contrast to the FFM’s atheoretical approach, the PI Behavioral Assessment was designed based on a theoretical model that originated 60 years before the Five-Factor Model became a widely-accepted framework of personality.

The PI Behavioral Assessment is based largely on personality trait theories of the physiological psychologist (and inventor of the lie detector) William Marston, as well as psychologist Prescott Lecky, a pioneer of self-consistency theory and advisor to Carl Rogers, John F. Kennedy, and others.

Marston theorized that a person’s self-perception in relation to his or her environment gave rise to four ‘Primary Emotions’ which he termed ‘Dominance’, ‘Inducement’, ‘Submission’, and ‘Compliance.’

In 1955, Arnold Daniels, founder of The Predictive Index, transformed Marston’s theory into an assessment designed to measure these drives specifically for use in the workplace. We call these drives ‘Primary Factors,’ and they include Dominance (Factor A), Extraversion (Factor B), Patience (Factor C), and Formality (Factor D).

In addition to the Primary Factors, The Predictive Index also measures a Secondary Factor (Factor E) which is used as a modifying factor to describe a respondent’s decision-making style in the expression of the four Primary Factors (A-D). See Table 1 for definitions of the Factors. Since 1955, the PI Behavioral Assessment has been used by over ten thousand organizations for the purposes of workplace decision-making, employee development, and conflict resolution. Today, PI has over 500 validation studies, each of which show a meaningful link between the PI Behavioral Assessment’s Factors and job performance.

Table 1 PI Behavioral Assessment Factor Descriptions.

A Meaningful Difference: Broad vs. Narrow Measurement

On the surface, the PI Behavioral Assessment and the FFM appear to measure completely different personality characteristics (with the exception of Extraversion); however, that isn’t the whole story. Understanding how they differ requires a deeper dive into the concepts of broad versus specific measurement. Recall that the FFM is not†theoretical in nature; it is merely an organizing structure derived statistically through factor analysis that suggests that the many traits, characteristics, and descriptions of human behavior identified in personality research converge on about five independent, but broad, factors. Each of these factors (Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism) is a general personality factor that is composed of many facets, or narrow personality traits (Ashton, 1998).

As such, the FFM can be thought of as a hierarchical model where narrow, more specific scales are combined together to measure a broader, higher-order characteristic. While experts agree with this overall conclusion, there is little agreement about how many facets exist within each of the five factors. For example, Costa and McCrae (1992) identified six specific-facet traits for each of the five factors on their NEO PI-R assessment (one of the better established FFM assessments). In their model, Conscientiousness is composed of the facets of Self-Efficacy, Orderliness, Dutifulness, Achievement-Striving, Self-Discipline, and Cautiousness. On the other hand, DeYoung, Quilty and Peterson (2007) conducted research which suggested a 10-facet structure (two for each broad FFM trait). The point here is not to debate how many facets thereare, but to simply raise awareness that they exist and that it matters for understanding the PI Behavioral Assessment.

The facets matter because the PI Behavioral Assessment’s Factors share relationships with a few of the Five Factors broadly, but also correlate closely with some conceptually aligned FFM facets. From that perspective, there is plenty of overlap between the models, but the PI Behavioral Assessment is purposefully more specific in its measurement because it is designed for workplace applications.

For example:

Factor A – Dominance shares variance with the FFM Extraversion scale, specifically Assertiveness, which DeYoung et al. (2007) identified as one of two facet scales underlying Extraversion. It also has a small but meaningful negative relationship with Conformity–a facet of Conscientiousness.

Factor B – Extraversion aligns with the FFM scale Extraversion, but also has a solid relationship with the facet of Sociability, which refers to being comfortable interacting with others.

Factor C – Patience correlates with FFM Agreeableness and its facet of being Good-Natured, but it is also related to the Calmness and Patience components of Neuroticism.

Factor D – Formality correlates with Conscientiousness and facets like Methodicalness, while also being negatively correlated with being Non-Planful.

If you are wondering about the specific relationship between the FFM and the PI Behavioral Assessment, Table 2 shows the general findings from PI’s recent construct validation studies (Foster et al., 2016). The icons in the table are provided to give a general understanding of the overlap of the relationships rather than to address specific variance accounted for between the PI Factors and the FFM scales. In other words, we don’t purport the PI Behavioral Assessment Factors to be exact measures of any of the FFM traits, but as one would expect, they are clearly related. (Note that the Secondary Factor E is omitted from this discussion as it is not part of the primary PI Behavioral Assessment measurement model. This paper is focused on the relevant comparisons with the FFM.)

Table 2: Relationship between PI Primary Factors and FFM Scales.

Broad vs. Narrow When Predicting Job Performance

Do the broad FFM scales predict performance better, or do narrow-band personality traits like those on the PI Behavioral Assessment show more powerful predictive relationships? As is typically the case in social science, the answer isn’t that simple, and it continues to be the subject of debate. The debate has a name, ​the bandwidth-fidelity dilemma​, and a significant amount of research exists evaluating and supporting both sides of the argument. On one hand, researchers such as Ones and Viswesvaran (1996) have found that broad FFM measures outperform more narrow ones, while others have found that facet scales predict job performance above and beyond the FFM scales (Paunonen, 1998; Paunonen & Nicol, 2001; Stewart, 1999). A lot of research exists on the topic of the bandwidth-fidelity dilemma, so rather than enter the fray, the PI perspective is that there is legitimacy to both approaches.

This also means that, when it comes to the PI Behavioral Assessment versus the FFM, there is no reason to think that the FFM holds a distinct and clear advantage in job performance prediction over the PI Behavioral Assessment. In fact, Hogan and Roberts (1996) argue that the use of narrow personality traits accounts for variance that is situation-specific, which aligns with PI’s philosophy that every job is specific in as much as it is exists in its own context, culture, manager, etc.

PI has expansive research to back-up the claims that the PI Behavioral Assessment is predictive of job performance. Our Validity Vault (an archive of all client research since 1992) boasts over 350 criterion validity studies, in which 94% of the samples studied had significant relationships between the PI Behavioral Assessment Factors and job performance. These studies span 11 different industries and nearly 120 different unique job types (based on O*Net designations). Furthermore, multiple meta-analytic studies have been conducted, comparing the PI Behavioral Assessment Factors to job performance. For example, in a meta-analytic study conducted by Caveon Test Security and the Drasgow Consulting Group in which they analyzed a PI Behavioral Assessment results from a sample of 1,104 employees in 26 job roles, Foster et al. (2015) reported that that the strengths of the relationships mirrored those from FFM meta-analyses conducted by ​Barrick and Mount (1991) and Tett, Jackson, and Rothstein (1991).

The PI Behavioral Assessment predicts just as well as most FFM assessments. What’s even more amazing is that the PI Behavioral Assessment takes 6 minutes on average to complete, while many Five Factor Assessments take 15 to 20 minutes.

Workplace-Specific Traits that Predict vs. Broad Ones that Don’t Always

Speaking of prediction, most FFM research points to one key factor that predicts across most job roles: Conscientiousness (Schmidt, Oh, & Schaffer, 2016). Recent meta-analytic work by Schmidt et al. (2016) shows that Conscientiousness remains the strongest personality-based predictor, while Agreeableness, Extraversion, and Neuroticism each only account for 1% of job performance on average. In other words, of the Five Factors, only Conscientiousness matters, and Openness to Experience–one fifth of the entire model–doesn’t even show up on the prediction radar.

With the PI Behavioral Assessment, we see that each Factor plays a role in prediction, depending on the situation. The scales are meant to be workplace-oriented and not broad, and when we review our Validity Vault and the nearly 5,000 significant correlations between PI Behavioral Assessment scores and job performance measures, we find that each of the four Primary Factors relates to job performance in about 25% of the performance measures (different Factors typically relate to different performance outcomes within a role, or multiple Factors will relate to the same performance outcome). In other words, the BA is an efficient assessment because it doesn’t waste people’s time by asking questions that don’t typically matter for job performance.

The Behavioral Assessment Doesn’t Measure Neuroticism, But That May Be Good

One of the emerging concerns with the FFM is the extent to which it may lead to adverse impact (e.g., unfair hiring) for people with certain psychological conditions. In other words, while not intentional, a selection system that knocks candidates out of a hiring system due to overly high Neuroticism scores could lead to inadvertent adverse impact against people with very real medical or psychological conditions. For example, in the U.S., the Americans with Disabilities Act of 1990 (ADA), enforced by the Equal Employment Opportunity Commission, makes it illegal to discriminate against people on the basis of disability, including psychological disorders.

This concern is valid for FFM assessments that expressly measure Neuroticism. Higher scores on Neuroticism are linked to mood disorders (e.g., depression, bipolar disorder, anxiety disorders). Elevated neuroticism is also correlated with the prevalence of personality disorders (Ormel, ​Riese, & Rosmalen​, 2012; Ormel et al., 2013; Zinbarg et al., 2016). This isn’t to say that higher Neuroticism (as measured as part of a normal personality assessment) always creates conditions of adverse impact; however, the potential is there, and it is worth being aware of it.

With regard to the PI Behavioral Assessment, we can’t say that it is perfectly free of any relation with Neuroticism. Realistically, only one PI Behavioral Assessment Factor has any type of consistent relationship to Neuroticism: Factor C (Patience), and that relationship isn’t strong. The statistics show that there is only about 6% of shared variance between the PI Patience Scale and the FFM Neuroticism scale. So depending on your perspective, it may be a benefit that the Behavioral Assessment doesn’t measure Neuroticism–this may protect you from decisions that could unintentionally lead to adverse impact.

The PI Behavioral Assessment Doesn’t Measure Openness to Experience

Openness to Experience sounds like a very important personality characteristic, but that turns out not to be the case–at least not for predicting job performance. Research shows that Openness to Experience is rarely a predictor of job performance (Griffin & Hesketh, 2004). To date, no research has shown that the PI Behavioral Assessment shares any meaningful relationship to Openness to Experience or any of its facets, such as being inquisitive, being creative, having aesthetic appreciation, or being unconventional.

Some PI experts may at first be confused because some PI Factors appear to assess aspects of Openness to Experience, but it is important to differentiate between outcome behaviors and the characteristics driving them. It is best to think of Openness to Experience as a trait which is largely about being curious for the sake of being curious. The PI Behavioral Assessment can tell us if people are variety seekers (Factor C), risk-takers (Factor D), or willing to consider other people’s ideas (Factor A), but none of these Factors (or their combinations) really get at the drivers of curiosity and inquisitiveness that underlie the FFM Openness to Experience scale.

Summary

The PI Behavioral Assessment is theoretical in nature, and it is a more specific measure of workplace personality characteristics than the Five Factor Model. This isn’t to say that one model is better than the other. In fact, the FFM enjoys numerous advantages because it is so well-researched and readily accepted as a commercially viable approach to personality-based job prediction. However, when it comes to job-related measures, the research (both ours and others’) is clear: narrow measures can be very effective, and all four of the PI Behavioral Assessment scales are related to workplace performance, compared to only a handful of the scales in the FFM. We have hundreds of validation studies and meta-analytic research that show that–even as a six minute assessment–the PI Behavioral Assessment can meet, if not beat, FFM assessments when predicting job performance in most workplace applications. While the PI Behavioral Assessment doesn’t measure Neuroticism or Openness to Experience very well, it could actually be a good thing (with respect to Neuroticism) or really not that important (as with Openness to Experience).

In conclusion, the differences between the FFM and the PI Behavioral Assessment aren’t likely to be the reason your talent decision-making or talent development work succeeds or fails. The real priorities should be making sure you understand the assessment and its applications, that you are trained on best practices, and that you use the assessment the way it is intended to be used. If those practices are followed, either assessment can be an extremely valuable weapon in your talent management arsenal.

References

• Ashton, M. C. (1998). Personality and job performance: the importance of narrow traits. Journal of Organizational Behavior, 19, ​289-303.
• Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. ​Personnel Psychology, 44, ​1-26.
• Costa, P., & McCrae, R. (1992). ​Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI)​. Odessa, FL: Psychological Assessment Resources.
• DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the Big Five. ​Journal of Personality and Social Psychology, 93​(5), 880-896.
• Foster, D., Maynes, D., Miller, J., Chernyshenko, S., Mead, A., & Drasgow, F. (2015). Meta-analysis report of The Predictive Index (Form IV)​. Wellesley Hills, MA: PI Worldwide.
• Foster, D., Maynes, D., Miller, J., Chernyshenko, S., Mead, A., Drasgow, F., Poepsel, M., Barnett, G., & Drazewski, P. (2016). ​The Predictive Index Behavioral Assessment technical manual​ [technical report]. Westwood, MA: The Predictive Index.
• Griffin, B., & Hesketh, B. 2004. Why openness to experience is not a good predictor of job performance. ​International Journal of Selection and Assessment, 12,​ 243-251.
• Hogan, J., & Roberts, B. (1996). Issues and non-issues in the fidelity-bandwidth trade-off. Journal of Organizational Behavior, 17​(6), 627-637.
• Ones, D. S., & Viswesvaran, C. (1996). Bandwidth-fidelity dilemma in personality measurement for personnel selection. ​Journal of Organizational Behavior​, ​17​(6), 609-626.
• Ormel, J., Riese, H., & Rosmalen, J. G. (2012). Interpreting neuroticism scores across the adult life course: Immutable or experience-dependent set points of negative affect? ​Clinical Psychology Review,​ ​32​(1): 71–9.
• Ormel, J., Bastiaansen, A., Riese, H., Bos, E. H., Servaas, M., Ellenbogen, M., Rosmalen, J. G., & Aleman, A. (2013). The biological and psychological basis of neuroticism: Current status and future directions. ​Neuroscience and Biobehavioral Reviews, 37​(1): 59–72.
• Paunonen, S. V. (1998). Hierarchical organization of personality and prediction of behavior. Journal of Personality and Social Psychology, 74​, 538-556.
• Paunonen, S. V. & Nicol, A. A. M. (2001). The personality hierarchy and the prediction of work behaviors. In B. W. Roberts & R. Hogan (Eds.). ​Personality psychology in the workplace (pp. 161-191). Washington, DC: American Psychological Association.
• Schmidt, F. L., Oh, I., & Shaffer, J. A. (2016). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 100 years of research findings. In P. A. Pavlou (Ed.), ​Fox School of Business, Temple University Research Paper Series​. Philadelphia, PA: Temple University.
• Stewart, G. L. (1999). Trait bandwidth and stages of job performance: Assessing differential effects for conscientiousness and its subtraits. ​Journal of Applied Psychology, ​84​(6), 959-968.
• Tett, R. P., Jackson, D. N., & Rothstein, M. (1991). Personality measures as predictors of job performance: A meta-analytic review. ​Personnel Psychology, 44​(4), 703-742.
• Tupes, E. C., & Christal, R. C. (1961). ​Recurrent personality factors based on trait ratings. [technical report]. Lackland Air Force Base, TX: USAF.
• Zinbarg, R. E., Mineka, S., Bobova, L., Craske, M. G., Vrshek-Schallhorn, S., Griffith, J. W., Wolitzky-Taylor, K., Waters, A., Sumner, J. & Anand, D. (2016). Testing a hierarchical model of neuroticism and its cognitive facets: Latent structure and prospective prediction of first onsets of anxiety and unipolar mood disorders during 3 years in late adolescence. Clinical Psychological Science, 4​(5), 805-824.