Behind the scenes of Talent Optimization

It looks like magic but it’s actually science.

Built for business

The Predictive Index—or PI for short—has developed a proven methodology that allows businesses to understand what drives their workforce and optimize their talent strategy. 

For 65 years, we’ve supported organizations large and small, helping them to better understand their people. And with more than 350 validity studies under our belt, we know for a fact that what we offer produces tangible business results—including decreased turnover, increased profitability, and improved performance metrics. 

Adherence and compliance

Our assessments and recommended practices were developed in a manner consistent with all widely-accepted standards and guidelines. These standards were designed by groups of industry experts to provide frameworks for determining the proper use of assessments and other selection procedures, as well as prevent discriminatory employment practices.

PI’s assessments and recommended practices comply with:

• Uniform Guidelines for the Development and Use of Personnel Selection Procedures (1978)
• American Psychological Association (APA)
• Society for Industrial and Organizational Psychology (SIOP)
• International Test Commission (ITC)

In addition, the PI Behavioral Assessment™ is certified under the European Federation of Psychologists’ Association’s (EFPA) Test Review Model. The PI Cognitive Assessment™ received the same certification in 2020.

A brief history of PI

In 1942, Arnold Daniels volunteered for the U.S. Army Air Corps. Shipped off to England, the flight navigator and his team soon logged more than 30 missions—all without a single combat casualty. (For reference, that’s an extraordinary track record.)

When commanders noted the team’s stellar performance, they sent a psychologist to work with Daniels—to study what made the team so successful.

This was Daniels’ first introduction to psychometric testing—and what would become a lifelong passion: solving business problems by better understanding individuals.

In 1955, he released the first iteration of the PI Behavioral Assessment. Since then the behavioral assessment has been republished multiple times to keep abreast with changes in modern psychometric practices and workforce applications.

When PI’s current owners, Mike Zani and Daniel Muzquiz, bought the company in 2014, they realized behavioral assessments have considerable predictive power when it comes to employee performance. Behavioral assessments have exceptional value for coaching, development, and change management, but the hiring process must consider cognitive ability as well. (Cognitive ability is the No. 1 predictor of on-the-job success.) Shortly thereafter, we introduced the PI Cognitive Assessment™ to our suite of tools.

Introducing talent optimization

In 2015, PI’s senior leaders noticed a concerning problem: Clients were making better hires, but their business results weren’t noticeably better overall. If these businesses had access to critical people data, why weren’t they seeing results throughout the organization?

When the PI leadership team started to dissect the issue, they noticed a common trend: In January, all these companies created business strategies to achieve their goals. However, in December, those desired results often weren’t realized. Something was happening between January and December—and it boiled down to unsolved people problems.

Following this realization, the team built a framework to help senior leaders align their business strategy with their people strategy—a discipline called “talent optimization.” Talent optimization comprises four aptitudes:

1. Design (designing talent strategy)
2. Hire (hiring the right people)
3. Inspire (inspiring performance)
4. Diagnose (diagnosing employee experience)

Talent optimization launched in early 2019 to great acclaim—and the PI suite of software, services, and workshops grew to support clients in executing all four aptitudes or talent optimization.

Since 2019. PI has surveyed 600+ executives for its annual State of Talent Optimization Report. That research has consistently demonstrated the business value of implementing talent optimization practices. 

How the PI Behavioral Assessment works

You can’t always see what drives and motivates someone from the surface. Sometimes the way an employee behaves is a result of how they think they’re supposed to act at work. But that doesn’t mean they’re naturally wired that way!

The PI Behavioral Assessment was created to evaluate people’s behavior at work—both their inherent behavioral drives and their perceptions of what behaviors are expected of them. This helps employers understand what naturally drives and motivates their employees, as well as where they may be stretching outside their comfort zone to perform their current role.

The PI Behavioral Assessment is untimed, takes approximately six minutes to complete, and employs a free-choice (as opposed to forced-choice) format. 

The assessment is composed of two questions; each comes with a list of 86 adjectives. (The same adjectives are used for both questions). The first question asks respondents to choose adjectives that describe how they think others expect them to act. The second question asks respondents to choose adjectives that describe who they really are. Each adjective is associated with one of the four behavioral factors below.

Understanding the four factors

PI’s behavioral assessment measures four behavioral factors (also known as “drives”) in people. Each factor is denoted by a letter. 

Dominance is the degree to which individuals seek to control their environment. Individuals who score high on this dimension tend to be independent, assertive, and self-confident. Individuals who score low on this dimension tend to be agreeable, cooperative, and accommodating.

Extraversion is the degree to which individuals seek social interaction with other people. Individuals who score high on this dimension tend to be outgoing, persuasive, and socially poised. Individuals who score low on this dimension tend to be serious, introspective, and task-oriented.

Patience is the degree to which individuals seek consistency and stability in their environment. Individuals who score high on this dimension tend to be patient, consistent, and deliberate. Individuals who score low on this dimension tend to be fast-paced, urgent, and intense.

Formality is the degree to which individuals seek to conform to formal rules and structure. Individuals who score high on this dimension tend to be organized, precise, and self-disciplined. Individuals who score low on this dimension tend to be informal, casual, and uninhibited.

Factor placement and combinations

Here’s an example of what an individual’s behavioral pattern could look like:

The black triangle at the center of the graph marks the midpoint. Factors to the right of the midpoint are considered high, and factors to the left are considered low. The higher a factor is on the scale, the more pronounced that behavior is, and the more likely it is that the person will apply that behavior to situations at work. For example, if someone’s A drive is very far to the right of the midpoint, they’d be extremely dominant. Consider it like turning up the volume on a radio.

What’s interesting to note is that you’ll never see all factors fall on one side of the midpoint. 

There’s a balance of drives on each side of the midpoint. Think of the midpoint as the fulcrum of a scale. The four factors even each other out so that both the high and low sides are equally balanced. -The amount of each side must weigh the same for the sides to be level. We score the assessment this way in order to show what behaviors a person is most likely to respond with, as opposed to comparing the strengths of their drives with another person.

How results are measured

The Predictive Index Behavioral Assessment was created through a global sample of more than 10,000 people. Data collected from this sample was used to identify a normal range of behavioral factor levels for the adult working population (i.e., what’s high, average, and low).

Typically, benchmarked samples only allow you to see how someone falls on a spectrum compared to others in the working population. PI Behavioral Assessment results are standardized using this benchmarking methodology. However, our behavioral assessment takes an additional step: It provides a lens into people’s distinct, within-person drives.

In other words, it measures each behavioral drive compared to the other behavioral drives within that unique individual. So if an individual’s behavioral results indicate a high degree of extraversion, that’s compared to where their dominance, patience, and formality factors fall. It’s not a measure of how extraverted they are compared to the rest of the population.

This within-person insight is accomplished through a measurement model that uses normative regression models to create standardized scores conditional on a respondent’s response level on the free-choice adjective checklist. The standardized scores ensure that results can be compared between candidates when making hiring decisions.

Finding the right behavioral fit

Behavioral assessment results aren’t meant to be the primary decision-making factor in the hiring process. The results are simply one data point of many to be considered (behavioral fit, cognitive fit, experience, references, etc.).

When candidates take the PI Behavioral Assessment as part of the hiring process, their behavioral pattern is compared to the Job Target for the given role (see the PI Job Assessment™ section below for more information on Job Targets). Based on how well the candidate’s behavioral drives match the Job Target, the software assigns them a Match Score between 1-10 (1 being not a match, 10 being a strong match). This score isn’t meant to exclude applicants but rather rank order them based on behavioral fit for the role.

If there’s an applicant you really like who doesn’t have a strong Match Score, interview them anyway. The PI software generates Interview Guides, which can help you probe into any behavioral gaps between the person and the Job Target. For example, if a role requires following a strict process and a candidate’s behavioral pattern shows they have low formality, you might say the following during an interview: “Describe an experience when you needed to work within existing rules and procedures.” Their response will give you a better understanding of how they work within established rules and processes.

As the person responsible for hiring, you ultimately make the decision on who’s the best choice. The PI Behavioral Assessment provides one data point that can help you make a more informed, objective decision about who to hire.

How the PI Cognitive Assessment works

The PI Cognitive Assessment is a timed, 12-minute assessment designed to measure cognitive ability. This is an individual’s capacity to learn quickly, grasp new concepts, adapt to changing circumstances, and understand complexity in the workplace. 

Cognitive ability is empirically the best standalone predictor of training success and job performance. Conclusive research finds that when measured, specific cognitive-related abilities are so highly correlated that they represent a single underlying factor known as ‘g’–or general cognitive ability. This means that when a person scores highly on a ‘g’ measure like this one, they’re likely to be strong in the area of specific cognitive-related abilities, demonstrating the capabilities of performing at higher levels for workplace tasks. 

The PI Cognitive Assessment consists of 50 multiple-choice questions across three categories—verbal, numerical, and abstract reasoning—and nine subcategories. The assessment is divided into 10 pages, each of which contains five questions.

Candidates are asked to answer as many questions correctly as possible within the allotted time. They may leave questions blank and move on to the following pages. Likewise, they may come back and revise any answers they’ve selected or left blank.

At the end of the allotted time, the candidate’s results are converted into a score based on the total number of correct answers. This score is then scaled and presented to the admin as a number between 100 and 450.

The test-engine builds each cognitive assessment so that candidates experience a unique set of questions, minimizing the risk of cheating. This also creates the possibility for a second assessment. Some companies may choose to administer the cognitive assessment a second time if the test taker wasn’t given a fair opportunity the first time around (due to illness, internet outage, etc.). Additionally, some companies choose to implement a standard testing policy in which they administer the cognitive assessment twice: the first time remotely, and the second time in person. 

Finding the right cognitive fit

There are no “good” or “bad” scores for the cognitive assessment. Rather, the goal is to determine if a candidate meets the cognitive requirements of a given role. We don’t recommend setting one general cognitive target for all roles, because that could result in candidates being excluded even though they have the cognitive ability to perform the role they applied for. A good fit shows an increased likelihood of success through training and job performance, while weaker fit may indicate difficulties in getting up to speed quickly, catching on, or figuring things out. 

It’s best to start by identifying a recommended target score based on the role and work environment. The level of job complexity and organizational factors help shape the cognitive demands of the job and these are what the software considers when identifying a recommended target score. For example, if an organization works in an industry that’s rapidly changing—such as technology—cognitive requirements may be higher to account for the constantly changing needs of the role. The PI Job Assessment helps hiring managers set a defensible, appropriate Job Target for the demands of a particular job, rather than expecting them to simply hire whichever candidate has the highest cognitive score.

How the PI Job Assessment works

Job descriptions may tell us a lot about tasks and hard skills, but they tell us little about the kind of person who will succeed best in that job. Enter the PI Job Assessment, which helps hiring managers determine the behavioral traits and cognitive ability needed for success in the role.

Behavioral traits: The PI Job Assessment consists of 90 task statements, each corresponding to one of the primary factors measured by the PI Behavioral Assessment. For example, task statements related to highly-structured work are associated with factor D (Formality). 

Like other job analysis surveys, the PI Job Assessment is intended to be completed by people familiar with the job role (e.g., manager, high performers currently in the role, stakeholders served by the role). Each rater’s response creates a Job Pattern, and these Job Patterns are averaged to create the Job Target that can be used for candidate selection. 

Cognitive ability: The PI Job Assessment includes a cognitive section that informs target scores for the PI Cognitive Assessment. 

How PI Team Discovery works

PI Team Discovery helps both leaders and team members visualize the behavioral makeup of their teams. Having seen their behavioral data plotted on a full team chart, they can then compare that team makeup with the agreed-upon strategy. By showing where they are aligned – and where their teams may have gaps – Team Discovery informs decisions vital to talent strategy. Those include:

1. Strategy identification: Which strategies does the company need to focus on for the business to be successful? 
2. Strategic alignment: How well do people, culture, and jobs align to the demands of the strategies? 
3. Organizational readiness: Is the company ready to pursue its strategies? What gaps need to be addressed or monitored?

From a list of 20 strategic statements, team members select the ones that best reflect the strategies they feel their team is pursuing over the next 12 months. 

The 20 statements are the strategic behaviors that best differentiate between the four strategic classes (Exploring, Producing, Stabilizing, and Cultivating) while remaining broad enough to apply to most businesses. These strategic classes are based on the work of Miles and Snow, whose strategy model has been widely used in management and organizational research for decades. The mechanism for aligning talent strategies with the four classes of business strategies described above was inspired by the Competing Values Framework—a widely applied model for describing and diagnosing organizational culture as it relates to business strategy and workplace behavior. 

The 20 statements were field tested with more than 600 people in two separate studies, and concurrent construct validation was conducted with the Organizational Culture Assessment Instrument (OCAI). Pilot testing was also conducted with executive teams from multiple organizations working across a variety of industries. Assessment responses are aggregated and mapped to the strategy types or “quadrants.” When used in conjunction with the PI Behavioral Assessment, senior leaders can view alignment and misalignment between their team’s behavioral emphasis and their chosen strategic priorities. The 2021 State of Talent Optimization Report identified having the right executive team in place to execute the business strategy as one of the talent optimization behaviors that translated to positive business outcomes. 

How the PI Employee Experience Survey works

The PI Employee Experience Survey™ is a theory-based, self-report measure of employee engagement—which is defined as an employee’s emotional commitment to their organization and its goals.

The survey provides employees a chance to provide candid, confidential feedback across four categories strongly related to engagement:

1. Job fit
2. Relationship with manager
3. Interactions with people on their team
4. Organizational experience

A research effort involving approximately 3,000 responses—collected over the course of two weeks in February 2019—was conducted to identify the most effective survey items. The core PI Employee Experience survey is composed of 50 Likert-scaled items covering the four categories above plus an additional category of general engagement questions. Additionally, there’s a section for open-ended responses so employees can offer further insight into their experience.

Engagement as a construct is considered an outcome measure—and is not directly actionable. In order to provide direction for leaders and managers to effectively plan their actions, they need to understand which items are most related to engagement. The reports generated by the PI Employee Experience Survey offer actionable next steps, listed in order of importance. Engagement items are ranked based on impact, which is determined with correlational analysis. For example, if the statement “The people I work with have the right skills to produce high quality work” received a lower score than “I believe my organization has an outstanding future,” the latter could still be marked as a higher priority action item if it had a stronger impact on engagement.

In addition, if a team scores lower on an engagement item than the overall organization—or if the organization scores lower than the benchmark score—those items will be weighted more strongly as areas for improvement. 

Assessments you can rely on.

The Predictive Index’s science team and our solutions have continuously evolved since 1955. We’re constantly running validity studies, making sure our assessments are free of bias, and updating our assessments to maintain their psychometric properties. The Predictive Index employs I/O psychologists, data scientists, and psychometricians to ensure the science behind PI is credible, accurate, and—most importantly—useful for our clients.

Validity

PI treats validation as an ongoing process. Our psychometricians are continuously adding to PI’s portfolio of validation research, which currently includes more than 350 validity studies, spanning many different industries, regions, and job roles. We also conduct studies for construct validation, content validation, and use case validation.

Validation begins in development with careful content validation, field testing, form construction, and construct validation. But this work continues even when the instruments are in the field. PI researchers validate new use cases for assessments, monitor the instruments’ performance, and practice continuous improvements to ensure that the assessments continue to yield valuable, interpretable, and actionable insights for clients.

Reliability

An assessment cannot be valid if it’s not reliable and accurate. PI’s psychometricians conduct reliability studies to report different facets of accuracy for the assessments. These include the accuracy of scores (internal consistency), stability over time (test-retest reliability), and reliability of the psychological constructs that are measured (factor reliability). 

These studies help to quantify measurement error of the assessment to ensure they’re reliable enough to support their intended use cases. Reliability studies also help provide guidance to PI clients on how to effectively use the results of PI assessments.

Fairness

Fairness has emerged as its own domain of psychometric practice. An assessment’s validity depends on the instrument being fair and unbiased for its intended populations. Fairness encompasses many facets of the assessment system. 

Fairness may refer to a lack of score bias, which PI documents in differential item functioning (DIF) studies. It can also refer to the access, administration, interpretation, and impact of the assessment. PI’s psyshometricians provide guidance to clients on how to use PI assessments in a way that’s fair to all respondents. They also work closely with PI’s product developers and consultants to make fairness a consideration in every part of a client’s PI experience.