Reliability and Validity of the PI Cognitive Assessment

The PI Cognitive Assessment was developed in 2010 in response to client demand for a short cognitive ability assessment which could provide a valid predictor for job performance.

The development work was conducted in accordance with Principles for the Validation and Use of Personnel Selection Procedures (SIOP, 2003), and the Standards for Educational and Psychological Testing.

Reliability Evidence 

Reliability refers to the precision of the scores and their consistency across testing instances. Reliability can be established in various ways such as test-retest reliability and internal consistency. Strong test-retest reliability indicates that participants’ scores remain relatively stable over time. Internal consistency indicates that the questions on an assessment consistently capture the same construct. 

Numerous studies have examined the internal consistency and test-retest reliability of the PI Cognitive Assessment, all yielding results indicating at least acceptable levels of reliability for the assessment’s intended use.

Validity Evidence 

Validity refers to evidence supporting interpretations and use cases for an assessment. Specifically, strong validity indicates that an assessment adequately captures the construct it’s intended to measure or adequately predicts the outcomes it is intended to predict. The cognitive assessment has demonstrated correlations with performance ranging from .29 to .44. Fairness can mean different things in different contexts, and much of this responsibility falls on PI as the assessment publisher. We provide translations and extended time formats to support access to the construct for the general population. We conduct statistical analysis of all questions on the PI Cognitive Assessment to ensure that there is no measurement bias between different groups of people. 

Treatment of participants and use of scores is also the responsibility of the client. We support our clients with materials to ensure that they use the PI Cognitive Assessment in a fair, defensible manner. PI provides clients with an Administrator’s Guide to help ensure that clients treat participants fairly in the assessment process, and PI provides a Job Assessment tool and clear, well-designed reports to help ensure that the scores are interpreted and used in a defensible way. 

Norming

The PI Cognitive Assessment is criterion-referenced rather than norm-referenced. In the simplest possible terms, this means that CA scores are based upon one’s own achievement on the assessment (i.e., the number of questions answered correctly) as opposed to the relationship of one’s scores as compared with those of another participant or population of participants.

Global Norming

The PI Cognitive Assessment is designed to be used with a global workforce, so the content is designed to be regionally and culturally neutral. For example, verbal items do not use idioms and abstract reasoning items do not use latin characters for patterns or rotations. Much of the work around this happens during the item development phase before the assessment goes to the field. The items go through both a cultural review and a translation process. Information about both of these processes is provided in the Technical Manual, and the instructions for those reviewers are provided in the appendices.