Our executive assessment validities are industry-leading so you can confidently use the results to make better decisions.
Assessment Validity is the single most important criterion when selecting a talent assessment partner. That’s why we conduct research on the relationship between our executive assessment results and job-related outcomes.
Talent Assessment Validity Research
A meta-analysis published in the Journal of Applied Psychology showed that the effect size of the relationship between Individual Assessment Ratings and Supervisor Ratings of Performance was .30 Source. That’s a strong relationship according to the most recent effect size interpretation guidelines Source.
That meta-analysis included 37 studies.
The individual study effect sizes ranged from .00 (no relationship) to .50 (very strong relationship).
Think about that!
This means that there is a drastic difference in the quality of talent assessment providers!
Using Statistical Analysis to Establish Assessment Validity
A validation study involves statistical analysis of the relationship between numerical predictor variables (e.g., assessment ratings) and quantifiable outcome variables (e.g., job performance ratings). This analysis yields an effect size statistic. This statistic tells you the strength of the relationship between the predictor variables and outcome variables. So, by reviewing assessment validity statistics, you’ll know how well the tool predicts things that you important to you.
PCI’s Talent Assessment Validity Evidence
We are proud of the extensive talent assessment validity work that we’ve done to show the predictive power of our assessments. Our research has consistently shown effect sizes that are toward the top end of the range reported in the meta-analysis. We attribute this to 3 things. First, we have deeply experienced assessors. Second, we have a rigorous and highly structured process. Third, we use tools that measure personality, cognitive ability, and leadership style reliably and accurately. Learn more about our executive leadership assessments here.
Read about some of our validity results in the 5 case studies below.
Our researchers designed a confidential online survey that supervisors completed to rate employees who were hired using the PCI assessment. This approach allows for ratings that are less likely to be politically influenced because they aren’t shared with the organization.
There was a large relationship between our Overall Assessment Rating and Supervisory ratings of Performance and Potential. Our client also provided EEOC data, which showed no differences in the Overall Rating based on gender or race.
Similar to Study #1, there was a strong relationship between the PCI Overall Rating and confidential supervisory ratings. We also found a strong relationship between the overall rating and retention.
In this study, we investigated the validity of our executive assessment process at the competency level. Supervisor ratings from the annual performance review were correlated with PCI Assessor ratings on those same competencies. The large to very large effect sizes are industry-leading.
This assessment validity study had two components.
Predicting Advancement Potential
Results of the first component are shown in the top chart. Our client provided Talent Review Ratings on 206 leaders who were hired using the PCI assessment. The Talent Review Process involved a team of leaders who discussed each employee’s “versatility”. Each employee was assigned a rating on a 5-point scale, with 5 being the most versatile (i.e., the highest potential to advance). We mapped these ratings to the PCI Overall Rating (also a 5-point scale). The Hit Rate of 85% indicates strong predictive validity. We presented this study at the 2013 Society for Industrial Organizational Psychology (SIOP). The study was also included in a meta-analysis on the validity of assessments published in the prestigious Journal of Applied Psychology. Source
Predicting Performance
The results of the second component are shown in the bottom chart. Supervisors rated their direct reports on Korn Ferry competencies during the annual review process. Those ratings were rolled up into a Leadership Skills Rating. Supervisors also assigned an Operating Goals Rating that was based on objective metrics. We designed a customized assessment and report to measure and rate the Korn Ferry competencies. With a subsample of the larger study (n=48) who were rated on these competencies, we found a large relationship between our ratings and supervisor ratings.
In this validity study, we tracked performance ratings over a 3-year period. Year-over-year, leaders who were rated Outstanding by PCI received the highest performance ratings. Those rated Marginal received the lowest. This client also provided data on who was selected for Succession Planning. The difference between those rated Marginal versus Outstanding by PCI is substantial.
Beyond the Numbers
While assessment validity statistics are important, they don’t tell the whole story. We take even more pride in bringing the numbers to life through our custom-written reports. We tell a clear story based on the objective data and weave in insights from the interview to illustrate how the person is likely to behave at work. Your organizational culture and the role requirements serve as additional context, allowing us to further refine the report content to your unique needs.
Beyond Validity
In addition to boosting your ability to predict performance, conducting individual assessments after your internal interviews has a great deal of utility. Unlike information gathered from your interviews, the assessment report can be leveraged far beyond the hiring decision.
The selection report includes custom-written suggestions for onboarding and developing the new hire, based on their individual strengths and opportunities. You can also request that we write a developmental report for the new hire to raise their self-awareness and jump-start their growth. This will have the added benefit of enhancing engagement and ultimately, keeping top talent.