Validated Talent Assessments
Validated Talent Assessments Are Hard to Come By
A meta-analysis published in the Journal of Applied Psychology showed that the effect size (i.e., strength of relationship) between Individual Assessment ratings and Supervisor ratings of performance was .30. That’s a strong relationship according to the most recent effect size interpretation guidelines. The meta-analysis included 37 studies, representing data from a variety of different executive assessment firms. The individual study effect sizes ranged from .00 (no relationship) to .50 (a very large relationship). This means that some executive assessments do NOT predict job-related outcomes. Ours do, with effect sizes at the high end of the study range.
Our Individual Assessment Results
Unlike most executive assessment firms, we conduct validation studies to investigate how well our results predict job performance and advancement potential.
A validation study involves a statistical analysis of the relationship between numerical predictor variables (e.g., assessment ratings) and quantifiable outcome variables (e.g., job performance ratings). The analysis yields an effect size statistic, which indicates the strength of the relationship. So, by reviewing validity statistics, you’ll know how well a firm’s executive assessments predict performance, turnover, etc.
Because science is at the core of what we do, we love it when our clients ask us to conduct a validation study. We’ve been fortunate to have clients who value data and give us access to their metrics.
Click on the tabs below to view short summaries of 5 of our validation studies.
Sample: 253 professionals through senior executives who participated in PCI’s Executive Assessment as part of the hiring process.
Method: We designed a confidential survey that supervisors completed to rate direct reports who did the assessment between 1 and 5 years prior. This allowed for ratings that were less politically influenced because they weren’t shared with the organization.
Key Results:
- Very Large Effect Size (.48) – relationship between PCI Ratings and Supervisor Ratings of Overall Performance
- Large Effect Size (.35) – relationship between PCI Ratings and Supervisor Ratings of Advancement Potential
- EEO Data for North American participants showed no differences in PCI Overall Rating by Gender or Race
Sample: 52 global senior leaders hired with PCI’s EA process
Outcome Variables: Supervisor Ratings
Key Results: This client had a Global Competency Model that was rated by PCI’s EA process and by supervisors during the annual review process. Some of the strongest correlations reflected the company culture.
Sample: 206 leaders who were hired using PCI’s Executive Assessment and later evaluated by the company’s Talent Review Committee.
Key Results:
- 85% Hit Rate – Talent Review Versatility Rating (1 to 5) and Overall PCI Rating (1 to 3) Potential Rating
Acknowledgments: This study was peer reviewed by SIOP, accepted, and presented at the 2013 SIOP Conference. It was also included in a meta-analysis on the validity of assessments published in the prestigious, peer-reviewed Journal of Applied Psychology.
Methodology: Senior leaders who participated in the PCI EA process pre-hire were tracked over a 3-year period. Outcome variables included a wide range of performance metrics and Talent Review ratings for Succession Planning.
Sample: 77 Leaders hired with the PCI EA process
Outcome Variables: Supervisor ratings on a confidential survey and turnover statistics.
Key Results:
Percent Still Employed
Beyond the Numbers
While assessment validity statistics are important, they don’t tell the whole story. We take even more pride in bringing the numbers to life through our custom-written reports. We tell a clear story based on the objective data and weave in insights from the interview to illustrate how the person is likely to behave at work. Your organizational culture and the role requirements serve as additional context, allowing us to further refine the report content to your unique needs.
Beyond Validity
In addition to boosting your ability to predict performance, conducting individual assessments after your internal interviews has a great deal of utility. Unlike information gathered from your interviews, the assessment report can be leveraged far beyond the hiring decision.
The selection report includes custom-written suggestions for onboarding and developing the new hire, based on their individual strengths and opportunities. You can also request that we write a developmental report for the new hire to raise their self-awareness and jump-start their growth. This will have the added benefit of enhancing engagement and ultimately, keeping top talent.
To see a sample report, message Dr. Keith Francoeur on LinkedIn.