Rock Paper Scissors
One NFL expert gave the Eagles an “A+” for their 2021 draft selections and ranked them as 4th best out of the 32 teams. As someone who roots for the Eagles, I hope that their selections are as good as forecast. I had my doubts as to how they would do, however, when the Philadelphia Inquirer reported that prior to the draft, first year Eagles’ coach Nick Sirianni and his assistants challenged draft prospects in impromptu games to measure the level of their competitiveness. Coach Sirianni revealed that he played Rock, Paper, Scissors and assessed the level of competitiveness by whether the prospect talked trash in response to Coach Sirianni’s talking trash.
Photo Credit: Michael Smith
Had the Eagles come up with a new way to assess talent previously overlooked?
And, more relevantly for HR, how would you have reacted if you learned that the CEO or a top manager in your organization intended to utilize such a technique for assessing an important characteristic of applicants? Perhaps you are aware of instances where a manager has drawn a strong conclusion from suspect data.
Given that HR oversees the selection processes their organization uses, it is crucial that the assessments being made and used to make decisions are valid ones.
Given that HR oversees the selection processes their organization uses, it is crucial that the assessments being made and used to make decisions are valid ones. The problem is that we are not good at assessing our own capabilities of making accurate judgments. Each of us, including you and me, are prone to having overconfidence bias. Even Nobel Laurette Daniel Kahnemann, the Israeli-American psychologist who spent a lifetime studying judgment and decision-making, notes, “I’ve been studying intuition for 45 years, and I’m no better than when I started. I make extreme predictions. I’m overconfident. I fall for every one of the biases.”
Organizations have a heavy reliance on the interview to select employees. It might very well have been the employment interview conducted by the Eagles where prospects played Rock, Paper, Scissors, or answered Jeopardy questions unknowingly being assessed for their competitive juices.
When constructed and administered properly, the interview as a selection method has demonstrated moderately high validity with job performance. It can be particularly effective when combined with other assessments, particularly if those assessments are based on an identification of critical job dimensions. HR must help organizations use the interview properly.
When constructed and administered properly, the interview as a selection method has demonstrated moderately high validity with job performance...HR must help organizations use the interview properly.
During my career as HR manager, I would venture to say I have conducted hundreds of interviews and facilitated discussions with managers and other decision-makers focused on candidate evaluation. Broadening the team of interviewers provided a check against individual biases. Exclusively using job-related questions was also key.
I find wisdom in the 2016 resignation letter written by former Philadelphia 76ers General Manager Sam Hinkie. In a section titled, The Importance of intellectual humility, he writes:
A way to prop up this kind of humility is to keep score. Use a decision journal. Write in your own words what you think will happen and why before a decision. Refer back to it later. See if you were right, and for the right reasons (think Bill Belichick’s famous 4th down decision against Indianapolis in 2009 which summarizes to: good decision, didn’t work). Reading your own past reasoning in your own words in your own handwriting time after time causes the tides of humility to gather at your feet. I’m often in waist-deep water here.
How would the Eagles decision-making process fare if they kept score as Sam Hinkie recommends?
Each year, the NFL holds a Combine during which prospects are measured on numerous dimensions, including speed and agility tests, physical strength, physical measurements, cognitive tests, interviews, drug screening, proneness to injury, etc. Because sports performance is so well-defined and measured, it provides one of best environments for conducting all sorts of analyses. Just consider Michael Lewis’ Moneyball if you want an example of how a vast amount of data can be used to detect the predictive validity of all sorts of factors with later major league baseball performance. In a 2011 study, researchers compared the NFL Combine assessments with collegiate performance and found in favor of collegiate performance. This doesn’t mean that these tests were incapable of predicting later performance in the NFL. They just weren’t as good at predicting it as a player’s collegiate performance was.
If the Eagles want to assess the level of competitiveness of their candidate pool, they shouldn’t be playing Rock, Paper, Scissors and thinking they have made a valid assessment. They should look at the player’s competitive history. Or, perhaps they should have the players take one of the well-developed, scientifically valid personality assessments of competitiveness.
In 2019, I attended a session on oddball interview questions at the annual Society for Industrial and Organizational Psychology conference. The panelists were asked to comment on the usefulness and validity of questions such as, “Would you rather fight one horse-sized duck or 20 duck-sized horses?”
One panelist said that the usefulness of these types of interview questions were for the candidates not the organziations because it should help them conclude that this was not an organization or manager they should want to work for. Another panelist who had extensive experience working as an expert witness in EEOC cases said that if he learned that any of his clients were using such questions, they should not call him to represent them in any discrimination lawsuit. They should simply get out their checkbook and settle.
Let’s not try and decide whether the candidate who chooses to fight one horse-sized duck is more competitive than the one who wants to fight 20 duck-sized horses. And, let’s keep score.
Go Eagles!
Gerry Brandon, PhD, SPHR, SHRM-SCP is the Director of the Graduate Programs in Human Resource Development at Villanova and an Associate Professor. Connect with him on LinkedIn and learn more about him here!
