Accelerating Talent Acquisition Processes with a High-Volume Metacognition Solution

Context

Recruiting and onboarding new employees is a necessary, but pricey, business activity. One of the most effective ways to select the right employees in a timely manner is to incorporate reliable and valid psychometric instruments early in the recruitment process. A Kenyan HR Consultancy recently contacted JVR Psychometrics seeking a cost-effective and time-efficient way to identify top talent from a pool of 500 applicants in the banking and finance industry. The client indicated they were under a time crunch and needed all assessments to be completed online within a 3-week period. To guide them in their shortlisting process, we recommended they use the Metacognitive Assessment Series (MAS) – a combination of short subtests that evaluate various factors of cognitive ability and decision-making confidence.

Why the MAS?

1. Detect unusual testing behaviours, despite unsupervised assessment

• a. Timed subtests: Each item in the MAS subtests is timed. While these time intervals are reasonable enough for candidates to attempt the question, it leaves minimal time for the use of calculators or other “cheating” devices. In the current study, there was a small subset of candidates (n=3) who provided the feedback that the time intervals were too short for them to adequately attempt answering the questions. Since this was not the experience of the majority, the client communicated that these candidates had therefore screened themselves out of the process.

• b. Nonserious attempts are flagged by the MAS system: Answering questions too quickly, or randomly, alerts the practitioner/test administrator that the candidate has not taken the test session seriously, resulting in a nonvalid test session report. In the current study, there was a number of candidates (n=24) whose test-taking behaviours were flagged, resulting in a nonserious alert being sent to the client. The client decided to screen these candidates out, rather than permit them to complete the assessment again.

• c. The MAS system also flags unusual login attempts: To prevent candidates from logging in on multiple devices, or logging in to see a question, then logging off to work out the answers to the question without the time limit, and then log on again to answer, the system locks out the candidate after detecting suspicious login behaviour. The questionnaire can be unlocked by the test administrator if they learn that the candidate had connectivity issues.

2. Resource efficiency

• a. Cost: MAS reports are competitively priced, providing comprehensive information within budget constraints.

• b. Time: The MAS batteries are completed within 20 minutes, saving applicants time and data costs.

• c. User-friendly Outputs: The functionality to generate report extracts, where all applicants for a role could be seen in a single view, rather than with multiple PDF reports, added to the usability of the MAS reports. It was this function in particular that saved the HR consultants significant time, since they could identify their top potential candidates with minimal effort.

3. Insight into decision-making confidence.

• a. Additional value-add: There are few assessments on the market that assess metacognition – or how applicants think about their thinking. By assessing both cognitive ability as well as cognitive confidence, the MAS range informs practitioners on more than just whether candidates can do the job, but also provides insight into how they’re going to do the job.

• b. Applicants’ confidence in their cognitive abilities influences their eventual work performance. Generally, even though underconfident individuals have the necessary cognitive ability, they tend to be overly reliant on colleagues for validation and assistance, avoid risks and stifle innovative ideas, and hesitate to take initiative or make decisions, ultimately negatively influencing their productivity. Inversely, overconfident individuals have an inflated sense of their cognitive ability, resulting in decisions being made without adequate information or consideration of potential risks, resistance to constructive feedback which hinders personal/professional growth, and can create friction within teams, which again has a negative influence on overall productivity.

4. Flexibility.

It follows that the skills required for different job roles vary. With a range of preselected batteries, the MAS range allows practitioners to use the subtests that make the most sense for the role in question.

For this client, it was agreed that the MAS Business Fundamentals Advanced battery (assessing Financial Reasoning, Vocabulary, and Working Memory) was adequate to screen out candidates applying for legal, procurement, and operational business roles. Similarly, applicants for roles with a stronger financial focus, such as auditors, banking and business officers, finance and treasury officers, and research and assurance associates, completed the MAS Numerical Advanced battery (assessing Financial Reasoning, Mental Arithmetic, and Numerical Pattern Recognition).

5. Easy comparison across test-takers.

Even though the subtests may differ between candidates, all MAS reports provide a normed Fundamental Abilities Quotient (FAQ) – an overall estimate of general ability – allowing for a standard score which all candidates can be compared against.

For this client, being able to compare across a standard FAQ score had double value.

• a) They were able to rank and compare all candidates who applied for the same role across both their overall FAQ score as well as their performance on the subtests. Practically, this meant establishing a cutoff FAQ score from which to “screen out” from, and then from the “screened in” group, doing a second level of comparison based on the specific skills/subtests most necessary to the role.

• b) Once the project was completed, the standard FAQ score and confidence ratings could be combined into a talent plot to establish the profile of successful candidates across roles. As the MAS becomes used more regularly by the client, the use of such plots could be broadened to talent audits within specific teams or across the organisation. For demonstrative purposes for this case study, the total candidate pool’s (n=473) decision-making confidence plot is illustrated below. As expected, a large group of applicants demonstrated moderate ability and moderate confidence.

Concluding Comments

While no single assessment should be the sole deciding factor in hiring, incorporating short, quality screeners such as the MAS in the recruitment process can significantly enhance efficiency. Given that cognitive ability and receptiveness to feedback are large determinants of whether the employee will be able to deal with the complexity of the job, it stands to reason that your screener should include a cognitive element. By screening in applicants that have the adequate baseline skills required, you save time in the later stages of hiring, ultimately influencing how quickly your organisation can get back to optimal productivity levels. The MAS provides a reliable, resource-efficient, and insightful solution to streamline recruitment and build a strong, capable workforce.