A CykoMetrix Spotlight Production

Every week, the Spotlight shines on an amazing professional with a story to tell and lessons to teach. Welcome to the CykoMetrix Spotlight.

The following is an adapted transcript of the exchange between Sylvain Rochon, CMO at CykoMetrix as host, and Andrea Wright, Solutions Marketing Manager at Altus Assessments. (See altusassessments.com)

Sylvain Rochon: Welcome to psychometric Spotlight. My name is Sylvain Rochon, I’m the chief marketing officer at CykoMetrix, a leading edge combinatorial psychometric and human data analytics company that brings the employee assessment industry to the cloud with instant assessments in depth analysis, trait measurements and team-based reporting features that simplify informed decision making around recruiting, training and managing today’s modern workplace. Today, we have yet again another psychometric spotlight and my guest is Andrea Wright. She is a solutions marketing manager at Altus Assessments. A wonderful young woman making her way into the assessment industry. I’m really happy to meet with her.

She leads the research to bring admissions assessment and program management solutions into new program areas at Altus. After a career in secondary school teaching – that’s where I started – she worked for seven years in graduate recruitment and admissions at Smith School of Business, one of Canada’s top business schools located at Queen’s University. After completing her master’s degree in education at Queens, she joined Altus to support their growth and advocacy for more holistic admissions processes in higher education.

Andrea lives with her son in Kingston. Enjoys being active, reading, baking and recently became a new plant parent. That is exciting especially since its spring. Welcome to the Spotlight, Andrea

Andrea Wright: Thanks Sylvain.  It’s great to be here.

Sylvain: I’m really happy about the topic we chose together.  It’s been a while since you last talked to me, you went to a conference. You’re very busy at Altus doing what you do best. We chose a topic that I suspect, is really important for Altus, but I’m going to let you speak to that. The subject is, using the data, psychometric data, to help people discover themselves. So how does that work? How does Altus and you, and your special sauce manage to help people discover themselves?

Andrea: A really good question, Sylvain. Altus takes the approach of psychometric assessments, but takes at that level of understanding students better from the perspective of a program who is considering them for admissions. Traditionally, you know, our largest market that we do this in is medical education and Health Sciences. So, undergrad, medical education, veterinarian, dentistry, nursing, for example. These psychometric assessments help programs to, you know, get a clear picture or add more pixels to the picture of what an applicant brings to the table.

So, you know, traditionally they look at GPA. They look at their grades to get into a program. But the question is, we know that the student can pass these courses. Academically, they’ll be fine, but will they be able to be an effective member of the team? Will they have the self-awareness to know when they’re taking on too much or when they have to put up their hand and say, “I need help.” Will they have the motivation to complete a competitive academic program?

So, the psychometric assessments uncover those skill sets coming into the program so that programs are making sure that they’re accepting people that are going to be successful in the program and successful as professionals once they’re done with their degree.

Sylvain: Cool. What are the different assessments that you actively use? Vocational assessments? Career assessment? What are the assessments that you most commonly use?

Andrea: So, we have three. Our flagship assessment is an open response summative situational judgment test called the Casper Test, which was created in 2014 out of McMaster University in Hamilton. So, like I said, it’s open response, it’s scenario-based and it presents the applicant with a number of personal or professional real-life scenarios or dilemmas. It encourages them to illicit genuine responses to how they would respond in that situation and that sort of gives insight into their overall skill sets of collaboration, ethics, empathy, motivation, resilience.

So, this test has been written by over 600 thousand applicants since 2014, across a multitude of program areas. We also have Snapshot, which is a one-way asynchronous interview tool, a video interview tool, where applicants are asked three probing questions to uncover their communication skills, their motivation for their profession, and their level of self-awareness and they record the answers.  It gives the program some insight into who the applicant is presenting as a professional. Really, when you see someone talking about what they want to do, it really uncovers that motivation and that desire.

We also have a fit alignment assessment called Duet that’s used in our Health Sciences verticals. What that does is the program fills out an assessment and creates a program profile. So, alumni, faculty, administrators fill it out and then a program profile is generated. Then, every student that applies to the program, also completes this 15-minute assessment. Then, there is a fit report that comes out that shows you the students who aligned most with the values of your program. Be it, pedagogical values, academic values, that kind of thing. So, these are the three main assessments that we use in all this.

Sylvain:  I am unsure, but I suspect these assessments are applied to students that are reasonably old, maybe they are in high school or even middle school? What are the age ranges where you’re actually starting to measure them?

Andrea: So, the youngest age range is actually students coming out of their last year of high school going into undergraduate programs and the assessments are tailored to someone with that level of life experience. Then the next group would be graduate-level students. So, individuals coming out of an undergraduate degree, who are going into a graduate program. And again, the scenarios are a little more different, a little more mature because of course, they’ve had a lot more life experience, different things and also tailoring it even more so, for say, things like residency programs for doctors, so they’re a little, a little further along in their professional journey.

Sylvain: How do you account… we talked with people from all over the world, and it’s come up a few times that being able to use assessments that are self discovery or self assessments… how do you account for cultural differences? How do different people taking the assessments interpret the questions? How does Casper and the other assessments figure that out?

Andrea: That’s a really great question. So, we have a research team that is filled with a number of psychometricians and people with PhDs in this area who work very closely with our content team to ensure that we are always aware and keeping our eye out for things like power dynamics. Different ways that different cultures would interpret certain things. We’re making sure it’s much more generic and not very culture specific. We are also in multiple countries.  We are in Canada, the US, UK, Australia, and others. So, we tailor our content for that audience.

So, there would be different wording, there would be different turns of phrases and different experiences in those different cultures, so we’re very aware of it. We’re also in Canada. Something I’m particularly proud of what we’re doing is, we’re working with an organization called Kiikew which is an indigenous organization that is helping us in partnership to inform our content to align with indigenous ways of knowing so that we’re really understanding the unique experience of multiple groups within Canadian Society. We’re always monitoring the quality of each test that we put out in each year. We’re doing more research with partner schools and organizations

Sylvain: Excellent. The other question would be, do you have any metrics that shows the success of going through this process of discovery? The goal is, in my understanding, to help the student better place themselves in colleges, universities and eventually in their career. So, you have an idea of how beneficial this approach is versus, nothing. Or use another process perhaps?

Andrea: Yeah. We do a number of predictive validity research and very strong metrics to show that an applicant success on this type of assessment will predict likely their success in the program and as a professional. So that’s a big metric that we use from that piece. Another thing that we do is we also have evidence to show using these assessments helps to mitigate demographic differences in admissions. So the Casper test, when we work with programs on the weighting of GPA and Casper and other non-cognitive measures, the results is that individuals from under-represented minorities or different lower socioeconomic groups actually perform higher on Casper than their counterparts because they’re bringing to the table some more unique life experiences and things work experience, let’s say.

That really helps to show that they have more to offer than just, “I got good grades in high school.” So, these are some of the snapshot of the metrics we use to show how these benefit an applicant, for their chances to get into the program and the likelihood that they’ll be a successful professional because they possess the skills that are needed to be professional, serving communities.

Sylvain: I think that’s an amazing approach. Of course, we’re pro-data. That’s our whole game. We’re a bunch of engineers and technologists here. You mentioned something very important that we hear a lot about, which is, how do you avoid bias? Because the job market is full of bias but psychometric data, if it’s constructed properly or the assessments themselves, are the attempt  to avoid the emergence of any kind of bias because the data just speaks for itself in the responses. So, can you speak a bit to how they approach health with diversity in the workplace? Bias avoidance? Have you heard any responses from your clients and other organizations to that effect?

Andrea: Well, what is great about Casper is that the people that rated are actually human raters, so right away you’re saying, “oh okay, what does that mean for bias?” All of them are highly trained in implicit bias and they’re also trained to not look at things like spelling and grammar or, you know, colloquial phrases, that kind of thing. They’re assessing the quality of the response in the decision-making that’s going into it. Nine different people evaluate one applicant. So, you’re getting nine different impressions and the average score is calculated into as a zed-score, or z-score, and that is what is sent to programs.

So, the programs are receiving a number. They’re not seeing the applicant because they’re not that in that regards with Casper. So that helps very much. But and we do a lot of QA, QI of our raters to ensure, to monitor that kind of implicit bias. We recognize that we will never be able to eliminate that. We are humans, this is what we do, right? But being actively aware that it is present in decision-making and selection is the first step. Also monitoring that very closely and doing continuous research and case studies helps us to really monitor that and to manage it as best as we can and look like that partnership that I spoke to you earlier, that’s just one step.

I know my colleagues in research could speak to this much more effectively than I could, but that’s just sort of a little snapshot of what we try to do.

Sylvain: Oh, that’s excellent. We have to recognize that we are inherently biased-prone. It’s just human nature. We try not to, those of us who are aware of that, we try, but we are biased.  But you’ve put some mechanisms in place to minimize it and to make sure that there’s none down the line there, the bias becomes basically, well, it’s not an issue anymore because you provide just a number and numbers are agnostic. They don’t have a religion. So that’s awesome. I think that’s amazing what you’re doing.

I guess I have a final question.  We’re talking about schools and entry, but you also work directly with private sector, for promotions, for example, or other areas that are not specifically set to help people place from school to workplace. You help in related areas like promotions, from position to position within the private company, for example. Are there any specific industry that you’re focusing on?

Andrea: At this point, we only work within education. So, with admissions, program experience software and analytics software. So, we haven’t, moved out into any kind of additional sector because the mission of the company is to a world served by exceptional professionals. You know our mission, the purpose-driven piece, that starts in the training and the education piece. So right now, that is our focus.


Sylvain: Okay. It’s perfectly fine. You found a niche.

Andrea: There’s so much to do just within that space, right?

Sylvain: Yeah. There’s a lot of variations and different things you can do just in that space and especially being a company that’s broader than just Canada, the different approaches and education systems, and I mean there’s a lot to do absolutely. Well thank you so much Andrea for taking part in the Spotlight, for being in the Spotlight yourself and explaining to us what Altus assessment does and the excellent work that they do in fact and your contribution to the world of assessments in psychometrics. Thank you so much.

Andrea: You’re welcome Sylvain thanks for having me. It’s been a pleasure.

Sylvain: Thank you.

Andrea Wright – www.linkedin.com/in/andrea-wright-41b43458

Andrea Wright is a Solutions Marketing Manager at Altus Assessments and leads research to bring their admissions assessments and program management solutions into new program areas. After a career in secondary school teaching, she worked for seven years in graduate recruitment and admissions at Smith School of Business, one of Canada’s top business schools located at Queen’s University. After completing her Masters degree in Education at Queen’s, she joined Altus to support their growth and advocacy for more holistic admissions processes in higher education. Andrea lives with her son in Kingston, enjoys being active, reading, baking and recently became a new plant parent.

About CykoMetrix – www.CykoMetrix.com

CykoMetrix is a leading edge combinatorial psychometric and human data analytics company that brings the employee assessment industry to the cloud, with instant assessments, in-depth analysis, trait measurements, and team-based reporting features that simplify informed decision-making around recruiting, training, and managing today’s modern workplace.