A CykoMetrix Spotlight Production

Every week, the Spotlight shines on an amazing professional with a story to tell and lessons to teach. Welcome to the CykoMetrix Spotlight.

The following is an adapted transcript of the exchange between Sylvain Rochon, CMO at CykoMetrix as host, and Gustavo Miranda, a Business Intelligence developer working in the children’s mental health field.

Sylvain Rochon: Hello, welcome to CykoMetrix Spotlight. My name is Sylvain Rochon. I am the Chief Marketing Officer at CykoMetrix. A leading-edge combinatorial psychometric and human data analytics company that brings the employee assessment industry to the cloud with instant assessments, in-depth analysis, trait measurements, and team-based reporting features that simplify informed decision-making around recruiting, training, and managing today’s modern workplace.

Today, I have with me an interesting guest.  His name is Gustavo Miranda. He’s a business intelligence developer in kids’ mental health. He has worked in the Children’s Mental Health sector in Ontario for over 10 years as a program evaluator and in decision support, and currently as a BI developer. During that time, he has modeled and analyzed results from child psychometric measures such as the BCFPI, CAFAS, and ASEBA as well as HR, performance, health, and client satisfaction data, and combinations thereof.

So, we’ll talk about a lot of data today. Welcome Gustavo to the Spotlight.

We tend to interview a lot of people that are psychometricians, specifically people that are consulting, training, and staffing companies, all sorts of people. Very few people actually work with kid’s data.  You are one such person that analyzes and crunches and does your magic. Tell me a little bit about your work first and then we’ll go into details.

Gustavo: Sure. You’re used to working with psychometric data that predicts where people fit in a workplace, and it helps people develop teams. I work with psychometric data that helps people understand what kinds of services children need, and how well they improve when they get those services.

When I graduated from college, I started volunteering at a kid’s mental health center. I was doing some research and some coding, and eventually, I ended up developing program evaluations which are exactly what I explained. You take a program, say a CBT program or a group therapy program, you do a psychometric measure ahead of time, you do one after. In a perfect world, you would do one like 6 months after for a follow-up, but that’s very difficult. Then, you run some stats on it, a T-test or an ANOVA, and you see how well it’s going based on that. Also based on how the clients feel about it, satisfaction surveys, how the staff feels about it, and how other community partners feel about it.

When I started doing that, a lot of that work was very tedious, right? So, a lot of it was cutting and pasting data. Getting it into the right structure. Then, next year or next quarter, when you have to do the same thing, doing all that manual labor again. Then, because either I’m lazy or efficient, you can pick the word, I thought there has to be a better way, right? How is it that I can go on a website for curtains, and I can just click a dropdown and see a bunch of different options and slice the data, but I can’t do this with something that’s, I think, a little more important (or at least as important) as curtains, right? And so, I started getting kind of good at Excel. One thing that was always hard was getting a distinct count of clients. So about once a month, I would Google “how to get a distinct count in Excel” and one day something popped up that said “do this with Power Pivot”, and overnight, one day, Excel added the functionality that lets you build a whole data model, so, a relational table structure.

A few months later they included something else called Power Query which lets you automate the restructuring of data and that just changed everything. Now all those weeks of work which was cutting and pasting and restructuring was just automatic, it was overnight. Now, we spent almost no time on that, and we spent all our time writing decent reports, getting information to where it has to go.

My niche is Power Pivot and Power Query. That’s what I’m talking about. They’re business intelligence tools. So, for the most part, when you see demos online, it’s all about sales. It’s all about churn, things like that. Churn is also important for us. What I really got interested in is how you structure that psychometric data, so that you can build fast reports off it.

There is a certain way to do it because you have the individual items. Then you have the scales that those items load onto. There are always, in my field anyway, a few critical items: anything endorsing self-harm or substance use; tendencies toward violence are something that you always want to flag. Now that we have the ability to automate that data structuring and to automate the report, you can make that available to the workers, the frontline staff; it’s a lot easier, and you can roll up those numbers to leadership a lot more easily too.

Sylvain: Right. You created a system using Excel to take the data that’s been picked up through assessments and provide reliable reports without a lot of data entry or a little, if any. Just the entry point, I guess the data for executives and frontline workers, I’m assuming you’re talking about care workers, nurses, and others. Okay. Since we’re talking about kids, in this case, what ages are we talking about? Kids are a bit different when you want to get data from them. How does the data get gathered with different age groups?

Gustavo: Yeah, so we don’t do a lot of psychometric stuff with kids that are 0 to 6. Although there are some measures, I couldn’t name them off the top of my head, where the parents would rate the kids on certain things. There are other measures tangential to the kids. You might have a measure where a parent is rating how often they’ve used certain kinds of discipline over a certain time period, right? Based on that you can predict the success of the strategies that they’re using. You can suggest a certain parenting program, or put them in a certain stream.

Then, there are other measures that are interesting where sometimes kids as young as 11 can fill out certain things. There’s one system, ASEBA that I mentioned, that has measures the parents can fill out, the child can fill out, and the teacher could fill out or a care worker. That gets really interesting because then you could see if certain problems or in certain domains or in certain areas. If attention deficit is high in school but not at home, then you might think, well, maybe this client doesn’t have that problem overall, it might be something just about that environment.

A lot of times, kids were very honest about it. I say kids, obviously older teens. They’re very honest about substance use and then the parents would have no idea. The other thing that’s interesting from my point of view is that it was so hard before to look at those side by side in any kind of aggregated way. Again, when I started using these BI tools, they got so much easier.

Sylvain: Well, it’s something that we’ve also noticed in our system. You have a different assessment, and they live in data silos. You can take a PDF and look at both of them and do comparisons, like eyeballing it, but it’s really hard to determine especially if there’s a lot of information. What’s really happening between the two? If there are more pulses, we call them pulses when there’s like a moment in time when we measure something, and then you are going to compare 4 or 5 PDFs, right? It becomes unmanageable. So, in our system, all the assessments are in one data engine similar to data in an Excel sheet. Then you can play with the data and get lots of information out of it that you wouldn’t be able to otherwise, so I get that. You did this all in Excel which is really neat.

Gustavo: Well, these tools Power Pivot and Power Query (So, Power Query is the ETL tool and Power Pivot is more the analysis tool, so you can write your sum, you can write your same period last year, your year over year change, and things like that) but these tools exist in Excel, also exist in a program called Power BI, and more and more they exist in the cloud too. A lot of the restructuring that you would do with Power Query first now, in Excel and Power BI you can do right on the web. If you have a certain license. I would say, just do it using Excel or Power BI if you’re starting out.

But you mentioned, taking data from Excel sheets. The amazing thing about Power Query isn’t just that you can restructure Excel sheets, is that you can pull data and restructure from almost anywhere. So, you can actually pull out the tables from a PDF, restructure in a certain way, and put them side-by-side in a data model with, I don’t know, data from a government API, right? Or with your psychometric data. You could pull data from Statistics Canada, for example, about the number of kids in certain Geographic areas, and then you could put other data about how many kids you’re serving from those same areas. You link them with your geography table, and then you can see how it matches up.

If you have some kind of government data or any kind of wide-ranging Geographic data about certain psychometric issues, presenting problems that happened in different areas or at different rates, and you’re measuring something similar in your area [then] you can see, “Oh, look, these kids that we serve in Ottawa. We serve 20% of kids with anger issues, but in Ottawa in general, there are 30%. Maybe we’re underserving. Or in Toronto, we’re serving let’s say 40% of our male clients present with trauma but in the population, it’s only 20%. Maybe this is our specialty or maybe we should look at other things too.” So that’s really where it gets amazing. It’s how easy it is to break down the silos now.

Sylvain: That’s really neat. I was not aware of all that stuff. I’m not a developer myself.

Gustavo: Actually, let me just mention on that note that a lot of stuff that you do in Power Query, a lot of it is point and click. It’s really amazing how easy it could be to save yourself sometimes days of work with just a point-and-click. Once you get into the actual language, which is called the M language, then you can do some really crazy stuff.

Sylvain: Well, you’re doing that, and you’ve been doing this for a number of years in the health care industry, mental health. How prevalent is this type of work in the world? There are a lot of tools that you mentioned, Power BI for example, as a standard BI tool, but I’ve noticed that there are a lot of people that are complaining that it’s hard to get data that they want in the psychometric fields. So, it sounds to me, from me listening to a bunch of people that what you’re doing is a bit unique, do you think so?

Gustavo: I’ll say it might be unique in the grand scheme of things. Obviously, these things are used more in business settings, and usually, Community Mental Health agencies don’t have tons of funding, might have 1 or 2 data people. Sometimes it might be a finance person who’s doing this stuff because they’re kind of good with Excel, or it might be IT because there might be someone in IT who’s good at SQL.

 It’s not as pervasive but I will say that around the time I started using it, I’ve known at least 2 other people at big agencies that caught onto it, and we all sort of came to the same conclusion for the price point that it’s at. A lot of stuff you can do for free if you have Excel. But at the price point that it’s at, I think for a non-profit, you could get it for 6 bucks a user with the cloud service where you can share securely and everything. It could be a game changer for kids’ mental health if we could somehow, in my opinion, get the training out there so the analysts could use these tools themselves and other guys more so than certain agencies could curate the data themselves and then share it with other agencies. But for me, the main thing is, how much labor you save in terms of the data analysis and how much stress.

Sylvain: The service level, they want to have more information and more data to make determinations or take decisions. But there’s all this work that has to happen before they get that reliable data. As you said, you can do it the traditional way just by manipulating manually which is a huge expenditure of time and resources. Or if you can find a way to automate and offer a tool where the end user can just pull whatever data they want. It can become a massive thing that has huge value because you’re getting what you want, you’re not expending those resources and all that time.

Gustavo: It’s also that, a lot of the work you’re doing is in terms of quality improvement of the product itself. You know, you’re always building on what you have before, whereas before, you would take a few weeks to put together a report and they would want, “Oh, can we split this up by quarter instead of by month or vice versa?” It’s going to take another three weeks, right? But now, you just put that in the drill down and so you’re not going back and forth. You’re just building on what you have. So as a developer, you have more options to just sort of quickly design a report and adapt it to different users and changing needs, as opposed to going back and re- doing that work when people realize that what they asked for was exactly what they wanted and that’s the way that it should be with the technology that we have now.

Sylvain: Well, at least that’s the way we think as technologists. It must be efficient. Otherwise, what are we doing, right?

Gustavo: But that’s the thing: in IT development, that’s been how it works, right? You build version 1, version 2, and version 3. You don’t go back and do everything. But we haven’t really had that option in data analysis and research and fields like that before.

Sylvain: I think you alluded to it but let’s talk about this a bit more; is there a plan to deliver what you’ve built, and perhaps some of the colleagues you mentioned that are also clued in, as a product that would be distributed to a broader audience that also needs that kind of connectivity?

Gustavo: I’ve done certain things where I’ve worked for a client information system, which is like an ERP but for the clinical, where I fixed up a model that a lot of different agencies use. Right now, I’m working on some projects that different agencies use. The ministry, I doubt they’re using Power BI, they’re probably using some other enterprise system, but the ministry of Ontario is now undergoing a very long project to deliver these kinds of metrics on a regular basis to agencies.

Sylvain: So there’s some interest and understanding from different stakeholders that this is a good thing, good value. Then there is some effort to actually do some kind of distribution of those systems.

Gustavo: Yeah. On a more philosophical level and I think now there’s more of an understanding that’s the first thing you get, (especially on such a massive project the ministry projects and the Ontario-wide project) the first thing you get is not going to be great, right? It’s going to be a process we’re going to put something out, there are going to be mistakes in terms of how people submit the data there are going to be mistakes in how they show the data, there are going to be mistakes in what they call mistakes, even. But over time, again, as I said before as you build on what you’ve done before. They’re asking for feedback from their stakeholders and I think over time there’s going to be something decent.

Sylvain: Well, it is software development, I understand that. Iteration is key. But you got to start somewhere and then people get also acclimated to the software on version 1. For example, in version 1, they get to understand. They get to see a bit of the value of it. Then as you roll out new features, it gets auto-adopted. Well, that’s really good. Do you foresee, because we’re both in Ontario here and health care is provincial in Canada, — and for those in the States who are listening to this — do you foresee the healthcare system broadly adopting this since it’s already in the works?

Gustavo: The Health Care System already has CIHI, I can’t remember exactly what it stands for, but it’s the Canadian database of Hospital data for intakes and discharges.

So, there’s already a huge BI system that Canadian hospitals and health care workers can access or at least data workers in those hospitals can access them and build things for their staff. Ontario also has one so. Mostly they pull from CIHI, then they add some other data. So, those are already existing in health care but in kids mental health, I think there’s been some progress made. But also, because more and more people are using these tools that I mentioned, which make it very easy and accessible, I think the stakeholders are savvier about what to ask for and what to expect, and what’s possible.

They know what they can build with their systems now and there’s some expectation that eventually any kind of provincial system is going to provide functionality that’s as good, with the understanding that is more complex because you’re bringing in data from different sources and that have to talk to each other. Even harder is the fact that, as mentioned, some of these agencies don’t have the budget for a big data team. It might be like a finance guy and an IT guy, some of them have volunteers who work on stuff like that, so they can’t all provide the data in the ways that the ministry needs it. So that’s another front that they have to fight on, right? They have to somehow build capacity in these other agencies to provide that kind of data to them.

Sylvain: You probably anticipate data from kids’ mental health, what you’re working on eventually getting absorbed or getting input into the main, I forget you mention the name, the main provincial Healthcare System, perhaps? It would be combined or at least accessible to the same people.

Gustavo: I’m not sure. I mean, it’s not unthinkable, but it’s sort of its own system for now.

Sylvain: Yeah, that’s what I figure. Well, there’s still a lot of work to be done, it seems but you started.

Gustavo: There’s always another version to put out.

Sylvain: It never ends.

Gustavo: It’s funny when I started doing this stuff, people thought it was magic what I was doing. Now the expectations have caught up with the software.

Sylvain: Well, they can’t have enough, right?

Gustavo: Yeah.

Sylvain: Well, that’s good. That means there’s a push forwards. Resources are available now to actually do the work more, right? And the higher quality the more data we had then the better we’re in a position to make good decisions. That’s kind of the point for us in psychometrics. Bring good data into the hands of the right people and they can make good decisions. It doesn’t matter what it is but it’s always about the value of the data. Sometimes it’s quantity so you can have benchmarking comparisons and whatnot too, and you’re helping out with that. That’s great.

Gustavo: Yeah, for sure. I mean what’s interesting is to bring it all around. Let’s say that you wanted to compare the data, I mean, I’m pretty sure it’s not your model (or is it?) but usually, your main client wouldn’t have access to that data, right? Or would they? But let’s say they did, hypothetically. Let’s say they had one training platform for their HR type training and then, they had your training platform and your psychometric data, you could always pull the data from your system and the data from the other system. You can say, based on these different psychometric profiles, how well are these clients doing this other training? Or what’s the churn based on these different psychometric profiles, right? It’s always an endless cycle where this data is taken up by the other data and analyzed in different ways.

Sylvain: Yeah, slicing and dicing, right? Well, thanks a lot, Gustavo, for illuminating us on what you’re doing because a lot of the behind-the-scenes of what’s really happening in certain areas is underappreciated.  Children’s healthcare I think, or mental health specifically is something that concerns everybody, every parent, definitely.

Gustavo: Especially in the last couple of years.

Sylvain: Yeah, especially. So, that’s great work. Thank you so much for what you’re doing in the area and for participating in the Spotlight.

Gustavo: Thanks.

About Gustavo Miranda – www.linkedin.com/in/gustavo-miranda-mhpes

Gustavo Miranda has worked in the Children’s Mental Health sector in Ontario for over 10 years, as a Program Evaluator, in Decision Support, and currently as a Business Intelligence Developer.  In that time, he has modelled and analyzed results from child psychometric measures such as the BCFPI, CAFAS, and ASEBA; as well as HR, performance, health, and client satisfaction data, and combinations thereof.

About CykoMetrix – www.CykoMetrix.com

CykoMetrix is a leading edge combinatorial psychometric and human data analytics company that brings the employee assessment industry to the cloud, with instant assessments, in-depth analysis, trait measurements, and team-based reporting features that simplify informed decision-making around recruiting, training, and managing today’s modern workplace.