J.D. Mosley-Matchett, PhD (00:00)
Welcome to another episode of AI Update brought to you by InforMaven. I'm J.D. Mosley-Matchett, the founder and CEO of InforMaven. And our guest for today's discussion is Dr. Bradley Coverdale, the Manager of Academic Program Assessment and Program Health at the University of Maryland Global Campus and the founder of BC Data Insights.

He helps higher education leaders and strategic professionals use AI to make smart decisions, whether they're managing programs or building their Plan B. Welcome to the podcast, Brad.

Dr. Bradley Coverdale (00:37)
Thanks for having me, JD.

J.D. Mosley-Matchett, PhD (00:39)
We met at the 2024 AIR Forum in Denver, didn't we? Isn't it incredible how quickly AI has advanced since then?

Dr. Bradley Coverdale (00:45)
Yep.

Oh my goodness. I thought when we had, when I put that group together for us to get started, I was just at the cusp of thinking, especially cause Tableau was starting to

J.D. Mosley-Matchett, PhD (00:51)
You

Dr. Bradley Coverdale (01:01)
about Einstein and what that demo was. And so I was very curious to see where analytics and assessment and AI were all going to start to intermingle.

J.D. Mosley-Matchett, PhD (01:13)
And now you have a new company. That's pretty exciting.

Dr. Bradley Coverdale (01:17)
Yes, I have been really working and doing a lot of work with BC Data Insights, really focusing on helping strategic and analytic professionals to help them to plan using their skills and expertise that they already have now and moving at one hour at a time. We know that as professionals, we don't have a lot of time to really work.

having something that they can build momentum on for when the unfortunate thing happens is really a lot better than them having nothing at all.

J.D. Mosley-Matchett, PhD (01:49)
That is so true. So you have some ideas that our audience of higher education administrators may find helpful when considering how AI can impact the way colleges and universities are being operated. So let's explore those thoughts starting with this first question.

When considering how administrators assess the various programs that their institutions offer, are there new ways that AI could shift how

Program health is currently being defined and measured.

Dr. Bradley Coverdale (02:21)
Yes, there's a lot of different ways. I think the first thing, especially when we have any conversation about program health, it's really important for us to first think about, what is program health? And define that to start with. A lot of different institutions are going to have a lot of different definitions with that. In some cases, some focus really on the learning outcomes. Others will focus on the traditional metrics of headcount, retention, course success, graduation.

And it is important when you're moving in that direction to think about, what's good and what's bad. And more the sense of, and this is one the things we heard a lot with, within even my institution of, well, Brad, when do I know when to intervene? When do I know when to make that next change? Where do I need to have? Because in some cases you've got, you could have a program director that is managing five, six programs and not to mention all of the different courses that are going.

that are going with them. And so it's a lot of time on top of all of the other expectations that we have for our program directors, department chairs, and the like. So it's really, really important as we move forward and even to get a baseline of what is it that you're wanting to measure and how are you going to measure it. Once we have that, well, then you can really start to ask those questions too using Generative AI

to start thinking of what are you starting to look at? So if you have, for example, if you have a platform that you can bring in your proprietary information, then you have an option then to be able to look at those trends and say, help me understand what's going on with our particular students. Help me understand where are our potential gaps. A lot of what we see too often in higher ed is that this data is siloed.

And so academic affairs isn't communicating with student affairs. And there's a lot of rich information for us to be able to, for us to really be able to move forward. If we're talking about persistence, for example, and our students coming in that next term, well, it's really important if we already know what our, what our chances are, or if there are students that are less likely to come back, that's really important for our program directors to know. So they know from the beginning,

"This is the target that the university has set for me, but this helps me to get an idea of who are these students I'm working with and which ones are planning to come back and which are not." Another thing is getting goals, being able to get that information whether you're surveying at the beginning of your term, whether you're surveying when students come in, really getting a good understanding of why are your students there.

And if you don't necessarily know that, you could use Deep Research as an example to go and do a Deep Research of, could check your own Reddit, you could see other, what are people saying on the internet when you're talking about with brand identity and get a sense of, well, who are our students and where are they going and what are, and why are they here? Because students that come to an R1 are likely going to have a very different

reasoning for be here than someone going to community college, someone going to an open institution, someone attending online. It's really important that we are addressing those particular needs. And in some cases, not everyone's here for here necessarily for credentials. Some are here to transfer. Some are here to just get a couple of courses to meet a requirement, whether for job or to be able to move forward.

And so it's really important as we are tracking and saying, well, what goals are we meeting that we understand what the student goals are as well. So that's better aligned. It also helps when we're talking, we're meeting with the needs of our employers as well. So having that extra bit of information really helps us to see where do all of these pieces come into play. And again, Generative AI can be used as kind of your assistant.

It can be used as a good thought scenario. So I use it all the time of saying we have run into this challenge where I define what type of students we have and start thinking about, tell me what are the barriers that I'm not seeing when I'm just looking at the data, whether it's student life challenges that are potentially happening, things that are happening right now.

right now in the world with us having military students, our students are all over the place and we already know in some instances they're not able to complete their courses the behest of our student affairs and our great success coaches to be able to communicate that information to the schools.

The academic team should know that information as well.

J.D. Mosley-Matchett, PhD (07:20)
That's true.

So if there are inefficiencies in how student learning and program outcomes are currently being assessed, how could we use AI in a way that would prevent us from simply speeding up and magnifying those weaknesses?

Dr. Bradley Coverdale (07:37)
Sure. One of the biggest things we need to be thinking about, and this is one of the things that we are even starting to test as we're moving forward with new forms of assessment, is really looking at trying to keep things as simplified as possible, especially with the size of our institution. We have a lot of data that is coming in, and to do standard juried assessment just doesn't work. Sampling just doesn't work. So we're trying to see what can we do

to kind of mimic that. And what we are looking at is keeping the human in the loop in the sense of that if we're having our course learning outcomes to have, our portfolio directors to be able to identify, which are our exemplars and which, so you have your expert to identify which are your best choices and evidence to show them to show the AI to train it. Say, this is our best example.

Institutions feel like they know they're putting their resources in the right direction and at the same time, program chairs feel like they're finally being heard. It's like, well, I've been advocating this for years. It's like, well, you have, but now we've got external validation to go with it. The other way to be thinking about that would be in the instance that you don't have an industry partner,

you could ask chat to act as an agent of that industry partner. So, you could have a separate ChatGPT that would be running, for example, that would allow you to simulate that.

J.D. Mosley-Matchett, PhD (09:04)
Those are such great examples and really powerful thoughts.

Okay, we all know that AI can generate information faster than humans can, but if our faculty and staff are already overwhelmed with work, how are they supposed to use all of the additional insights that AI can generate?

Dr. Bradley Coverdale (09:19)
you

Absolutely. And this is one of the great things that I am super excited for where analytical AI is going. I think it has a lot of potential, but it also has a lot of cautions for us to be able to get there. Because as they say, garbage in garbage out. If you don't already have a strong data warehouse or a strong data structure that you can trust from the beginning, you're going to have some challenges as you're moving forward.

One of the things that we are looking into is, as we're building our own platform, is really trying to control what's going to be in our platform. One of the things I've heard very loud and clear from my teams in particular is: "Brad, we need this assessment process to be cleaner. We need it to be easier. We need to be more to reference. Don't give me a bunch of Excel files to go look at. Don't find me a bunch of word documents. Put it all in one place." And so, Generative AI is a

perfect place for having that kind of repository and then asking the AI, "Okay, you've given us these annual reviews for the past five years. What are you noticing? What are the trends that we're still seeing and how do we move forward?" Even transforming that more into a chatbot and really pushing them to do conversations and such. So now they don't feel like they've got to keep pulling things from the data warehouse. They don't feel like they've got to

keep generating another Excel file. They have what they need. I heard from one of our upper leadership that they go into each and every one of our annual reports to pull the action plans and do that manually. And I'm like, that is a ton of work when we have over 100 programs. You could imagine each program probably has three to five action items, action plans that they're working on in a year.

And something like that is insane where instead, one quick extraction, one quick say, provide me this table of what the action plans are for this year and tracking which of these have been approved, which of these haven't been approved, why haven't they been approved? Really being able to bring that into really into one location. I think,

the RAG technology gives us a lot of real flexibility that we move in that we don't allow the internet to be able to impact that particular space. That's our library. That's where we can trust and feel that we still have that source of truth. And I know that's probably an area that's been a contention of some of what's getting shared out and such. I think that those kind of privacies will be very helpful.

for moving forward. But once we have that information, then we can then say back to the academic teams, "Okay, based on the goals that have been set by the university, based on where you've set with your action plans, this is where we're at. Now what's the next step? Where do you need to do further?" And really, one of the things I have loved as an identity, and I really think we as higher ed need to think about that moving forward,

is that programs really need to start thinking themselves as many business owners and realize that they cannot be the only one that is doing everything. So again, like we had talked about before, in some instances, it's going to be that conversation back with your tutor and with what's going on with advising, those different elements to really see what's working, what's not working,

and to again, to do what you can to provide that overall student experience.

J.D. Mosley-Matchett, PhD (12:55)
I love the idea of AI as a unifying process.

Now, I think everyone agrees that it's more important than ever for institutions of higher education to be strategic. So how can AI make our strategy development easier and faster?

Dr. Bradley Coverdale (13:13)
Sure, I think one of the best ways for it to be able to, as we move forward again, when we have those goals, is it helps us to see where are we aligned. One of the challenges that I've seen all too often is people come up with this idea and then someone starts coming up with the processes, but they don't see whether or not those processes are aligned, those targets are aligned. And then it flows all the way down to where people are like, where do we go from here? How do we get

this implemented. And I think at the same time, this especially as we're moving the strategy, it will help us to see where should we be putting our resources, where should we not be putting our

It won't be more of thinking in the sense of getting objective and rubric based so that we can make those decisions because unfortunately we don't have unlimited resources.

Some things are going to get funded. Some things are not. Some programs are going to have to cycle out because it's their time. If someone was doing an eBay e-commerce webpage right now, that would probably not be necessarily the best of moves for moving That time came and that time has passed. And I think using AI

to help us to really think about, well, what are the other things I'm not thinking When I'm communicating with my teams and I say, this is the problem.

Help me understand where are the flaws, where are the questions that they're going to have, because that'll make me sharper and make it so that we have a lot less time taken for us to be able to move forward. It's a lot less needed for clarity sake. We can now start moving, especially, my goodness, even with what is happening now with chat transcripts and keeping everyone up to date. We keep so many meetings

recorded now because there's just so much information. It's like I just need this quick takeaway And I don't remember what decision we made four months ago and no one wants to take notes the way that we had to before. They all want to be contributing to the conversation because we're moving at the speed of information so quickly.

A couple months ago, you said we were doing this. Are we still doing this? Are we moving in that direction? How does that line up with where you're going? I think those are a lot of great tools for us to continue to be building that strategy and really to move those forward.

J.D. Mosley-Matchett, PhD (15:36)
Now, is it smart to just keep doing things faster or are there risks that we should consider before simply speeding up our current processes and practices?

Dr. Bradley Coverdale (15:46)
Oh yeah, there's definitely risks and this is one of the things I've really appreciated with AI is it really questions us to go, are we doing the right thing? And are we set because it is as good or as bad as, you program it to

more so than, more so than I think any other technology that we've had before, cause it will do exactly what you tell it to.

I can ask it and say, well, why'd you do it like this? It will tell me. And so then I can start to understand, "Oh no, that's not what I meant." And this is the fallacy for us to be able to move these things forward. But again, it comes back to what we said before. I've seen so many people look now and being like, we now need to have a very clear data dictionary. We need to have a very clear set of data for us to be moving forward.

And using AI as the driver because they're recognizing that we can't move forward and we're going to fall behind our competitors because we've been putting up with bad practices and bad processes that we always said we get to someday. Well, someday has arrived. And now it's the question of, you going to put in the time, the effort and the resources to do it? Nobody wants to go through 300 dashboards and say, these are still good. These are not still good, but somebody needs to do that.

Nobody wants to necessarily go through a bunch of different processes and things that have been happening. But at the same time, those need to be mapped out so that we can understand what can be automated, what can't be automated. Because you can't get to faster if you don't already have a process that manually you agree with and you recognize where the potential flaws and biases are with it.

So in that sense, this is still why we still cannot go fully automated with everything. There still has to be a human to be monitoring what processes are being done. Are they meeting certain requirements? Are they meeting certain quality expectations, especially as we move forward? I mean, one of the big things we've seen a lot, especially in the higher ed space, talking about automation is degree transcripts

for validation and verification purposes. Those are a lot of manual work that eventually we want to get to an automated process. But when there's a bunch of nuance and caveats that an AI is not going to know about those unless you put those into the script. And that's one of the great things AI is constantly learning, but we have to be willing to be patient and continue to teach it and

provide it that source information.

J.D. Mosley-Matchett, PhD (18:21)
You're absolutely right. There still is a lot of work to

However, one of the beauties of AI is that it can help us understand what's going on in those processes and what we need to tighten up. So it's a two-edged

Dr. Bradley Coverdale (18:36)
Yes, yes.

J.D. Mosley-Matchett, PhD (18:37)
⁓ this is great. So I really appreciate your time with us today. It's been a powerful discussion and you've pointed out some frequently overlooked consequences when higher education administrators fail to be strategic about implementing AI. And I love the way that you pointed out that saving time and effort shouldn't be the only goals when adopting AI because

this new technology really requires us to think in new ways if we want to effectively benefit our institutions. Thank you so much for sharing those insights with us.

Dr. Bradley Coverdale (19:07)
Mm-hmm.

Absolutely, thanks for inviting me.

J.D. Mosley-Matchett, PhD (19:17)
For more information about AI news and trends that are directly impacting administrators in higher education, please follow InforMaven on LinkedIn and visit our website at InforMaven.

dot AI.