AI Didn’t Break Higher Ed—It Just Pulled Back the Curtain: An Interview with Patrick Dempsey

30% of students right now are enrolled in degree programs that are going to make them less money than had they not gone to college.

Anand: Higher education is facing its biggest identity crisis in generations. But if you ask today’s guests, AI didn’t cause the crisis, it just exposed it. Patrick Dempsey works at the intersection of human learning and artificial intelligence. And he’s uninterested in being polite conversations about incremental change. He argues that our universities are suffering from institutional antibodies that attack innovation and that our obsession with rigor is actually just pedagogical hazing. Also says that maybe a 19-year-old YouTuber might understand how to earn attention better than a tenure professor. He’s here to tell us why shared governance might just sink the ship and why 5-year-olds outperform MBAs and problem solving and what higher ed needs to do to survive in a world where knowledge is free and abundant. Patrick, welcome to the podcast.

Patrick: Thank you so much. Wow, that is a great encapsulation of many different strands of thought and uh I feel the pressure already to be able to uh call on them, but I’m ready.

Anand: Well, we’ve been looking forward to this for quite a while. We appreciate you being here.

Stefan: Awesome. Yeah. Thanks so much for being on the podcast and yeah, I love the intro too because I think it really encapsulates you know um what I like about reading uh you know all your LinkedIn uh commentary uh you know and your Substack because I think you’re one of the few people uh who really understands kind of the magnitude of the changes that are needed not just to help the students who are enrolled in our institutions but also kind of maintained the relevance of those institutions. Now, we’re going to talk in a little bit about, you know, some of the mechanisms that like exist to kind of like maybe stall change. Um, maybe I could describe it that way, but there also seems to be like some reasons that people want to maintain the status quo. Uh, even even if they’re not stated, uh, you know, explicitly when, you know, with all your work in universities, what do you think are some of those reasons like why they don’t want to maybe do more than reshuffle the deck chairs?

Patrick: Wow, that is a that was a starting starting off easy, the softball to start. Uh there’s definitely a sense of and I think there’s definitely a sense of we’re we’re operating in a in a few different models in higher ed where there’s this idea that education is the great equalizer and we start championing this message of access and and equity for all. But when it really comes down to it, a lot of folks that are working in the field, especially tenured faculty, they they grew up in a different environment, right? And the story that I think a lot of folks tell themselves is that, you know, they walked uphill both ways in in the snow and they really bootstrapped. Uh, and it really is a meritocracy. And while we say these things about equity or access, the underlying assumption is that well, college actually isn’t for everyone. It it never has been and it never should be. there really should be a limit on uh who comes in and who who comes out the other side. You know, this really is a mechanism for separating folks that you know have what it takes to um journey through the ed higher educational experience. So I think there is in conversations that I’ve had there’s this underlying um unspoken assumption that you know I am really here to say if you don’t have what it takes you don’t have what it takes and that’s really not my problem right. I’m a defender of my field I’m a defender of this discipline I’m really like operating maybe on a like an old Greek um concept of what a university is and supposed to be and who’s supposed to have access to it.

Anand: So, um, not to put too fine a point on it, you know, as somebody who’s been, uh, steeped in that, uh, kind of the atmosphere of higher education for 30 years and and as a faculty member, there’s so much about that that you say that really resonates with me because I find myself falling into it. I see my colleagues falling into it. I I love the way you refer to this as kind of institutional antibodies, some of the mechanisms and process that sustains all of that. And then also the concern about shared governance, you know, assuming that we have just infinite time to reach consensus. Uh I’ve chaired our faculty council a couple of times and uh I’ve been through that process and there’s a lot about it that I like. Uh but there’s also a a set of challenges that we’re facing that that our institutional models just aren’t meant to address. They’re just not equipped to be able to address a lot of that. So I’m wondering a little bit about how when do you think governance crosses the line into, you know, how far does it go? does it cross into a point of of sabotaging our very enterprise? And maybe tied to that is what’s the role for tenure in all of this? If tenure didn’t exist, would that change the way that education’s able to adapt?

Patrick: So, yeah, I I think that governance is specifically formulated to do what it it’s doing. And of course, like anyone that has a position to influence the outcome would have self-interest. And so of course you’re not going to say that my department that has six students in it should be shut down. Like you’re never going to say that because that’s your job. Like of course um and and again I think higher ed has changed so much in it is such a connected ecosystem and there are so many constraints and opportunities that it faces that these decisions about that we own the curriculum. They’re they’re strategic decisions whether we want to say it or not. Once we got to the place where we said everyone should get to education, now we’re competing for these students. And so every decision we make is part of the value proposition that we’re making to students. And of course, you are making a decision that is isolated on your interest. And we set it up so that we’re forcing, let’s say, chairs or program directors to compete with their peers for the same students because they’re not really involved in enrollment uh per se, right? So they go to conferences or whatever they’re doing, but they’re part-time marketers or they’re part-time recruiters and again we’re they’re getting at the same body of of students because we’re not the marketing department. And so all of these things really become strategic decisions that are being made by people who don’t have insight into the strategic realities of the institution. So I don’t know that that necessarily has to do with tenure per se. Um, I think if I’m going to again not put too fine a point on it, I think your R1’s where like folks are doing essential research and essential is obviously a a term that can be argued in different ways. But for almost all instructors, faculty that are doing research outside of that, then I think the concept doesn’t make a ton of sense, right? We’re the field is so big, you’re what you’re doing isn’t that impactful.

Patrick: just to just to put it again and and yes there are institutions that make a big difference in their research and the rest of us are publishing papers that our mothers won’t even read and so to the extent that we have a say in the strategy I mean we should have some input but it shouldn’t be binding

Anand: yeah so for those of us that are working from the inside um what’s that mechanism for us to kind of break free of some of that how do we how do we move beyond some of the constraints of shared governance that you know many of us have just been you know ju just been part of for so many decades. Uh it’s kind of hard for us to even get out of that mindset like what what would you tell faculty in that position?

Patrick: So I worked at an institution and the president just said this is what’s going to be and you’ve got x 3 years to get on board and I’ll write you the recommendation if you don’t want to. And I think that we have this idea that consensus comes as we build buyin. But no matter what you say as a leader, people are going to be annoyed by it or at least some people are going to be annoyed by it. And so culture changes as success happens. And so declaring what’s going to happen this is who we are as an institution. I think that’s what people actually need to know. This is this is actually our they need to understand it is a business. While we do provide a different service from let’s say a tech company, it is a business and it faces real business constraints. So I think just positioning it that way telling us how we’re going where we’re going into the future and how we’re going to get there. Then all of a sudden we have the mindset that we are in this together, right? I am not necessarily competing with my colleague over there same student. So I think it’s just a this shift of transparency and honesty like just say these are our budget shortfalls. This is what needs to happen because of it. these are the students we’re going after. This is how we go after them and this is the role that you play in that journey because it’s not like you’re not you don’t have a role anymore. It’s just your role is different given the realities of the institution as an industry as it is today.

Anand: That makes a a lot of sense to me. I mean I remember having an instance at an institution where I was in a department with multiple disciplines in that department and we were just pitted against each other because the department had to make a recommendation for what position they were going to request. And I finally just said to my colleagues, why are we doing the dean’s job for them? Um, you know, they should be making that decision. So, we should each each discipline put forward their best proposal. Let the dean make that decision rather than forcing us to fight it out and then they can simply say, “Well, I’m following the will of the department.” Um, yeah, that strong leadership model is something we definitely have space for. We probably need more of it. Um, maybe we can get into a little more about what’s holding some of the the leadership back and preventing them from pro providing that kind of vision. Yeah. Oh, Stefan, are you you’re muted? I don’t know if you were chiming in.

Stefan: Yeah. So, you know, one thing, you know, you you talked about and I assume uh you know, you’ve spoken about is, you know, AI has disrupted higher education, but it’s really kind of just exposed to a lot of things and you know, some of the things that you you know, you and Anand were just talking about, right? There’s budget shortfalls compounded by the demographic cliff. you know, there’s kind of an an attack on higher ed from some different places, right? There’s declining interest in some majors. Maybe there’s too much emphasis on, you know, publication and in nonr1 uh institutions. And how do you see kind of the, you know, the AI is as kind of being the trigger there? Maybe, you know, the straw that broke the camel’s back or, you know, it’s kind of just kind of what what’s pushing things over the edge.

Patrick: Yeah. Yeah, I mean clearly it is has shown that assessment the or credentials are the thing that we’ve always done and that they’ve always been proxies. I mean I’m not the only person to say that they’ve always been things that stood in for other things and it’s not really higher education’s fault. It is the the accreditation and funding. And so it’s all of these elements that work together, right? we still have to say to get federal funding that our students that we’re we’re reading these these standards that we didn’t actually create and it’s all pure like it’s it’s just this whole it’s it’s like a it’s like Bernie made off it’s like it’s like a multi-ter marketing scheme where we’re all just in it to keep ourselves in it. Um and so yeah, AI exposes that look look your expertise wasn’t actually rare like you were just gatekeeping. Um I published about this in my dissertation and in the book that I published in 2021 like the internet already disrupted the historical purpose of the institution which was the university the the place that brought in these various thoughts and so the internet already broke that model but we still operated on it. And now AI just is like not only do students does everyone have access to all of this information now it has access to all the outputs that that information can create. And so yeah it’s crisis time. It is I just at any time someone makes a post I can just post the scene from the Wizard of Oz where they’re trembling but then Toto goes up and pulls the curtain and he’s like just just pretend this isn’t happening. Pretend everyone doesn’t already know this. And so that’s why I think ex expertise actually isn’t that rare and that’s why to to connect and maybe this jumps in into the YouTube expertise is everywhere now and it’s coming in different forms and it’s not primarily white collar or academic.

Stefan: Yeah. And I I think you know obviously the you know like you said first the internet and then the gen AI right it it makes like all the content available and it can be kind of reasonably taught right from from a variety of kind of people or technologies right people as the YouTubers or just kind of the technology itself. So how do you see kind of the you know the role right and some of this comes from even you know the the discussions we had from before about you know publishing and that type of stuff like what do you kind of see the optimal role of you know a teacher or professor you know becomes in this world what should they be focusing on

Patrick: so it definitely depends on the discipline right I mean if you are teaching one of these essential disciplines where the where you the 1% of the population does need that terminal degree it’s a medical doctor JD or something. So or and engineering and even so so yeah what something that the YouTube and the creator economy really teaches us is that the value of the instructor is in what I just say the heruristics that they provide. So they don’t start with here’s a textbook and read it and then at the end of the time we’re going to apply this information. They say here’s how on day one you can be successful because of all of these things I’ve learned. and they just create a mental model of the entire universe of thought in that field. And then they use that mental model to break it down into further mental models. And so what they do instead of saying here’s the breath, but eventually you only need to know this, they say you actually only need this to start and that’s going to get you going and then you get to the breath. So it’s the exact opposite model that we use in higher education. They make it simple. They make it feel like you can do it. And as I say, they really give you a they give you a secret. Um, and they give you a structure and they give you a system for operationalizing their field of uh, expertise.

Anand: It’s great. Yeah, I’m really intrigued by this and I I love that explanation of how we’re moving to a different understanding of expertise. I I think for traditional academics, you know, this you refer to this traditional rigor as maybe pedagogical hazing. Uh, and you know, I I see some of that. So, we went through this, so you’re going to have to go through it as well. Um, how I guess part of what I’m I’m wondering about is maybe speak a little bit more to this, and you were already talking about this a bit in terms of the dissemination of information, the types of expertise that are available now, but how do we get faculty to shift from that mindset? Is this the kind of thing that because we’ve been so inculcated in it, it’s pretty hard for current faculty. Is this about changing it within training, graduate school? um is it and and that’s one one thing that as an aside I’ve always kind of joked about how faculty are never really taught how to teach. Um graduate school is more about exploring your discipline and they expect you to learn how to teach and figure it out on your own. And so more of it is just habits that are being passed down. Some of them bad, some of them not very good. Um how do we provide that kind of training to faculty to to get out of that?

Patrick: Yeah, I think that’s exactly right. And I’ve had uh many conversations and it really is uh and the honest ones have just been like if the students have access to this then what is my role? And it really actually gets a little bit more practical than they’re like I have to teach for another 10 years before I can retire and they really and what’s fascinating to me is that they have never conceived of their role as what it I think it actually is because I think what the YouTubers and the creators are doing is what faculty generally are doing. they just don’t see their value in that. They do see their value in the dispensing of really ubiquitous information and that’s not their value. And so I think it’s just helping them reframe that these are things that you’re already doing. Yes, this is going to require a new skill and I do think it comes down to your subject matter expertise is not the breath. It is the fine point that you put on it. It is the heristics, the mental models, everything that you’ve learned from experience is why you have your job. Um, at least presently AI can put together frameworks if you ask it to. But a creator or a subject matter expert who knows this, who’s able to package that, that is their unique value proposition. And so that I think is the the training is just showing like you are you do have the expertise, but now we just have to package it in a different way.

Stefan: And how much of that, you know, you talk about how uh, you know, there’s a difference between a coach and a judge, right? And, you know, even just kind of what we, you know, just discussed about kind of the faculty member being the the judge who, you know, it functions as the gatekeeper, right? Like, uh, you know, most of my academic background is, you know, in coaching and coaching debate on and I we were debate partners as we mentioned on here a few times. How how might you articulate you know what you’ve been talking about in terms of like how a faculty member would coach a student right to to kind of understand kind of the content and you know these different ways of learning.

Patrick: Yeah. So I think I go back to the marshmallow experiment. So I think that has been really fascinating now they’ve been talking about that more um it’s really opened uh faculty’s eyes. So, just to explain to everyone that the marshmallow tower experiment, it’s not the marshmallow experiment where you’re delaying gratification. It’s this marshmallow tower experiment, which is really more about team building, but what the researchers found is um they give uh groups a uh a marshmallow, like 20 pieces of spaghettis, a yard of duct tape, and some string, and they have to build a tower, a the highest tower possible with a marshmallow on top in 18 minutes.

Explore

Patrick: And what they found is that kindergarteners, 5-year-olds build towers that are 26 1/2 in tall on average. And MBA students with 18 years of school and professional experience build 10-in towers. And so the idea is that like school actually is has the wor the negative effect on on students on learners ability to actually apply things. So that’s the initial frame is that we’re actually going about this the wrong way. Like from 0 to 5 we learn everything by experience and then after from five to 22 we learn by information and then think you can apply. We sort of follow Bloom’s taxonomy as if it is they are steps to climb and as long as the student has enough information they’ll be able to apply it in all different ways.

Patrick: And so I think most people don’t realize and it goes to the rigger conversation that ambiguity critical thinking this the so ambiguity doesn’t lead to critical thinking. Critical thinking is an actual cognitive skill. It’s domain specific and it requires actual practice and so you need deliberate practice of the thing, the help of a teacher, a protective environment. You need reflection and repetition, problem solving, feedback, alternatives. So everything that happened with the kindergarteners, they just kept trying and they were in a safe space. So it was didn’t matter if it fell down and that’s why they succeeded. And so that’s again if we’re adding to the heuristics, we’re adding the creation of this space where their expertise creates the environment where you build the marshmallow tower. So by the end of it, you’ve gotten a 26-in tower instead of a 10-in tower. But fundamentally, we most instructors believe most faculty believe that be they learned despite their conditions because they had bad professors. And so they think that learning happens through ambiguity. They think that critical thinking happens because we don’t give students the rules. We want to purposely be vague. We we don’t want to spoon feed them. Spoon feed them. But that’s the exact opposite of what we know about expertise and how it develops. And that’s why I always say there’s a reason that professional athletes have coaches who are significantly worse at the sport than they are because they’re able to pinpoint the things that they need to specifically do to increase their performance that one step more. And that is the role of the educator is you don’t have to actually be the best. You just have to be able to identify the performance of the individual at the time and give them the feedback and opportunities to to increase it that to the next level.

Stefan: There’s so much there to unpack. You know, one thing that you noted about the coach role, um sometimes the best coach is somebody that wasn’t necessarily the most successful in that event. Um we’ve said this about debate coaches often, you know, sometimes you’ll say, “Hey, this is the best debater that just graduated last year. They’re going to coach.” And they’re not always the best coach. Sometimes the best coach is somebody that was, you know, certainly solid and had the foundation but wasn’t the best, but they were more reflective in the practice and they have a lot more to offer. And it kind of, this is something I don’t think many of us really consider when we think about selection of students for graduate programs. We’re selecting the student that’s always the best in that area of expertise. They’re not necessarily built for coaching. Um, so if we’re thinking about this and going back through the process of of who would make the best professor as coach rather than instructor as judge, do you think maybe we we really have to consider graduate school admissions? Does that change the way we would even consider what the the path might be for that development of expertise?

Patrick: I love that idea. And if we can connect it to the person that’s going to be the pure researcher and then the person that needs to be the teacher and I think we do need to define that uh more clearly and we do need to give folks uh at least even if we’re not selecting pre-selecting that exposure to the cognitive sciences and to other domains that are relevant like user experience design or instructional design. um just any of these ideas that end up wrapping around or coaching like literally coaching or or or counseling like whatever that framework should be the role is not dispensing of knowledge and that’s actually the opportunity I think that’s the great opportunity that AI also has is like do you because most instructors that are faculty that are honest about it they don’t want they’re so bored after 10 years like this same lesson and they know and they’ll tell they’ll they you know I can do this without thinking and they they do right they come into the class you know the slides you know the joke that you need to say you know when that person’s going to be confused and so there is no adaptivity to that there’s no joy left in the the profession but when you take this approach that students are creating and you’re coming alongside that’s I think just such an exciting opportunity so again I don’t know that if we need to pre-seelelect but we certainly need to uh emphasize these other aspect ects. Yes, some level of subject matter expertise is table stakes, but I don’t I think we can lower that a little bit because of AI and what are these other qualities that actually become more important.

Stefan: Yeah. You know, I think that that’s really interesting and maybe that also points out when you see some programs that have a research professor or professor of practice um and there’s that distinction where where they really are developing it that way. You know, the last thing I wanted to to ask about the marshmallow tower, it the what really intrigued me about this is also the sense that maybe the 5-year-olds didn’t know they couldn’t do it. Um, and maybe the MBA students have been told so many times, no, you have to do a certain way, you can’t do certain things. Um, does that change the mindset, do you think?

Patrick: I think it does because the I guess the research was saying they they spent so much time doing the the planning and the project and so I sort of present it this way is we have this old model where it’s I’m going to give you the information and then you’re going to write the essay and there’s that’s still so prevalent, right? Well, I’ll see courses where there’s four two four assessments in the entire thing and that’s it. That’s your grade and that was the only time you got to demonstrate your knowledge. So that’s the traditional model. Um, and then we get to like that MBA model which is which four years ago I thought was great. Like, hey, here’s a process and let’s break it down and let’s chunk it and you’re going to have to build this skill and then we’ll add that skill. So we took that big idea and we and we just made it into smaller ideas that built on to each other. And I think that’s a ton of what the MBA students do, but they still only have that one iteration. And then what the kindergarteners do and what the creator economy is telling us and what we all know from our life before and after education. It’s just generating stuff. It’s just trying stuff. It’s just not thinking that there’s some specific outcome or constraints that I have to worry about or some predefined way that I’m expected to do this work. So yeah, I think it’s that ability to just not care and just go after it. And that’s what I mean clearly. I always I give this example a lot. You take the kindergarten classroom and the context is student work. The environment is informed. It it’s recursive upon itself. And so all the student work is everywhere. And you can walk down the hall and those skin’s bright and it’s colorful. And then you come to college and what is it? It’s a person standing in front of the room with barren walls.

So it’s that implied and that hidden and that recursive and living curriculum that’s alive in kindergarten.

Stefan: You could just watch year after year through elementary school to high school to college how much we strip away actual student input into the learning process or student output and that correlation with the lack of creativity and curiosity because of there is you know the uh reminds me of a couple things uh you know when uh ChatGPT 3.5 came out and I kind of saw the information and I was chatting with my sister about this who’s a elementary education professor and you know once she played around with it a little bit and realize that students can learn from the con. She goes, “Well, we just need to structure more of education like we have K5.” She said, you know, just that’s all that’s all we need to do like, you know, not that that would be easy, but she’s like, there’s the answer. Like, you know, this isn’t complicated. That made me think of that. The other thing you know I was talking with uh Anan uh you know we were at a convention last weekend and you know people were talking about a lot of employment disruptions and job changes maybe a shift to entrepreneurship and I was kind of wondering out loud whether people who hadn’t been educated uh you know beyond a certain level might be a little bit better off that they’re just kind of they’re used to adapting uh you know on the fly they’re a little more used to learning on their own uh waiting than rather than waiting for somebody um you know to to tell them, you know, what they need to learn that they they may very well be more uh resourceful. And I’m wondering, you know, if you thought about uh you know, how, you know, beyond kind of the existing, you know, kind of structures and how we need to change that. Um, you know, people who have kind of spent their lives learning differently in a way that we we’ve kind of said, oh, they’re not learning, right? We’ve said, well, you know, you’re not going to college, you’re not educated. um you know how how some of that might kind of play out and whether whether you know how how those people may thrive or maybe even be better off.

Patrick: Yeah, I definitely like the metaphor of entrepreneurship um as a potential path. There’s definitely myriad I think solutions that will come because of the AI disruption. uh and I and I can’t recall the specific research, but there’s a ton of research on these unschooled communities uh where they will present them like traditional academic materials and and they’ll fail and then they’ll ask them these questions that require these academic abilities in ways that make sense to them. I think it was uh the Brazilian uh street kids who who are are vendors and and uh entrepreneurs and they’re able to answer these sort of advanced mathematical concepts when framed in the context of how they actually live and work. Uh again presented as academic concepts they they are not able to answer. So I think it is fascinating and of course school does better at school when we measure school by school and so of course students who get exposed to standardized tests do better on standardized tests. Like that’s clearly question begging. Um but that’s not how the world works and that’s not how people work. Uh and it’s not the only way to approach education. So yeah, I’m super excited about this idea of entrepreneurship. AI just tears down the barriers and going back to the very beginning that’s one of the the definitely the biggest fear for a lot of people the scarcity model that we’ve operated on it tears down so many barriers so much opportunity exists for folks to actually create things that are meaningful to try ideas to test things to to collaborate and to not feel like there’s a ceiling on their potential even if they haven’t come through the existing channels so I definitely hope that there are some models that that emerge for this target set of like 18 to 20 22 well now I guess it’s like 18 to 26 year olds that maybe still are delaying entering the workforce but are doing workforce type things and again it’s not strictly utilitarian but it’s that mindset of creating and solving problems and having impact so I I definitely feel like that’s a great opportunity for some uh institutions to take a different claim

Anand: yeah I incorporating that into the classroom um earlier you know even when we think about it most uh most programs it’s a matter of getting through the traditional system and then okay now we prepare you for the workforce and it’s that last semester where we start talking about real world applications um and it’s just we’re illquipped to uh for many faculty to be able to do that earlier and maybe this is the model to be able to do it um I want to ask you about another mechanism that some faculty have been using to trying to adapt to some of the challenges of AI and you know I’m sympathetic to many colleagues that are not really sure what to do about their assignments their assessment, how they manage all of this, and and you’ve you’ve t you’ve been able to really outline some of the real concerns and problems with that model of expertise. But we’ve had a lot of colleagues that and and Stephan and I have advocated for more of a role of oral communication in the classroom for assessment and for interaction, and some colleagues have moved back toward uh using oral exams. Um but I I get the sense that you don’t think that’s really the the best pathway forward. Is it just a stop gap? Is it something that um could be uh a mechanism that provides for better engagement? What do you think about the use of oral exams to be able to address some of the challenges of AI in the classroom?

Patrick: Well, you set me up on this one uh since it’s my current series that I’m wrapping up and I’m so excited. This is this will I guess air after this, but uh tomorrow it’s when when privilege becomes pedagogy and it’s the the final appeal to oral exams. And so every assessment is going to be preferential to students with certain natural abilities. So I think we just say that. So it’s not like I I’m never against any sort of assessment per se. It’s just that they always do advantage certain students. And oral exams are no different. So the research on this is is pretty clear. So there’s two types of oral exams, structured and unstructured. And unstructured is generally what we’re going to see in in classes. And generally again what we think we want. We want to have a conversation. We want to see how students respond. and we want to take the conversation where it goes. And that is completely invalid just because we’re biased people. We all know this when and I learned this early on although I didn’t wasn’t able to articulate it. If in week one you do a great job, you’re going to get an A. Like that’s just it. You have and I didn’t know what I was doing at the time. So you create the halo bias like this student’s really good. And I’ve had this as an instructor where I’m like, “Wait a second, this student is an A student and this week they submitted a B work and like my cognitive dissonance just kicks in and I’m like, well, it’s a fluke.” So they get an A. And then the opposite is also true. So it’s not that I don’t think talking to students is useful. I think that’s probably very useful, especially in low stakes uh assessment opportunities. And I think actually although I think people might react differently to this, if you have an AI like listening to a conversation, that can actually be a great opportunity because you can ask it about convergent and divergent and emergent thinking within that which we would never be able to as humans pick up in the conversation. Again, we’re going to have our biases. So I actually do I am very bullish on AI and oral exams because I think you can like in this conversation we could see how well I did and what opportunities I would have to emerge and we create a knowledge graph and we could connect it all and what ideas did I come in and where did I go off on a tangent what new ideas should I explore after this conversation that I think is is going to be a really exciting opportunity but doing the old way that we’re just going to strip everything and I’m going to know that this student knows really you just know that this student has is be able to be composed under pressure and um probably has some sort of background and in some cases training in this like if you went let’s say to a private school um you’re probably going to be better prepared than than a student who went to a public school maybe didn’t have those opportunities. So again it all comes down to the uh the fact that every cognitive skill or performative skill is uh needs to be practiced and it’s domain specific. So that’s again I think most people use unstructured exams so I don’t love them. Yeah. So if anybody’s listening and wants to to do this as an experiment, take the transcript or take the recording and um post in the comments your evaluation of how Patrick did in this in this discussion. That’d be that’d be fascinating. It is good. I I do it sometimes and it is very cool. And if you just use those three frameworks of emergent, convergent, divergent, it’s really awesome and you can have uh like notebook or uh claude create a knowledge graph. So definitely do it and and share what you find.

Anand: So I I talked to somebody who has built an AI platform that’s conversational and they use it for training and for evaluative purposes um and they were training you know pharmacy students um to be able to see how well they can interact with patients. So in an instance where you know you’ve identified certainly the halo effect is a real concern for faculty in terms of assessment um also that sense that we’re preferencing certain types of skill sets uh and in this case oral communication and being able to think on your feet. Maybe there’s let me push this a little bit and say maybe the case is to be made that we recognize for that second concern that we do want to prioritize oral communication. We do want everyone to be able to develop that skill set. And so it’s unfair to some that haven’t had that experience yet, but maybe we build that in a little bit further. Is there a concern with maybe adopting that as the framework to say, “Hey, let’s do this across the board. Let’s incorporate um oral unstructured oral exams that are monitored or assessed by AI if we were able to do it that way.” Um would you be a little more comfortable with it? Is that maybe the better way to go forward or are you still hesitant about employing this?

Patrick: I don’t think my opinion actually is going to be binding on the industry, but I appreciate the push. uh if we’re going to develop the skill then of course and again that’s still is deliberate practice it’s a protective environment it’s all of the it’s feedback it’s that heristic is what is it why is some why does someone have charisma why is someone quick on their feet what are they drawing from what are they doing um I am in this conversation I feel better because you didn’t give me the questions beforehand sometimes I want to know the questions beforehand but if I do then sometimes I feel more nervous or if I’m going to do public speaking and I’m in front of a large audience, I might get nervous and I know my brain can actually stop working like you literally just like overwhelmed stop working voice gets shaky. So we do need to give so many opportunities for students to practice and if we go back to the marshmallow tower like this idea that we really should like grade on a curve, right? It really should be like you should be failing quote unquote failing to start with because you don’t have the skills yet and that should be okay. um we don’t have really a mechanism to account for that. So yeah, I’m all for you want to change your curriculum and this is a key element then you actually then as the instructor have to get good at it. You have to know how to coach it. So that’s it’s just becomes resource intensive but I’m all for it.

Stefan: So what are some different you know assessment uh methods you know you you might suggest right we obviously know that there’s problems with the paper right if nothing else the AI can write it regardless of you know uh how much how much it’s really demonstrating you know what kind of oral you know exams right we see you know some some limits there that you you’ve talked about I think some you know and some faculty obviously or maybe all right we need to figure out okay well how do I you know how do I give this student an A or a B which I guess is what we’re down to uh no nobody really gets worse than a B, right? It’s essentially pass fail without the P or the F. Um how do you think uh you know unless we’re talking about ungrading entirely, how do how do think the student should be evaluated?

Patrick: Yeah. So on ungrading, I like the overall thesis, but grades are are proxies also and they’re a way that we a shared communication which I think is important. Like dollars don’t mean anything, but we exchange them and we all recognize it. So it could be A’s, it could be 100, it could be bananas, it could be five stars, it doesn’t matter. That is a really helpful way to communicate to students. So I think there’s two aspects there where hey did you did you attempt this thing? Yes or no? Yes, pass fail. I guess I attempted it and and do did all the things that were required and then how close are you to how someone in your station should be, right? you’re not at that expert performance yet, which is really what we should be doing relative calibrated, but so yeah, that’s what that that grading represents. And it it does it shouldn’t be determinative of whether you get uh an A in the course, but it’s just telling you, hey, you’re at 87% of the way there. I’m going to keep pushing you so that you can get to 100% of the way there. Um, so in to the question, um, I think the marshmallow tower as a a format is really helpful. Think of as I call it like this big hairy audacious problem that students can’t solve um that they could use AI to start to solve and that becomes your course or your module and just it’s just something that is so big that they actually have to go through this iteration. They don’t know enough to do it well yet necessarily. And so I say if if AI can complete your assignment, your expectations are too low. Uh, and the other way I’ll phrase that is um that AI should make the work possible but not inevitable. And so I think that is we’re raising our expectations and we’re putting students in situations where a if they use AI it’s going to help them but they’re not going to get there. And that’s where our heuristics come in. Uh and then the other model that I would offer which is related is really if and you don’t have to train an AI but imagine you’re training an AI to create the output for the assignment. and I’ve posted on this before and and offered frameworks for it is what we’re really saying when when we’re assigning an essay or when we’re doing these equations or this report is these series of skills that together form that final product. And so an essay is to do that well, you need to know what a good essay is and what a bad essay is. You need to know what a good paragraph contains and how to actually structure a sentence and make a transition. And so as opposed to doing that, which AI can do, it’s really unpacking that’s almost an annotating that at this really atomized level and creating what I call is like a training pack. If you were going to have AI do this thing, well, what are all the things that you need to give it? A rubric, um, again, heruristics, examples, um, maybe research, maybe bad examples of things that you don’t want it to do. And so that becomes the work of the student is that that metacognitive level which is actually what we were always after anyway. When I I I can explain it this way is often what instructors will do is they’ll assign some highstakes assessment given the context that they approached it with. So when I was given this, I went home and I studied with my friends and I made flashcards and we stayed up till midnight and ate pizza and we quizzed each other and then we did this research. So that sort of becomes the course like those things become so hey make flashcards or use notebook LM and here’s something I created for you because so many students don’t know that I never read the book but there’s this book called I think it’s called the unwritten rules of college and I’m like yes that’s it that’s why I was successful and you’re not because there’s all these unwritten rules. So it really is this make these things explicit and then force students to do the work to understand them.

Anand: you know, that’s really a a a great way to start to frame our understanding of where students are and and um what they’re dealing with. And you write quite a bit about kind of the psychology of the learner. Um what else are we missing about the student brain in 2025? You know, what is it that maybe we should use to recalibrate our expectations or how we prepare to meet the needs of these students?

Patrick: So I coined this term um algoi which is about the the your al the algorithm that students are operating by and we use this I use this idea from the creator economy of the click-through rate and this idea that if you’re on a social media feed you get obviously you see videos but I’m going to use specifically the example of ads and so if you get a passive ad where you’re just scrolling and Facebook knows something about you then a good conversion rate a good rate that you’ll click on that that it’ll it’ll earn your attention is 1.7%. So yeah, so most of the time 98% of the time you’re not clicking on the ad that’s given to you. Now a high intent search on the other hand like you’re interested in something. You type in best uh hotels in New York City 5% up to 20% like so 20% would be in like you have created the killer ad once a user enters those terms. So in the best of circumstances, this is the filter that students have been trained to operate. Well, you don’t have to like it, but it’s just the filter. They are conditioned to ignore almost everything. And on the best of days, they’re there’s a one in five chance that they’re going to tune in to what you’re offering them. That’s a high intent search. And so that’s the beginning framework is that we that is the context that we’re operating in. They they know there are better alternatives. They just were watching a YouTube video that explained it or a Tik Tok that explained what they’re about to learn better than you’re about to explain for 45 minutes and they learned it in 45 seconds. So, we are competing in this environment. And so, the goal isn’t to completely um give in to this to to say, okay, short attention spans and we’re going to do this and I’m going to become the creator. But it’s to recognize that we actually do need to build uh cognitive endurance. Um, but that starts with actually earning their attention. Just because they’re there doesn’t mean they’re paying attention. We all know this. Like the only thing we can do is put your phones down. Well, what does that tell you? Like the feedback is I’m here. I’m paying a lot of money for this and I don’t care about what you’re going to tell me. That’s a market signal um that hey, they’re here, they’re paying. How do we close that last gap? And again, Alagoogi to me is this this idea that this is what the environment we’re competing in. And so now we need to do a better job of using principles to engage students.

Anand: Yeah, I could see that that close to 20% maybe for a course in their major, but you know, if you’re teaching a class that’s a requirement, um we’re lucky to get that 2% click-through rate in a lot of the the classes. Yeah. Um that’s a great way of framing it. um you know when we when you say that we need to build this kind of cognitive endurance um so let’s say we get over that first hurdle and we we earn their um their attention um and they they kind of accept that then how do we start to build that I mean in the course of one class it seems like it’s a feudal attempt.

Patrick: So there are techniques though I again we’re do like there’s creators are doing this you are enduring 45 minutes that is cognitive endurance but they’re using psychological techniques techqu to keep you engaged. So they are breaking it down into chunks. They are introducing heruristics along the way. They’re interrupting the pattern every minute or three so that you’re re-engaging. They’re they’re opening loops and closing loops. So this isn’t actually asking you to do anything that is like contrary or uh to your discipline like oh no, I’ve made my discipline interesting. Um this is just these are these are principles that we’re all compelled by. like we leave the classroom and if we’re I mean you’ve all presented at conferences, you’re sitting there, you’re presenting or you’re in the audience and the person’s boring and their slides are lame. You’re not enduring. You’re you’re checking your like you’re literally checking work email during a conference presentation. That’s how boring it is. So, we’re no different than our students. We’ve been equally conditioned. Um, so I think there are simple things that you can do to actually restructure that 45 minutes to at least mimic or start to apply these principles that do work that we know we are ourselves benefit from.

Stefan: Yeah. You know, it kind of reminds me I was talking to my son who who goes to a very large public university and uh he has he has a class where there’s like 400 students. He said, “Well, I go sometimes to listen to the professor read off the slides, but I don’t really know why I go other than I’m supposed to I’m supposed to be there.” He said he also emails out the slides so I can read through them, you know, myself on my own time, right? And then I can imagine the professor standing in front of this class and there are these students like on their phones or their laptops or just like falling asleep and you know this person’s probably getting really angry, right? that you know no one no one is paying attention and uh you know but but it’s kind of easy it’s easy it’s easy to see why and you know there may emerge tools uh that that make this possible right you’re you’re giving us some advice on you know some tools we we could use to you know kind of automatically edit the podcast you know at the end and I wonder if you know there may be some service that arrives that uh you know we’ll put put in these short segments move it around you know these types of things to to kind of increase the user the user engagement because I’ve heard other people say both on this podcast and you know other places that we we only have so much time and we only have so much attention and it’s kind of the attention economy. Um and you know we we are in there uh for our students attention and you know I’ve also told people you know sometimes you’re talking to faculty and they get upset about the phones and I said but you know you’re kind of like think about what you’re upset about right you’re upset that you know these people they’re supposed to be paying attention to you and they’re not which you know in any situation right is is is kind of frustrating. I said but you also want them to learn this content or this skill. I said they may very well just be at home. Like they may go home and watch a YouTube video or they just may look it up on chat GPT so to speak and ask ChatGpt because they realize they weren’t paying attention when you were speaking in class and they have a test on it tomorrow. So I was like, you know, this is not the only opportunity they have to learn this. And I think that we kind of think that, you know, it is the only opportunity they have to learn. But most of the students are still getting the grades if not better grades, right? maybe because of grade inflation or whatever, but it seems that it seems that they’re uh doing really well. So, it it just seems to me this fundamentally about reshaping, you know, how we not not just how we teach, but kind of what our expectations are for kind of what’s happening and especially in the traditional classroom as a block of time.

Patrick: Yeah, if I can I I think you’ve really hit on the the opportunity for us is we we aren’t the bottleneck of information anymore. And that’s an opportunity if you’re willing to look at it. And I and I did want to you you used the phrase several times of paying attention. And I think that’s the idea, right? It it there’s a cost to attention and like the paying attention as a phrase preceded the YouTube economy. So we’ve always been paying and so there should be a cost and our cost should be that we’re going to do our best to present this information in a way that makes sense to the students I have, not the students I want them to be. So,

Anand: Great. Um, when we’re thinking about then the role for students as they’re considering um, you know, what to do about going into higher education and they’re looking at options, what advice would you give to a student that’s graduating from high school? Is it um, knowing some of the the problems and the challenges that higher ed’s facing in dealing with AI? Would you suggest they still go into higher education? would they uh let’s assume that they have kind of the agency to figure out other paths or maybe how to navigate it differently. How would you frame this for the student? Um because I I think you’ve given a lot of great insight into what the instructors could be doing better and what we should be doing. Um but that’s going to take a little time.

Patrick: Yeah. For the student. So the number is $32,000 uh between 18 and 21 and that is invested by the time of retirement just in the S&P going to result in a million dollars and that is the high end of the difference of getting the degree and that is on the low end of what it might cost you to get a four-year degree. So we could just start let’s just start with some some financial literacy here. It’s hugely important. I mean, this is literally a life changing decision. Maybe for the positive or maybe for uh 40 years paying back loans that never uh, you know, amount to job uh positive job outcomes. I mean, 30% of students right now are enrolled in degree programs that are going to make them less money than had they not gone to college. One out of every three students has made a value tradeoff that will cost them money in the long run. That’s just just the facts. I’m not making this up. 40% of students are going to graduate with student loans and not have a degree. I mean, so these are the facts. There are students and we need students to go to school to get these advanced degrees where you do need professional certification and credentiing. There are students that come from wealthier backgrounds and want to continue in a legacy. And that’s a totally a valid value proposition for many families, for many individuals, and for many institutes of higher education. Uh as a as someone who went away to school and then ended up failing and doing well at the community college, I highly recommend if you think school is for you, starting there because that opens pathways. Um, but it really to me I would make the if you don’t know what you’re doing and I think my dad said if we’re asking students what college they want to go to and thinking they have a say, would you as an adult ask an 18-year-old for advice on your career? Like, no. So, why are we letting them make these decisions? Like, this is they they’re just not equipped to make these lifealtering decisions. But so if you’re convinced and you know what you want to do, you should definitely go to school. If you’re unsure, but you think you want to go to school, look at an option that is going to cost you less than $32,000 over four years. And then for everyone else, understand the trade-off that you’re willing to make because there are social experiences and people do have these transformative experiences when they go away to school. Um, and so that is valid if you’re willing to pay the the price.

Stefan: You know, I I thought a lot about that. you know, one one son who who went through college and you know, the the tuition was high. I was like, well, could have I could have bought him a house, right? It’s like where you know what I mean, that that’s life-changing, right? Um you know, was his degree uh life-changing? I don’t know. We’ll see. Um and I I think that, you know, I I don’t think sometimes universities are thinking about like how much this is kind of costing students. And you know regardless in any situation right unless the family’s just kind of able to pay for it you know and without maybe even any stress on their part right that we’re kind of take asking the students to gamble right where if the students are taking on any meaningful amount of debt right there regardless of of their degree path they may choose a degree that has a high rate of return and maybe they won’t succeed in that degree for any variety number of reasons or they may end up with a degree that that that’s not very useful uh to them one way or another Right. So, you know, at least with a home, I mean, you know, unless it burns down and it’s uninsured, you’re it’s a it’s a pretty it’s a it’s a pretty safe, you know, like investment. You can you can always sell it, you know, if something happens and and you can’t continue your mortgage, but uh you know, with with you can’t sell off well, maybe you can make it’s hard to sell off your debt, right? Um at least in any any any way that benefits you. So, I I don’t I think sometimes, you know, they uh the faculty just take it for granted, right? that there’s going to be some enormous positive return on investment that this is just a net gain for students and uh you know it’s not but you know I know too on you know maybe kind of ask a question you thought of maybe some ways that universities might you know want to reconfigure themselves that kind of both would reduce the value of the of the student experience I mean increase the value of the student experience and reduce the cost I don’t know you know

Patrick: yeah I I can just opine with a couple more drop a couple more stats. So yeah, and less than 18% of jobs actually require a bachelor’s degree or higher. Um, so that’s that’s huge. So just knowing that and I think it’s 75% of STEM majors, which is like, hey, this is the positive return don’t work in STEM fields. So really the the stats are about at on a good day, one in five students is going to find a positive economic ROI. So then I get the argument that it is more about these transformative experiences, but still I do want an ROI on that. Like if I’m in saddled with debt for the next 40 years because I had fun for the 840 nights that I spent uh sleeping in a dorm or like in a frat house. Like again it’s it’s an opportunity cost. It’s a cost-benefit analysis. And it is hard though for institutions to market something else. um because their cost structure as I wrote about are the features that they sell like these high this fancy building and this low ratio are what attract people to the institution in the first place. So when you stop doing those things how do you charge the price that you need to charge to generate the revenue to pay for the things that you need? So it again it becomes a really a circular um proposal for a lot of institutions that they can’t escape the the cost structure that they have.

Anand: Patrick, you’ve been so generous with your time. We really appreciate you meeting with us. It’s a fascinating discussion. Uh if you don’t mind, one last question before we go. Um you’ve said a number of times that you know the best time to plant a tree was 20 years ago. The second best time is today. If there’s a university leader or faculty member listening to this interview, um, they’re terrified but ready to move, what’s one thing that you would tell them they should do tomorrow morning?

Patrick: Wow, I feel like I I had all these like stats that like, look, I don’t have a screen or anything, so I just pulled all this stuff out and now you’re asking me this question, which now I’m completely blanking on. Um yeah just I was again it’s sort of stupid but just accept the reality just accept AI is out there it has the opportunity to it is reshaping everything and then just say that I am going to be this educator in this time in this reality and then just try some things just try some things I worked with an instructor who was going to redo their whole course was not going to redo anything and then ended up changing a few things and is now transformed forever is publishing a paper on the the pedagogy that they used and it was because they were willing to try one thing. Just take the first step. That’s it.

Anand: Great. So much but sorry I didn’t have GPT to give me any responses there.

Stefan: They probably would have said the same thing.

Anand: Only the best model would. Yeah. Yeah. Patrick, this has been fantastic. Thanks so much for taking the time. And for anybody that wants to find out more about your work, where can they find you?

Patrick: So I’m on uh LinkedIn at the Patrick Dempsey and then same on Substack the SEC I publish under the second draft on on Substack.

Stefan: Great. All right. Thanks a lot. Thanks so much. I appreciate it.