Speakers
France Hoang: Guest, Founder and CEO of BoodleBox
Anand: Host/Interviewer
Stefan: Host/Interviewer
France Hoang (0:00): We’ve been educating largely the same way for hundreds of years. AI has completely upended that because AI is about productivity, but learning is about productive struggle. And those two things are worlds apart. Chat GPT wants you to produce a result. It doesn’t care the process. It doesn’t care whether you struggle. In fact, it doesn’t want you to struggle. It wants to make it as easy as possible. How do we return productive struggle to the classroom? The first two prongs of what I would call AI readiness, the domain expertise and the AI enablement: knowing when and how to use AI, being better than AI. The third prong is what I call human excellence. It’s it’s being better at things that make you uniquely human. Things like collaboration, creativity, communication, critical thinking, and having this entrepreneurial mindset. Why I have a really hard time with academic integrity and catching students using AI, I think it’s a very harmful approach. It makes learning a fearful activity, which we never want. We want a love of learning. Encourage experimentation. Um, this AI should not be top down. There’s no right or wrong answers, right? There’s just experiments. Give them lots of room to make mistakes. Make mistakes mandatory.
(Main Intro Music)
Anand (1:09): Today, we’re really pleased to be joined by France Hoang, a remarkable leader whose life bridges public service, national security, law, and technology entrepreneurship. Born in Vietnam and raised in the US after arriving as a refugee, France graduated near the top of his class at West Point, served as a US Army officer and White House council, and went on to co-found and build multiple successful companies across defense, intelligence, and technology sectors. Now as the founder and CEO of BoodleBox, France is turning his attention to education, helping colleges and universities navigate the age of AI with tools that make learning more transparent, equitable, and human-centered. His work at the intersection of AI, edtech, and workforce readiness offers a unique vantage point for how higher ed is navigating the disruption of education by AI. We hope you enjoy this discussion with France Hoang.
Anand (2:01): We’re really excited to talk to you France. Thank you so much for taking time with us and just want to start a little bit with your personal background. It is so striking and uh really interesting and would love to hear just a little bit about how that background really informs the work that you’re doing now with educational technology with AI products and higher ed. Whether it’s you know your background being a, you know, a child refugee to West Point to military service to White House council, um you have lived so many different experiences. I imagine that probably gives you a different vantage point. How do you think that informs your work?
France Hoang (2:33): Uh Anand and Stefan, First off, that’s a very nice way to say I’m unemployable. So, thank you for that. Uh, also, more seriously, it’s just a... it’s great to be able to spend time talking to you all. Um, I I’ve admired both of yours work in this field of education and AI and being thought leaders and asking hard questions and being out there. So, I I’ve been looking forward to this conversation as well.
France Hoang (2:58): Um, I I do joke I’m unemployable at this point in my life. I’m a, as you mentioned, I’m a refugee from Vietnam. I grew up in Washington State. Um, I wanted to give back to this great country. So, I ended up going to West Point. Uh, graduated from there, went to Ranger School, served five years in active duty. Um, was a peacekeeper in Bosnia. Uh, then I... my first career pivot, ended up going to Georgetown Law School. Uh, full disclosure, I’m not a very good lawyer. Uh, but I still ended up working in the White House for a couple years as a lawyer to the president. And then because my risk tolerance was high, I ended up going back into service. Um, end up deploying to Afghanistan as a member of Army Special Forces unit, served in combat. Uh, came back and became an entrepreneur and have started multiple companies, a law firm and aerospace company.
France Hoang (3:46): Um, after I had a successful exit for one of those companies, um, I started serving on for-profit and non-profit boards. But I’ve always had a love of education. You know, if I had to pick three things that kind of have driven me in life, it’s a desire to serve, it’s my love of education, and it’s my deep belief that, you know, humans are at our best when we collaborate. And I certainly wouldn’t have had all these opportunities in life if it wasn’t for the power of education. You know, I’m... I literally came over on a plane and lived in a tent. All right. And then working in the White House... and only in America, but only as a result of American education as well.
France Hoang (4:22): And so, um, I’ve had a chance to be a trustee of a university and and BoodleBox now, you know, this this latest venture was really born out of a desire to to give back. Um, and this belief in human-AI collaboration. Um, I think we’re at this really interesting moment and Stefan and Anand, I would love to hear your perspectives on this as well. You know, one of my favorite uh TV series is The Men Who Built America. It it talks about the uh the Gilded Age 100 years ago, the ways that railroads connected us and and oil fueled new industries and manual work was changed by the assembly line. Where we’re in another Gilded Age now, right? But it’s it’s the internet connecting us and it’s data fueling, you know, industries and it’s mental work now. It’s intellectual work being automated by AI.
France Hoang (5:10): And so education I think um always has to adapt to the times, right? The assembly line changed the way we train people, right? They went from being artisans and apprentices to... we now have industrial engineers in a whole different field. Well, we’re seeing that same thing happen with intellectual work. Um, but I will say that I’m not 100% certain that AI is going to be a net positive. I think that is still a question mark and it’s I think it’s up to... up to us. Um, certainly you know, Stefan and Anand, you are doing your best to make sure it’s a net positive, the discussions you’re having, and I want to be a part of that discussion as well.
Anand (5:48): Well I think we’re probably on the same page. I’m not sure it’s going to be a net positive but uh that’s one of the reasons why we need to have these conversations so that we can try to influence as best we can.
Stefan (5:56): Yeah. Yeah. And I think you know regardless, even if it’s a net positive, there are going to be a lot of things that are very difficult um and that are very negative. And the only way we’re going to kind of... kind of minimize like those... those negative impacts is if people work together. And I I think the first part is kind of acknowledging the reality of like the challenges we’re facing. And then uh, you know, then directly kind of engaging it. You know, and I kind of just wanted to say is, you know, a personal note of thanks, like you and Zach especially at BoodleBox have always kind of supported my trial initiatives with many students uh, you know, both to kind of get them some familiarity with AI and some of the projects we’ve done.
Stefan (6:36): But, you know, one thing that’s kind of really important to me is kind of... I think one of the... the best chances we have is to minimize the work disruption, so to speak, is kind of to move on to a little bit more of an entrepreneurial model, at least for some people when... where people are kind of working with AIs. And that’s one thing that’s like especially uh, you know, unique to your platform. It gives many students kind of like an initial opportunity to see what it’s like to work with some different AIs, both in terms of the general AIs you have in the platforms but also in the... in some of the specialized uh bots.
But, you know, I know... I know that’s kind of foundational really, you know, to... to a lot of what your product is. And, you know, there’s a lot of faculty concerns obviously about academic integrity and, you know, those types of things. And I... I think it stems from, you know, kind of giving the students the... in some instances the same assignments that they were given before November 30th, uh, 2022 when these... when these tools became readily available. But, you know, I know you want to kind of build in some transparency, build in these work processes to kind of help them kind of move past uh, you know, kind of one of... one way of kind of assigning that. And I just wondered if you’d maybe kind of speak to kind of, you know, how you address kind of these issues with faculty and some of the unique parts of your platform.
France Hoang (7:50): Yeah. Um, so we’ve been educating largely the same way for hundreds of years, right? I give you some piece of information and then I want you to critically think about it and comprehend it and then you demonstrate that by producing an artifact of learning. The essay, take a quiz, produce a project. Um, AI has completely upended that because AI is about productivity, but learning is about productive struggle, and those two things are worlds apart, right? Like Chat GPT wants you to produce a result. It doesn’t care the process. It doesn’t care about uh annotating. It doesn’t care whether you struggle. In fact, it doesn’t want you to struggle. It wants to make it as easy as possible. But as educators, right, I want a student to struggle. I want them to break it apart. I want them to critically think.
France Hoang (8:41): And so, how do we solve this conundrum in the classroom? So, how do we return productive struggle to classroom? So, first off, I think we have to acknowledge there’s teaching with AI and there’s teaching about AI. And those are two different things. They’re related, but they’re not the same thing. So teaching with AI can help students develop what I call domain expertise. You have to be better than the AI to use AI.
France Hoang (9:02): I was... I just um before this call I was on with a law firm partner and we were talking about uh the impact of AI on the practice of law and one of the comments was, you know, because AI hallucinates, everything AI does has a 10 to 20% error rate. If you produce a partner-level AI, it doesn’t make everyone a law firm partner. In fact, all you’ve done is created an AI that’s actually only useful to law firm partners because only a law firm partner can identify the 10 to 20%. It actually makes law firm partners more valuable, not less valuable. And so, as educators, the only place a student can get domain expertise is in our classrooms. So, we’re actually more valuable, not less valuable as educators in an era of AI where domain expertise is important.
France Hoang (9:44): So, the first thing we have to do is create this domain expertise and we can do that with teaching with AI. And in BoodleBox, we have created a whole set of tools. You know, you can build tutor bots, you can create simulations, you can um have your students use not one but multiple AI tools. Um as Zach loves to say, this is not about learning a tool, it’s about learning a technology. And so you can give the opportunity for students to engage in productive struggle with different AIs and be good consumers of AI. And so I think that’s number one, do their main expertise.
France Hoang (10:15): The second... in... in order to learn about AI, um faculty, right, need to have the opportunity to see how students are using AI because you can’t teach something you can’t see. And so part of the early development of BoodleBox was an understanding that process mattered as much as product. And only by seeing the process as well as the product can we measure progress. And so we talk about process, product, and progress as being equally important.
France Hoang (10:44): And so in BoodleBox, when you use... a student uses BoodleBox, they can share their work with the professor in a way that they can see the process by which they produce something. And the students... the professors can assess the student’s progress in using AI. Um, without that, all you see is the product, right? And that I think the early outcries against AI were because all the professors saw was the product, and that’s no longer a measure of whether the students engaged in any productive struggle.
France Hoang (11:08): We’ve recently taken that up a notch and introduced something called AI Assignments. Think of that as a shared learning environment that you control, where you get to... instead of this wild west where the students are using all kinds of AI, I can create a sandbox where they can only use the AIs I authorize them to, or I... only the AIs I create, or all the AIs. It’s totally up to you. But a professor controls that and I can see all the student work inside that sandbox. And so we have solved the dual conundrum of student choice as well as educator control inside of BoodleBox.
Anand (11:45): One thing I really like... the... the way you’re characterizing this, that the productive struggle is so important. Um, and this process being equal or perhaps even more important than the product is a really important turning point in the way that we think about authentication and assessment. The other part of this that really intriges me though is that um it really ties into kind of teaching an entrepreneurial spirit. Um, and... and you’re more of an entrepreneur than... than almost anybody else that I know. So I’m... I’m wondering if you can kind of speak to this. Thinking as an entrepreneur, what is it that we can do to be able to instill those values, to be able to teach that process, to have that critical entrepreneurial perspective for students? Is it about process? What is it about the productive struggle that helps instill that? Because it really seems... and we... we’ve talked about this before and I know that... that you’ve... you’ve talked about this. That’s an important perspective for students to have in an AI world. If you’re not going to be entrepreneurial, you don’t have that same intrinsic motivation, that drive that’s going to be necessary.
France Hoang (12:40): Yeah. So, we talked about the first two prongs of what I would call AI readiness, the domain expertise and the AI enablement, knowing when and how to use AI, being better than AI. The third prong is what I call human excellence. It’s... it’s being better than things that make you uniquely human because those are things that have long-tail value in today’s world where increasingly we can outsource to AI. So things like collaboration, creativity, communication, critical thinking, and having this entrepreneurial mindset.
France Hoang (13:11): Um, by the way, one of the things I think sometimes holds us back as educators, and I’m... I’m just as guilty of this as anybody. I love being the sage on a stage, right? I love having all the answers. I love students asking hard questions and me demonstrating my mastery of this subject by giving this beautiful answer that lights them up and they walk away with their head exploded. Right? It’s scary to have to stand up in front of students and say, “I don’t know and you don’t know and we’re going to figure this out together and we’re going to make mistakes together.” I think that’s a... but that’s the position we’re in with AI.
France Hoang (13:43): Um, you know, we have... we have hundreds of schools that use BoodleBox across tens of thousands of users and we’re really privileged to be able to partner with educators and have them share their experience with us. By far the educators that have the best experience in the classroom of AIs are those who take a very much um shared learning approach. I’m learning, you’re learning, you’re going to make mistakes, I’m going to make mistakes, and that’s okay. And let’s do this together in an age of AI.
France Hoang (14:11): Um, in terms of like how do we go about doing that in a classroom? Um, I recently spoke at um a conference down in... in Alabama and it was to a bunch of administrators and they asked the question of like, what are the... what are the critical things we need to do to implement AI in our institution? And I gave them kind of three thoughts. The first is encourage experimentation. Um, this AI should not be top down. There’s no right or wrong answers, right? There’s just experiments. There’s just learning. So let a thousand flowers bloom. Give your students and professors a bunch of tools. Give them guardrails as well, but give them lots of room to make mistakes.
France Hoang (14:48): And in fact, that’s the second piece of advice, which is make mistakes mandatory. Make it required to do things that break. Do... make mistakes. It’s okay. It’s why I have a really hard time with um this idea of kind of like academic integrity and catching students using AI. I think it’s a very harmful approach, not just to the student... not just to the student-professor relationship, but also to the student approach to learning, right? It makes learning a fearful activity, which we never want. We want a love of learning.
France Hoang (15:20): Um, so encourage experimentation, make mistakes mandatory. And the third prong is share what you learn. Um, the institutions that have the best AI adoption are those where there’s a lot of peer learning occurring. Um, Pima Community College, um, there’s a wonderful um set of uh instructors there. They uh once a month on a Friday, they have a brown bag lunch. They come in and they just gather in a room and they just share what they’re using with AI and... and here’s use cases. Here’s my prompts. Here’s the bot I built.
France Hoang (15:50): Um, what’s absolutely amazing to me is how different everyone uses AI. Like I think there’s a natural assumption, like I use AI and everybody else uses sort of the same way. Totally not true. Like it... it is... I bet you if you looked at my prompt logs, you would be like absolutely astounded. Not because I’m amazing, but you’re like, “That is so different than how I use AI.” And I... same thing if I looked at your prompt logs.
Stefan (16:16): The... the other thing that you know kind of we’re at the... we were at the conference uh you know over the weekend Anand and I were at the liberal arts... and Zach spoke. And I think the other thing you know that’s really important uh for educators to understand is that they need to work through this process with the students, right? Because if they’re... they’re just assigning a paper, right, then, you know, it’s like, okay, well you go BoodleBox and you can pick one of the bots that, you know, you may use and, you know, you kind of write the paper. But you need to... you need to work with the students.
Stefan (16:43): And I think that’s a lot of what the... the educator brings, right, be... beyond the specialized knowledge, right, that you’re... you know, you’re kind of pointing out the, you know, the AI is still hallucinates too much to, you know, entirely rely on it. But the idea of getting the students to work together, which a little bit, okay, they’ve done group work. You know, we had group work and that doesn’t really change. Either professors said like, “Oh, we’ll do this project in a group,” right? They didn’t really kind of give you much guidance. But now we’re adding in the... the AIs, right? And kind of working through the students with that. If you’re going... I assigned once that group chat and it’s like, “All right, they’re chatting in there.” You got to... you got to monitor that chat. You got to remind the students to kind of follow up with the chat. You’ve got to teach them about... because you know, a lot of them, they just use Chat GPT. And you know, the other bots, like they... they have a lot of value. And you know, what you’re bringing in with the specialized bots that you kind of build in for people to use, or they want to build their own bot, right?
Stefan (17:34): And there’s a lot of kind of I think learning then, not just the content, but you’re kind of learning this new way to work and you’re learning this, you know, way to be entrepreneurial and talking to the students about it. And I think that’s a lot of what people bring. But I think sometimes people want to... you know, they... they want to kind of keep the... “Hey, they’re doing it” and then use the AI or use BoodleBox and think that’s just going to solve all their... their problems. But it requires being pretty proactive.
France Hoang (18:01): Yeah. I... I over the summer I heard a speaker, I won’t name the organization, but it’s one we all know. They were responsible for implementing AI across 300,000 users and it went terrible. Uh, there was a small group of folks that were early adopters, ducks to water. We probably fall into that bucket. And they see the prompt window and they see a world of possibilities. But most people are busy, they don’t understand the use cases, they don’t know where to get started. And so they really had to tie the use of AI to actual problems that needed to be solved. And that’s when they started to get adoption.
France Hoang (18:35): But then what they realized over a longer period of time is that’s fine for now, and we need to start there, right? We need to find early wins and easy use cases for AI because it should not be a solution in search of a problem. We should have problems that it actually solves natively and well. But if we really want to get the full um potential of AI realized... they... what they said was you really need to start rethinking thought... like workflows and processes in an era of AI from the beginning to end.
France Hoang (19:04): So put it another way, a lot of the use cases of AI right now is what I would call automating the past, or at best automating the present. Here’s the way we’re doing it. Where can we stick AI? Great. Look at this amazing use AI. But what we really should do, and look, I’m not saying this is where you start because this is a big lift. And... and this will probably scare away a lot of folks from AI, but ultimately I think we need to back up and say, “What should this look like in a world of AI from start to finish?” Right? In other words, I mean, here’s a good example, right? Yes, we should all learn to write essays and yes, essays are important, but essays as a diagnostic tool, are there better ways to diag... you know, to evaluate and assess a student in today’s era of AI? So, instead of fighting over the essay, maybe we should do something else instead. That’s just a very small example of reimagining the future versus automating the past.
Anand (20:00): I think you really hit the nail on the head with this and I see a lot of this happening even with small businesses thinking about how do we incorporate AI to do what we’re already doing and do that better. And that’s always the measure on the return on investment. So you’re right, it’s a heavy lift to think about, “Okay, let’s totally re-envision the... the whole enterprise of what we’re doing,” whether it’s in business or in the classroom or even just the institutional work that’s being done. But let’s say you... that they follow this, you have your... people start to envision it this way, they start to think top to bottom, how can AI revolutionize what it is that we’re doing in the classroom. Then looking ahead 5 to 10 years, what do you think would be the most transformative ways that AI would reshape higher ed teaching and learning? Or maybe even just with uh in terms of running the institution? How... how do you see that playing out?
France Hoang (20:50): Yeah, a couple things. One, um so we have uh... I’ll tell you a story and then I’ll answer the question. Um, we have a wonderful intern um this past summer at BoodleBox. Uh, she came from one of our uh institutions that was using BoodleBox. Uh, she is... I mean, we’re all AI immigrants at this point. You know, no one’s kind of completely grown up, but she’s pretty close to being a native. Like she’s been using it... when she was probably 16 when Chat GPT came out. So she’s been using it for a fair bit of time.
France Hoang (21:18): I remember giving her assignment of um, “Hey, we have a user interface. We want to get... we want to make this optimized for educators. Can you get some feedback on it?” Um, I thought she would go look at our user list, reach out to a bunch of educators, schedule calls. That’s not what she did. Her first step was actually to create over a hundred AI bots that simulated a wide variety of educators with different backgrounds and different needs. And she fed the UI to those bots and consolidated that feedback and iterated multiple times before she reached out to a human educator.
Anand (21:58): Wow.
France Hoang (21:59): So it wouldn’t even occur to me to have done that, right? Yeah. So the younger generation that is growing up, right, with AI, for them, they are natively reaching to it and they’re not using it the way we used it because we’re frankly a bit prejudiced by dozens of years of using Microsoft... and a couple decades of using Microsoft products, where we tend to look at software as an answer engine. They’re using it as a collaborator, as a thought partner, right? Not “give me the answer,” but “how do I find the answer?”
France Hoang (22:35): And so I think education, right, if we think about it, um, education, so to your point, right, process matters as much, maybe even more than product. Maybe what we need to equip students with are the ability to... to have more processes at their disposal. And so what could that look like? Right? In the short term, I think um one of our goals at BoodleBox was to create a portable platform. What does that mean? It means that when a student shows up at a BoodleBox um institution, uses BoodleBox, and they get access to it for the next two or four years, they will fill it with their prompts, their conversations, their chats, their collaborations, the chatbots they build, all of the materials from class, and when they graduate, they can take it with them.
France Hoang (23:19): So they have an AI toolbox for life. So somewhere back behind you, Anand and Stefan, there’s a... there’s a shelf of books from graduate or undergraduate school that you probably haven’t opened in a while. Imagine if that all lived in an AI platform and you could use that for the rest of your life, right? And so they... I think we are going to have a personal AI. I won’t call it an assistant. It’s more like a set of tools, the same way we have a personal email. And part of what education is going to do, I think, is teach us how to use that wisely, use that responsibly, as well as build it as part of our educational journey.
Stefan (23:58): Yeah. And that’s kind of really, you know, gets you thinking, you know, to back up a little bit. I remember, you know, I tried it myself first, of course, when it came out, but then there’s a New York Times article about... Anand and we tried to... we tried to track the guy down for an early interview. We had this article in the New York Times about like catching a student, right? Writing uh... writing with Chat GPT-3.5 at the time. And you know, we kind of had that, you know, big initial negative reaction, you know, and not really seeing where it was going in some instances and just worried about stopping the essay. But, you know, kind of been involved, you know, really from the outset. How would you, you know, like for the 10, 20, 30,000 ft view, maybe... maybe even from outer space, like how... how have you kind of seen it at a macro level, kind of attitudes change? And kind of maybe the different ways maybe you were approaching uh more people initially, but then, you know, they realized, okay, you were out there, so that changed, they approached you, but also they realized they... they started to need to make changes rather than from that initial, “Well, we have to just stop the students from using uh AI in their essays.”
France Hoang (25:03): Yeah. Mike Kent, who’s wonderful, talks about educators having gone through the five stages of grief with regards to AI in the classroom. And I think we’ve all seen that to certain extent. Um, it’s not going away. Um, I think there has been a realization as the technology has both gotten better and we’ve gotten more informed and educated about the technology, uh that it can be helpful, right? Not just... before all it was doing was substituting for student labor and producing results that causes all kinds of headaches in terms of our assessment. But all of a sudden the first time a professor can revise a syllabus, the first time a professor uh creates a simulation, or the first time um a professor creates a tutor bot that... that, you know, helps the student 24/7... um the first time they run an uh a group assignment where the students are jazzed because they’re getting personalized, differentiated, active learning... Um the... before it was all negatives, now there are some positives, right, Stefan?
France Hoang (26:05): And so I think there’s now more open-mindedness um at the best, or maybe the resignation that it’s not going away. But either way, there’s an understanding that this is something that the... the train has left the station and I need to uh get on in some way. So I think attitudes um have changed. Um, there’s also been tremendous work by the academic community, right? There is a whole list of folks that you and I know um that have done great work in terms of, you know, Mike Kenson grading the chat, you know... um there’s been great writing by um, you know, José Bowen, and there’s a... there’s a bunch of educators who have been at the fronting edge... front edge of this and... and from an educational... educator standpoint said, “Look, here’s the concerns, but here’s the opportunities” and have educated their peers. And I think that’s been um... that’s been really powerful in helping gain acceptance.
Anand (27:01): One thing that I’ve... I think I’ve heard you speak to and a lot of other people have talked a bit about is kind of sustainability and also responsible AI. Um, how would you operationalize that? When you... when you talk to educators, when you talk to administrators, what does it mean for them to consider responsible AI, whether it’s in practice, the use of the platform, the development of the platform? Um, what... what does that mean and then how does that get carried forward to the students?
France Hoang (27:26): Yeah. Um, I think responsible AI means whatever you want it to mean, but there’s lots of different definitions. How’s that for a lawyer answer, right? Lawyer... But I think there’s three things... I’ll tell you in our, you know, hundreds of conversations, three things kind of repeatedly come up.
France Hoang (27:42): Um, the first is one I was surprised by at first, but I’m not surprised anymore. It’s the environmental impact. Um, there’s a lot of concern about water consumption and energy consumption. Um, at BoodleBox, uh, we figured out a way to run large language models such that we utilize up to 96% less tokens to generate a chat than would otherwise be required. That’s 96% less cost. That’s 96% less environmental impact. That’s 96% less energy consumption. That both allows us to reduce the impact of this use of the... of the technology, but also greatly reduces our operating cost, and so we can make the platform affordable. Um, that comes up a lot, a surprising amount.
France Hoang (28:27): Um, second, related to that lower cost, um, I think part of being a responsible institution um is addressing the problem of equitable access. Um, almost every professor has had this experience if they allow AI in the classroom. Very quickly they realize there’s a subset of students whose outputs and outcomes are just better on a consistent basis. And maybe it doesn’t align to who they thought it would be. And when they scratch the surface and investigate, it turns out those students are paying for AI, you know, because they have resources, they’re paying for access to better tools, and they’re getting better outcomes. Well, you know, that’s probably not the world we want to live... in fact, I know it’s not the world we want to live in, right? And so providing all students, not just staff and faculty, but all students, equal access to the same great set of AI tools, um, I think is part of being respon... responsible AI at your institution.
France Hoang (29:15): Um, and the third thing is uh just being mindful of privacy and security. Uh, one of the reasons why institutions pay model providers uh is uh a guarantee that they won’t train in their data. Um, well you don’t have to... you don’t have to pay BoodleBox for that. That’s just built in. We don’t train on anybody’s data. We don’t allow your data to be used for any training. In fact, we anonymize all the prompts. Um, and we don’t send entire documents out ever. We... we use a system to selectively only pull the... the relevant portions. Um, so I think being responsible is all... mindful of what’s happening to the IP of your institution, of your educators, of your students. Um, you know, as the saying goes, if you’re not paying for the product, you are the product, or in this case, your data is.
Stefan (30:00): But you’ve been working with some institutions. I imagine sometimes too, you know, and you kind of alluded to it earlier, you know, they’re... they’re asking you kind of what... what to do, right? And I think sometimes people oversimplify it. You know, they’re like, “Well, what... what are the five steps, you know, I need to... I need to take, you know, in my 50,000 student school district or, you know, 20,000 student university uh to, you know, to do what I need to do with AI?” And it’s not kind of that simple, but are there some like kind of steps... like broad steps uh you suggest to people who are maybe kind of, you know, approaching you uh you know, a little bit like deer in the headlights?
France Hoang (30:37): Yeah, I think it depends on... no two institutions are going to have the exact same solution, right? Um, every institution is different size-wise, student population wise, where their faculty are, the governance structure. Um, I would say I think the most successful implementations of AI are done um in a shared way. What do I mean by that? I... it’s one where the administration, the faculty, and the students are all involved in some way. Um, you don’t want this to be top down, right? Um, it’s... it’s... if the CIO makes an independent choice and just kind of makes something or mandates it, that’s not going to be well-received. Um, equally, right, if the faculty choose a solution but don’t engage their administration about things like security and privacy and sustainability and accessibility, that doesn’t go well. And I’m also just continually shocked at how little students are invited to join this conversation, and they have the most at stake and they should be at the table as well. So I think number one Stefan is just involve everybody, right? Make this a shared... shared uh activity.
France Hoang (31:45): Um, second, uh it is uh I think patience uh is key. Um, for the folks who are implementing AI, I think there’s a belief that this is such amazing game-changing technology that it’s instantly going to take. Uh, adoption always is slower and takes longer than you expect. Period. Period. And we know this because we have the data. We can see it. Um, and so setting realistic expectations, right? If you, you know, you don’t want to be the faculty member that goes to your dean and says, “Hey, I want AI access for the platform and within a year we’re going to have 80% of people using this and we’re going to see massive returns on investment.” What you’re going to have is a very disgruntled administration when they actually ask for proof of that, right? It’s... you got to think long term here.
France Hoang (32:37): Um, and the third is uh as I mentioned before, right, just giving yourself um in your institution uh the permission to experiment and make mistakes and to learn from it. And I think that’s key. So those three things, right? Just um, you know, make this a shared decision, um you know, make sure that you have patience, and then you know, be open to learning and all that goes with it.
Anand (33:00): Yeah, you make a really important point uh in that last answer about the role of students uh and their own advocacy within this process. One thing that I found surprising and you already kind of alluded to this, there are surprising responses from students sometimes about concerns about the environment, certainly other concerns about copyright, creativity, uh and those in the arts. And I’ve heard that from students. There are a small subset of students that are even saying they want to be able to opt out. They don’t want to have to use AI in school. And you know, we’ve had some discussions... I’ve had discussions with other administrators, faculty about... a bit about this and that’s a tough one because, you know, I think that... that you can teach them about AI perhaps without forcing them to use AI. Um, I think that uh there’s something there though that... that maybe seems to be missing. You know, how would you... how would you advise a school when they have um students that are saying, “Look, because of environmental concerns, creativity, copyright issues, we don’t want to be forced to use this technology. We want to learn in the class without using AI.” How do you address those students?
France Hoang (34:06): Oh, it’s a great question. Um, first off, there’s a number... I... I... there’s been an interesting phenomenon where the best performing students are often the least likely to use AI.
Anand (34:14): Yeah.
France Hoang (34:15): Uh, for a few reasons. One is they’re the most concerned about affecting... being accused of academic integrity issues. So, they... they opt themselves out. Second, they’re genuinely concerned about their learning and they feel like AI undermines it. And third, there’s a bit of pride. “I’m a great writer. I’ve worked hard to be a great writer. Why am I... if I’m... why am I gonna outsource to AI?” So, ironically, those students may graduate less prepared for an AI-enabled world than the student, say the more average student who’s like, “I have no problems with it. I’m going to embrace it and find out how to use it responsibly.” So, interesting phenomenon.
France Hoang (34:50): Now, to your question though, um, look, I believe in academic choice and academic diversity. So, um, if a professor doesn’t want to use AI... I get asked this a lot. “Hey, what do you do about a professor who doesn’t want to use AI?” Then don’t make them use AI, right? Like that’s fine. And that’s great because I think students should have the choice. Um, there have been a number of institutions that label their um their course catalog. “This class does not allow AI.” “This class, you can opt in, it’s optional.” Or “This class requires AI.” That’s great. Then students can make a choice. And then when they show up to the classroom, there’s no issue. They know, right? They’ve already opted into a class that either allows AI or requires AI. And they have the tr... they... they had the choice to opt into a class that didn’t allow AI.
Anand (35:38): Yeah. But push back on that a little bit. Let’s take the... the... the other position that my role as an instructor is make sure that I’m concerned with student readiness. And if I’m really convinced the world is going to be AI inflected in pretty much every aspect of... of society... and... and some of us may be off the deep end that way. Stefan and I kind of fall into that category. Shouldn’t I then feel compelled to make sure students are learning about it and using it at some level so that they have some fluency with it? I mean that... is... is that justified? Or... or how do we... how do we...
France Hoang (36:08): Yeah, I think that’s justified. But my point is... or I think what the approach... not even my point... the approach other institutions have taken is they just let students be aware of that.
Anand (36:18): Yeah.
France Hoang (36:18): Right. Before they sign up for the class. So if they... if the student disagrees with that, then they don’t take that class.
Anand (36:25): The transparency.
France Hoang (36:26): Yeah. It’s just transparency and student choice.
Stefan (36:28): And you know there’s only, you know, so many... pick your battles, right, as they say. I’d really like all the students to use, you know, at least maybe even on one assignment, right? Teach a little bit of AI, but if somebody feels that strongly, I mean, you know, we’ve all kind of uh... we’re all old enough, or at least I’ll speak for myself, you know, you kind of change your mind on something over time or you know, you come to kind of see it um in a different way. (coughs) Excuse me. And that’s probably some of the ways uh you know, to move forward. Of course administrators are, you know, kind of... you know, you have faculty who kind of just refuse to use it and, you know, those types of things and, you know, you’re creating... you’re creating, you know, some opportunities uh for... for moving forward. So I... I think, you know, it’s kind of... let... let a thousand... thousand flowers bloom. You talk about experimentation. You know, some of the experimentation’s going to be uh, you know, be without AI, right? So, you know, I’d like to see everybody use it, but I... I realize that, you know, there are some limits there.
France Hoang (37:26): Yeah. Now, to your point though, right? Like how many calculator-only classes... math classes are there, right? Like you know, yeah, we kind... the academy has kind of decided like there’s no value in providing, you know, like... “I want to,” you know, there’s not many students opting in for “I want to do all my division longhand,” right? So, you know, I think this also may be a product of the time and place we’re in, in a bit of a transition.
Anand (37:48): Absolutely. I was thinking the same thing. I mean it’s a... it’s a messy transition as everybody’s figuring out what the norms are, um how it’s being used, what’s appropriate, not appropriate, and we see that change month by month almost. Uh, with your experience in dealing with so many different institutions, uh I’m wondering if there are any novel applications that you’ve seen, may... maybe interesting ways that schools have approached or maybe students have approached, you know, kind of like the... the example that you had of the intern and having a very different perspective on employing AI and doing something interesting. Do you... some examples you can share that maybe others can... can learn from.
France Hoang (38:22): Yeah, I’ll do two. So, um, Christine Colberg, Point Loma University, she teaches marketing. Um, she has her... she does the typical lecture. Students have to take notes. That hasn’t changed. What has changed is she requires her students to build a bot and to upload all of her notes... their notes into the bot. They have to create a marketing assistant. And then her... her assessments aren’t quizzes. They’re not essays. They’re real-world problems that the student is required to solve in collaboration with their AI assistant that they created. And... and they have to create their own individual bots. So, it’s not like one bot that has everybody’s notes or her notes.
Anand (38:59): So, they’re encouraged, they’re incentivized to take really good notes because... make sure their bot’s trained.
France Hoang (39:04): That’s right. And then as the class goes along, they do that individually and my understanding is later on they do it in a group, that they will collaborate with other students and other students’ bots that were created to solve real-world problems. So when that student graduates, they have learned marketing, they’ve learned how to train and build an AI assistant, and they’ve learned how to collaborate individually and with other people with AI assistants to solve a real-world problem. How much better prepared is that student than someone who takes a traditional marketing class?
Anand (39:36): Yeah, that’s great.
France Hoang (39:37): So, that’s one example. The second example is at West Point. Um, West Point was one of the very first institutions to adopt this amazing novel technology in 1820 called the chalkboard. Um, literally, it’s true. Um, but they didn’t view it as just a teaching tool, but they also viewed it as a learning tool. And so, there’s this fine tradition at West Point of what’s called “taking boards.” So, you do your homework on your own. You’re supposed to self-study. You show up to class and then the professor says, “Everyone take boards.” There’s a chalkboard behind every seat and they’ll call out a problem, “Problem five.” And you have to write your solution to problem five on the board. And then you can see everybody else’s work. And then they’ll... the professor will call on you, “Hey Cadet Hoang, brief your solution.” And then you have to sit in front... you have to stand in front of your... of your peers and brief your solution to the problem. So the flipped classroom, 19th-century style. Pretty awesome.
France Hoang (40:28): So there is um... there’s at least a couple professors at West Point who have adopted that for the AI age. So the students still have to do work on their own, but they’re allowed to use AI, and then they submit their prompt log to the professor. And then the professor brings the students in and instead of taking boards, the professor shows a conversation, the prompt chat history, and then has the student brief how they used AI. “Why did you pick that bot? What prompts did you use? Do you agree with the bot’s answer? How do you know it’s right? Did you verify it? Is this a hallucination? How do you know it’s not a hallucination?” So now it’s the flipped classroom of AI, where the student is having to def... explain and defend their use of AI in front of their peers. And that’s a great example of developing that domain expertise, that AI enablement, and that human excellence all at once.
Anand (41:15): And that... that back to that emphasis on the process and sharing that process with others. Um, that first example of building your own bot with your notes, it kind of reminds me of years ago a student taking great notes in class and copying them and selling to other people in the class. Maybe the new model is build a bot that you sell subscriptions so other people can use it to get ready.
France Hoang (41:37): Yeah. Yeah. Yeah.
Stefan (41:39): You know, the one other thing, you know, we... we kind of started on this... you’re, you know, you started when... when you mentioned, “Hey, I’m not... I’m not certain, you know, if this is going to be net desirable or not,” right? And I think anyone who’s... who’s kind of thought of that, right? Like, uh, you know, has these concerns. You know, today, I don’t know if you saw it, there was another Future of Life uh Institute uh letter saying, you know, before we kind of move into super intelligence, like they’re just kind of calling for, you know, people to be aware, right? Um, to... to raise some awareness. And I... I think in some ways, you know, professors who don’t want to teach with... teach with AI, right, it’s almost like, you know, they could do... accomplish a lot of raising kind of awareness of these issues and the students learning about AI, even if they taught a course against AI, or most of the assignments are, you know, involve some things... it would kind of really raise discussions of the issues. Because, you know, we’ve talked a lot about using AI, but I think these are kind of some of the... it’s as important that some of these other issues, you know, get raised and, you know, beyond energy and intellectual property. I mean, this is probably going to restructure our economy, to restructure our understanding of who we are, you know, warfare, everything. Um, and I’... I don’t know. I’d like... I’d like to... I’d like to see a little more discussion of that, especially, you know, the humanities.
France Hoang (42:53): Yeah, it’s uh... there are some big questions uh to be asked and I don’t know if answered, but at least debated and discussed. Um, but uh one of the reasons I’m very passionate about what we do is I really think that um if you want to affect how AI is used and if we want AI to be responsibly used, then we need to start where tomorrow’s workers are today, and that’s the classroom. So I talk about classrooms as being the front line of... of human-AI interaction. Like that’s... that’s where this is going to be decided. You know, that’s... every educator who’s listening to this, right? Your classroom matters. Matters so much. It literally matters for the future of humankind because it’s in your classrooms that students will develop that domain expertise, that... that AI enablement, and that human excellence that they’ll carry with them into the workforce. And if they learn how to have a responsible relationship with AI in your classrooms, they’re going to take that out into their communities, into uh their careers, and into the world.
Anand (43:53): Earlier you talked about the advice that you gave at a conference recently and I want to pick up on one of those points because I... I... I think it’s really great advice on thinking about how do we start to implement and move forward with AI. Certainly experimenting, encouraging people to play around with the tools, getting... developing some facility is important. One that I’m really intrigued by because I’ve... I’ve probably heard about this from a different perspective and I really like it, is the idea that there have to be mistakes. You’re expected to make mistakes. That’s uh not something that most people are comfortable with in higher ed. You know, we... we don’t want mistakes. We’re trying to minimize mistakes. So, what advice would you give to an instructor that’s saying, “Okay, I’m intrigued by this. Um, how far do you go? How do you u... model this for students? And why is it so important that we have mistakes within this process? Why shouldn’t we just teach them the right way to do it the first time and then there aren’t any mistakes?”
France Hoang (44:48): Yeah. Um, I think we’re at a moment where trying to label things as right or wrong does a disservice to the process. It gets in the way of learning. It gets in the way of progress and growth, right? Um, there is... uh there’s a lot we don’t know. And so if we only do the things that are right, that means we’re only doing the things that are safe. That means we’re not pushing the boundaries. We’re not pushing the frontiers. And so we need to give ourselves permission to make mistakes.
France Hoang (45:19): And if we do that, and we do that in a public way, if we stand in front of our students, say, “I’m going to be using AI in the classroom. I’m going to use it to generate lesson plans. I’m going to use it maybe to assess your work. I’m going to use it to try out different things. And I want your feedback. And some... what I’m going to do is going to be a mistake. It’s not going to have the outcome I intended. And I want you to call me out on it, or I’ll call myself on it.” Mhm. Um, you’re doing a couple of really great things. One is you’re modeling to your students how to approach learning and experimenting with new technology, right? And you’re giving them... when you are... when you are public with your mistakes and you’re okay with it, you give other people permission to make mistakes, which we don’t have enough of in today’s world, right? There’s too much pressure to be right, to be perfect. Social media is terrible at that. So I think it’s very powerful to model that ex... self-acceptance of... “the world’s big, mistakes.”
France Hoang (46:16): The second is, of course, you give your students then permission to make their mistakes and to be open. Um, there is a lot of stealth AI usage out there, right? Employees using it and not telling employers, students using it and not telling professors. Well, that’s a whole bunch of missed conversations, right? Faculty using it and not telling administration. We need those conversations in the open and we need to give ourselves and other people permission to engage in them.
Anand (46:43): Hey Stefan, I was going to say we’re probably running close... if you want to ask the last question uh give you a chance to wrap it up.
Stefan (46:50): Oh, I thought I uh asked... Oh, yeah. The um, you know, you’ve been involved in, you know, a lot of high-stakes domains, right? Where the military, law, aerospace... um, you know, how do you kind of translate, you know, that mindset like into kind of the educational space? Um, you know, you have like some recommendations for leadership that are similar uh but a bit different in education. And you know, you’re also dealing with innovation, right? And you know, as we’ve mentioned a couple times that, you know, a period of probably historical transition. So, how, you know, you’ve really been an amazing leader in these areas. How... how... what advice do you have for leaders in the education space?
France Hoang (47:29): Yeah, you’re... you’re being very kind, Stefan, by the way, in describing me that way. Um, I’m actually... I’ve been asked to give a another talk and I’m trying to think about themes of it and I think this is going to be my theme. Um, leadership in entrepreneurship is embracing fear and failure. Like it’s just having a life of constant fear and failure in front of you, right? Like if you want to lead, then you’re going to do things that are uncomfortable um that are going to make you afraid, right? Um, starting a business, right? The most likely result is it’s going to fail. And so you got to embrace that. Um, if you’re going to be in... if you’re going to be innovating, doing things that nobody else has done before, the most likely result of that is you’re going to make mistakes, right? You’re going to step on your own toes. You’re going to... you’re going to do things that are embarrassing.
France Hoang (48:22): And so, the price you pay for leading and innovating is failing and failing publicly. And not... and also the price you pay is living constantly with this fear of that. And so if you’re going to do that, you’re going to have to embrace that. What comes with that though, right, is when it does go right, when you do overcome that fear, when... when thing... when it doesn’t fail, it’s amazing, right? That is a real... that is something meaningful that no one can ever take away from you. And so by... by yes, you’re embracing a life of fear and failure, but you’re also opening yourself up to this uh... this wonderful opportunity to create something that didn’t exist before, to have real impact, and... and to do something that has long-term meaning um for you and for those around you.
Anand (49:13): That’s fantastic. I... I really enjoyed that message. Appreciate that message. Uh, and I think that is a great theme for you to think about for that next presentation. That’s a... a really important one. It’s, you’re right, we’ve driven ourselves, and I think especially in higher ed, we’re so risk-averse uh and so afraid of failure um that then that gets transmitted to our students as well. And it gets back to what we were talking about at the beginning, this entrepreneurial mindset, this perspective. You’re... it’s not something you can just adopt overnight. Uh, and it’s something we need to model for our students too and be willing to embrace failure and fear. Uh and recognize that that’s one of the costs of... of maybe leading to excellence.
Anand (49:53): Yeah. Thank you so much for meeting with us, to... talking with us. This has been a great conversation. We really enjoyed it.
France Hoang (49:58): Oh, thanks for having me. It’s been my absolute pleasure.
Stefan (50:01): Thanks a lot.
France Hoang (50:02): Thanks.