Videos in education: Quality assessment and considerations for the flipped classroom

Well, welcome everyone. We'll go ahead and get started and let others join today.

We'll be talking about videos in education, uh quality assessment and considerations for using video in the flipped classroom.

Um My name is Catherine Johnson. I'm Senior director of Institutional engagement here at osmosis uh supporting our partners across the globe in teaching and learning by osmosis, our agenda and objectives today.

So like any good educational activity, we do indeed have learning objectives. So by the end of our session today, uh everyone will be able to define the flipped classroom, describe when an educational video can be used in the to classroom setting, name, a tool that you can use to rate the quality of educational videos and identify at least one next step you can take to improve your use of videos in education.

Uh So our agenda is here. We're gonna spend a few minutes with some introductions and a quick poll question.

Um We'll cover the presentation which will be led by my colleague Sean Tackett and then we'll have plenty of time for a large group discussion, some quick participation guidelines.

Um If you can turn your camera on, we know that you may not all be able to do so. Uh But if you're able to turn your camera on, we would love to see you for community building.

I'm sure everyone knows just how difficult it is to lecture to an empty lecture hall. So that's why we like the cameras on.

Um feel free to rename yourself if you can. Um If you have questions, feel free to post them to chat, we'll pick them up as we go through or raise your hand, your digital hand and we will call on you as well.

Next slide. And when do you want me to, I release the pulse.

Uh Well, we have to do an introduction. OK.

We'll do that. So, uh so here we are.

So today's host, as I mentioned earlier, I'm Catherine Johnson, uh senior Director of Institutional Engagement at osmosis.

And I'm joined by uh Doctor Sean Tackett from uh Johns Hopkins Bayview Medical Center in Baltimore, Maryland in the US. And Shawn will be running uh most of the program today, Sean.

Would you like to uh introduce yourself? Yes, hi, I'm Sean and I uh I'm a, a general internist and I'm on the faculty at Johns Hopkins Baby Medical Center and I worked with osmosis for a while.

And uh yeah, so, and I know enough about the flip classroom and videos to just to be dangerous and to speak hopefully coherently and, and, and uh and use for you guys today.

Um So I think this is my turn to just kind of take, run the show. And so I'm gonna ii think I know how to do this.

I'm gonna try to release some polls just to kind of get to know you. We don't have a overwhelming number of people.

So that's kind of nice. Um But it's also kind of just useful for, for us and then we'll show you the results once you respond to the polls, um to find out kind of the background of the people who are, who are here today.

So there's two questions I think that you should be able to see here and I'm not gonna answer, but I II am a medical school.

I'd answer medical school and I would answer um faculty and I'm attentive to like the noon hour. I think people like to kind of sit back and eat lunch and, and not participate.

Uh And I understand if that's the case with you, you know, holding laundry or, or whatever. Um But uh you know, yeah, if we could because we don't have a huge group, uh feel free to speak up anytime interrupt, verbally.

Uh You also put things in the chat uh and we'll acknowledge him. So we've got 52% 13 out of 25 people advance in the pool.

I guess a few of us are probably not going to cause um maybe not just so I'm just gonna end the call. I wanna see what we got results.

So we have plurality, medical school nursing program, ea program and other health professions. You want to speak up point in the chat.

What other health professions? We, we didn't account for a whole and then mostly BW administrator, other, I don't know who the others are but feel free to also put in the chat or speak up.

So anybody wanna say, say something, hear, hear your voice. We're not seeing the poll results show up on the screen.

Oh, you didn't see it. I, we saw the polling questions but I didn't see the poll response.

So, oh, II do, I do. Oh, fantastic.

It's probably because I'm a cohost. OK.

Thank goodness. Um All right.

Very good. So we'll, we'll, we'll roll and um and so this is the, the, the canned aspect of, of the president.

We, we're, we're scattering polls just to kind of keep you awake and, and hopefully activate your knowledge as you're going through this.

But um some of this will just be sort of the expository kind of presentation. Uh And so starting with this um you know, hype and technology, you know, I think uh people have been enamored with the potential of technology to revolutionize education for a very long time.

And so here is one fun quote from 1913. Thus, Edison said it, books will soon be obsolete in the schools.

It is possible to teach every branch of human knowledge with the motion picture. Um And so this is now over 100 years ago uh with his Kineta Kinetic phone.

Um and we're, we're still not there yet, you know, we still use books. Uh We read articles, um a and videos are just sort of one element of, of good education.

So, um so we try to nuance what we say and not be too bullish on like videos are the answer for everything. Um But also kind of understand their, their, their potential role to improve education.

So, uh here we go with another pole. Hopefully I'm not gonna drive you guys too nuts by asking you to do these poll, but I'm gonna launch one and I'm just fair warning if you say yes, I'm, I'm gonna ask for somebody to actually do it.

I think you're allowed to change your vote once you enter it. I don't know, find out.

And I think so is a safe answer. You know, you know, the reality is as I'm saying that, I mean, I don't know who's answering what?

Right. So I can't like actually point my finger at anybody who said yes and, and, and say you, I know what the whole classroom is.

All right. Right.

OK. Good.

We did better with that one. Got more than half.

Well, over half we're finding I'm gonna end it. I'm gonna share results.

I think you can see that. But you're up if you can't.

Um, so half said, I think so. Some people were really kind of, you know, two people were not too sure.

And then, uh, and then so yes. So for anybody who said yes, thank you for saying that.

But then what, what would be your definition of a foot classroom? Maybe there's aut, so the students kind of do what would traditionally be homework inside of class.

So they have a guided practice together with their peers and the teacher that sounds pretty good. Anybody wanna, who wants that or, or, or have a different definition they go by.

Oh, how we do it uh where I work is where the students actually do the work online on a platform. Um So in their own time and then we normally recap uh on the next session, face to face.

So online session that is synchronous sounds pretty good too. And we also want to go by hi, everybody.

Um The way that we do a flip classroom is we give the students sort of material, reading material or prerecorded lecture materials and we let them go through it in their guided self directed learning time and then in class time we sort of discuss the work.

So what was the everybody's points? We sort of clarify any concepts that they don't understand or we can sort of have workshops based on the content.

So have sort of a pla practical application of the work that they've gone through within the classroom setting. Yeah.

Very good. I mean, all these sound like great ideas.

Uh, and let me, let me just, uh, see if anybody wants to respond to this is, is prereading. If I assign prereading and then have a class session, is that a flip classroom or is that, like, doesn't make the grade for you guys?

I would vote for that being flips classroom. Ok.

Ok. Yeah, just curious.

So, um so good and, and I mean, I think the answers I heard, where would fit with it? What I'm gonna tell you in, in the sense that like the foot classroom is the idea is that you take something that was like the traditional class time was for a lecture and maybe some questions or whatever and then you do your homework after the class and so you invert that order and you do kind of your homework before you show up to class.

And yes, that's, that's where the, the idea comes from. It was flip classroom inverted classroom initially and then it kind of came, came into um with, with flip classroom as the, I guess the primary way of people talking about it.

I think people will debate. I've, I've heard, I've heard people be very um kind of annoying about uh definition of ali classroom and what is not a flip classroom.

Uh And you know, saying prereading. No, that's not really the classroom.

Um So II II, don't you know, I don't have strong feelings one way or the other. II, but I can present, you know, this is uh a definition that came from a scholarly literature um on a review of the classrooms and they're trying to better understand it and recognize that there's heterogeneity in, in terms.

Um but I kind of liked it. So like, you know, three criteria move most information transmission, teaching out of class use, class time for learning activities are active and social and require students to complete pre and or post class activities to fully benefit from class work.

So I think that's, that's one definition it ii think it's a good one but there are others and um and this is the basic gist and also just while I have you and I apologize if this is like kind of remediation for some people who, who have seen it so many times they're tired of it.

Um But at least in my, in my own practice, I feel like people still get kind of confused around basic terms on related to online education.

Um And so just to say it out loud, so we're on the same page, you know, synchronous is when you do things at the same time, asynchronous is when you do things like different times in, in education.

And there are just some examples here. But the one I wanted to really point out is blended.

So uh cause any flip classroom and blended learning get conflated often. So blended is when you have in person plus online activities.

Um And so you can have a blended classroom if they do something online out of class and then you come to together, live in person in class, you can have a classroom that is not blended, that is fully online or you do sort of asynchronous work and then come together for a synchronous online session.

So just to try to disinvite um terms and to meet our learning for the day. All right.

And um and hopefully, so some of you will find this interesting uh the the theory. And so, you know, ii like this, I also like this theoretical model for the classroom is it's published.

And um and saying, you know, why is this a good thing to do? And, and when you're designing it, you know, being attended to these different principles and theories that, that kind of underlie the the intuition that, you know, that we all have that this model works.

And so um so, and, and you know, basically it, it's designed to increase extrinsic motivation, increase intrinsic motivation.

Um and then should also help better manage cognitive load and it does it kind of through these different mechanisms. So, you know, by um increasing sense of confidence that relatedness, sense of autonomy in the sense that they can kind of like do these things on their own, uh have control over their own experiences.

Um And tailored expertise, similar like the asynchronous aspect of it and, and self pacing uh helps too. So, so the largely this refers to like the asynchronous part where you can um do things at, at your own pace, um take as much time as you need to come prepared so that everybody's on the same level when they reach the, the synchronous session.

Um and then the sense of relatedness probably comes from more of the, you know, actually interaction acting with people.

Um And, and using that synchronous time more effectively. Uh All right, so we've gone through this and then the transition to think about videos and you know, videos is one method, you know, text audio video, kind of the three, you can blend and mix them all together.

But that's the way to transmit information uh for knowledge and largely for the asynchronous component of the folk classroom.

So uh as we're thinking about that, got another pole, oh maybe you can see a pool the entire time. So yeah, before we talk too much about videos, just wanna get your opinion.

So there's two questions. Uh I think you can see now your opinion about videos.

And then also like, when do you think they should be incorporated into a class? And while we possibly these polls too, I mean, obviously, if there's a question or something you wanna throw in there or uh or a comment you wanna make, you're, you're welcome to all right.

People are still responding and we're getting to that. Like, all right, this one even better than the last one.

There was a participation. Oh, yeah, I'm gonna end it.

Share results. Yeah, I believe you can see that.

So, this is a pretty friendly group to videos. Uh, one person said they're overhyped.

It'd be interesting to hear why somebody said that. I don't know who, who, who did, um, and I think people recognize, um, that, uh and I'm sorry, every now I'm pausing because like I see the chat light up and I'm trying to look at the chat.

And so, um but so yeah, I think I'm gonna stop doing that. And so if anybody see monitoring chat, just let me know.

Um that's something I need to respond to. I will let you know if something to respond to.

Um And then, you know, when, when's the best time to use them. And so again, here's like the, the recommendation that they can play a lot of rules.

Um Although mostly it makes sense in today's context around like the classroom that before class is prework is a good time to use it.

But there are other times too. So I think if I stop sharing the know.

Bye. So, videos and health education.

So what you're seeing here is uh something I help produce it. It's a last page in academic medicine journal.

And uh I wanted to before we really dug into like finding a good quality video to just put it in perspective. And so, you know, selecting videos, um you know, the, the credibility and relevance uh is always important, but it's all, it's, there are other things that are important too.

We've already touched on a little bit about the timing of when, you know, when you think you, you wanna use the video, uh making sure that aligns with your uh learning objectives and, and you use video because they're, you know, not all is about knowledge, sometimes they're like a movie recording or a or a videotape to trigger an emotional response or discussion.

And then, you know, beyond these factors, there are just technical aspects that make videos kind of complicated and, and you know, so if they are huge videos like, well, they, you know, can people actually access them, stream them.

Um and then, and then there are, you know, particularly with a any online learning, you have to think about copyright um I PS type of things and accessibility.

So just put it. Yeah, so just put it in perspective.

Um But today, you know, the focus will be on finding high quality video that would facilitate kind of a high quality foot classroom.

And one reason we, we wanna focus on this. So this review is from 2021 and uh they looked at studies examining the quality of professional medical education videos and this is just their excerpt from their conclusion, the review has demonstrated that many easily accessible videos on youtube presumably intended for professional medication are not high quality.

So this is the problem that uh that we also saw. So this is um you know, a, a study that we did to try to figure out uh to try to create an instrument for judging high quality educational videos.

So I'm gonna talk you through this and the key findings. Um Next.

So in the study, the IV QC instructional video quality uh checklist. Uh So we first we had to start out like what's an instructional video?

So, you know, that was actually harder to do than you might guess because there's so many like talking head videos and um con con style videos.

And there are many other words out that are out there that kind of refer to um this concept instructional video. So we, that was we focused our, our checklist on quality of those.

We defined it as a collective term used for referring to videos, cog cognitive learning goals including video tutorials, a instructional videos, instructional videos for declare of knowledge.

And then another thing that's different from is a demonstration of a procedure, right? Where somebody walks you through the steps of like how to insert an IV.

You know, we that's, that's different than kind of illustrating a concept in, in an instructional video. And then the other thing we realized kind of early on with this checklist is that um we can only look at general educational quality of a video.

But you know, we know, I don't think any single instrument could tell you if um you know, they're giving you the the if if what they're explaining is the consistent with the latest research or guidelines in, you know, cardiac or hypertension management, for example, right?

Or, or anything like that. So each, each, each subspecific subject matter probably need to be like added on to this.

But this is more about like the the quality of educational design. So our methods uh again, that's very much part of this.

And so, II know this is like pretty, pretty sensitive where our literature review encompassed. Uh article was on the video quality retention popularity.

There's a, there's a body of literature on the quality of reusable learning objects. It's actually kind of separate from q uh video quality.

There's one, there's several articles on quality repositories for open educational resources and quality of credibility, health information intended for the public multimedia principles, curriculum development or instructional design principles.

And so, you know, we went through this literature, we worked as a group to and we piloted and then as part of the the paper, you know, we rated 206 videos and we moved across a sort of five vendor sample, five or producer, video, producer sample.

And then with, you know, within those producers, we looked at intentionally looked at two different organ systems and then four different disciplines.

So this is the instrument and we're not gonna go obviously not gonna look at, you know, everyone of these items in, in detail on the checklist.

Um But they want to point out that, you know, we looked at four different categories. So educat educational design and it's related to alignment learning objectives.

Um Source reliability is more about like where the, you know, where source of, of the information that there is being presented, multimedia principles relates to how images and words relate to one another in, you know, in, in the video.

Um And then accessibility was something that we thought, thought was particularly important and that is related to control of the video.

But also um you know, can can it be accessible to a wide audience. So um so just to give you a chance to digest what you're what you're seeing right now.

Yeah, and I believe this will be the last pull of the session. So thank you for hanging in there.

Um So when you look at this, you think critically about it, if you had to pick one to optimize and obviously, there's, there's not like a yeah, there's not like a single right answer.

They should all be optimized, but if you had to pick one, all right, we're getting there. OK.

So let's show you the results of the poll. So OK, so like number one, if you ranked number one from the group.

So seems like source reliability comes down on top. And then close.

Second educational design, multimedia principle is um important but not perhaps is is the other accessibility kind of nice to have, maybe not necessary to have.

And that's interesting. That's something we talked about.

We talked about. The accessibility thing is like an addendum uh that, you know, you might use if you were trying to put it up worldwide as opposed to like for your your students.

So I can understand why people want it. Uh choose that is, is the most important.

So now let me show you like the actual findings from the 206 videos. Oh, so sorry, I just point out when you're using this, uh this is also intentionally designed to be able to be done one minute per video.

So you, you don't have to watch 20 minutes of video to like come up with a score on a checklist. Um So you have expose to the instructions for it too if you're if you ever become interested in using it.

Yeah, so the the results. So when we look at 26 videos, the total score was this is normalized sort of by this is sort of percent proportion of um points available.

So, you know, 70%. So actually not too bad, right before they do the worst, they got less than 50% of the points for source reliability.

Um And then they tended to do the best on multimedia principles which nobody thought was particularly important. Uh At least not as important as these other two elements.

Um And then to get into the weeds a little bit more and, and not gonna talk through everything, but to point out some multimedia principles, you know, where they do the best.

What does that really mean? Um You know, so we had two raters, that's why there's two columns here and then we took a mean, but across two raters, like there's 100% agreement that all the videos got points for overview before being ascribed in detail.

Um The visuals were kind of high quality uh you know, the the flow of information was good, narration was good. Um It was, it was relevant, the topic, um you know, the images and, and sounds are relevant to the topic and then, and then why did they do it so bad on source reliability?

It's more than 90% of the videos didn't provide any references at all for anybody to be able to tell um where they got their information.

Um And you know, II think, think about COVID, right? And how, how fast that that's been changing on us with, with the clinical evidence, you know, the most of the time, but when, when the video was updated or produced wasn't, wasn't indicated.

So I think these are kind of big problems for, for the state of uh a video. Um And, and again, hopefully this instrument is useful.

OK. Catherine, your hands raised, it is raised.

Not my virtual hands. My real hand.

Um How can you speak a little to the inter of reliability? Was that pretty strong um consistently through your um through your work?

Yes, it was. And, and you can, and here we present it, we just put, we thought rather than like a, a Kappa or some other like statistic that is commonly used because it wasn't really appropriate for like AAA one and zero whenever you get 100% when you get like a, a one and a one.

Oddly enough, this gives you like a bad Kappa statistic, although you think it'd be good. Um So, so that, that and understanding, we thought it's just best to present the, you know, the kind of the raw scores so people can make their own inferences.

So like, you know, there's a little bit more disagreement on the learning side of it. Um But otherwise you can see the numbers are fairly close with, with one another across all the videos.

Um One more thing that the participants um in your, in your study design here were mostly faculty. Where were they, were they medical or health profession students?

There's a, there's a bit of a conversation happening with um Lynette Van Der Meer. Oh, I'm sure I mispronounced her name uh about using this uh as a tool for students to assess the quality of videos they're accessing uh which I really think is a very interesting um idea.

But I wonder if that would, if, if there would be a really big delta between how faculty rate videos and how students do, there could be a big delta.

So the these were two students actually. So um in because in the the the five different vendors that we picked uh were kind of recommended by those two students is like all the ones that their, their classmates talked about and the two students from different schools.

So one from Miami, one from Colorado. Um and so, you know, so yeah, but we were thinking about it from the the students perspective in large part or, or for a faculty member selecting four students.

That's kind of how we orient ourselves. Mary Garceau had a question, what qualification did the two readers have?

And how is the number of readers decided? Why?

Two and not more uh well, the qualifications were medical students who were part of the project and available. And uh and then you know, why two and not more feasibility is the the simplest answer.

Um you know, but the statistics reach a, a much greater level of complexity when you have more than two readers. So that's also something we probably didn't feel like we, we didn't, didn't really want to deal with it unless we really had to.

But then, you know, the two raters came in pretty close with one another. And so we said that's OK.

Good enough, good enough to report. Sounds like good.

Um a good places uh Mary for uh followup studies. So, all right, carry on Sean.

Thank you. Oh, thank you.

Um Yeah. So you know, so then once you look at this at a very high level, when I told you, you know, we already thought about different sources of, of variation for, for um for these different metric metrics.

So this is like the other, this is kind of the one of the other key tables from the the paper. Um So what you're seeing on table two is ratings by vendor across the four domains.

And then, you know, we looked at like, I think three of them were commercial and two of them were free uh for people to access online.

Um And so, you know, we aggregated them just to see, but these are really more vendor related than anything. So the vendor comparison is more, more do and then, um and then down at the bottom would be the uh disciplines and, and type of the like the the topic of the video.

So we the two organ systems and then the four disciplines across the, the the topics. So, and then the take the take home points is, and the main take home point is this is that um you know, the vendor, what kind of was the key source of, of variation um uh of quality across the videos.

And the other thing to point out is that not every there was no vendor that did the best in everything or the worst in everything.

So you kind of had their strengths and weaknesses. So an interesting example was here like under e they had the highest multimedia principal score, but then they had the lowest source reliability score.

And you know, so I II felt like the instrument was useful for teasing out those differences. And so you can, again, you can make a judgment as to which one you think is more important when making recommendations for students.

Um Yeah, but even then I think the highest educational design was over here and see um and the highest accessibility we also see and you know, so it's kind of scattered.

So, so, you know, that's one of the things we wanted to ask you with the whole question was to think like where, you know, knowing that no, no vendors can optimize for everything.

Like where would you uh place your own emphasis? Um And then for completion, you know, what you can notice is that once you're, once you're looking at different topics kind of within a vendor or, or, or cross vendors, I mean, um you know, there is no significant differences across the board.

So, so the other implication for, for like a student or for an educator is that um you know, you, you if you find a a vendor who is rating highly on this instrument there for one video or for five videos, they're probably gonna do it for almost all of their videos.

And so, so if you're sampling, trying to figure out like, well, what do I recommend? If you, if you do find somebody that like has a high quality rating on the instrument, they're gonna have similar body ratings and, and, and you wouldn't need to go, like, you invent the wheel every time you want uh find a different topic.

Um Yes. So this slide, so, you know, it's kind of gone through the, the factors for selecting a high quality video.

So then there, but um, but just to say that it's not the only important uh thing when considering what makes for a high quality video.

And we talked about, you know, alignment with your own uh educational objectives and other design features. But once you implement, you know, if, if nobody's watching your videos, they can't learn from them.

And, and you know, and, and so this, this illustration here is, is so, and the other nice thing, right is that, um if you give somebody a reading by and large, you're not gonna be able to tell if they read it.

But if you do assign somebody a video, um and, and it's kind of in the right interface often you can track and, and see who if they've done it or, and sometimes even like the individuals done it.

Um And, and when they've done it, so that's what this, this is showing and showing it like um this is for uh uh for like this is data for across three cohorts at one school.

Um And so the green arrows point to whenever the faculty recommended a, a particular video um that, you know, usage went way up for the, for the cohort and aggregate.

And then also there was additional data that that showed like that they were watching kind of the right videos, genetics class and they were watching like videos on genetic disorders.

Um And then, then I'm gonna let Catherine kind of take it from here because she presented this recently at quit and gonna be fill in any other any gaps that I missed and then and also kind of guide the rest of the discussion.

Super, thank you, Shawn. You did a pretty good job with this slide.

So as Shawn mentioned, uh we were at the uh double AMC mid bi conference uh last week talking about videos and education.

And so just to add a little further context to to this slide, as Shawn mentioned, it is three cohorts and you can see those peaks with the green arrows are the incoming class over three years.

And when we look at the actual consumption of the the actual viewing of the related videos, they did correspond directly to what faculty had made as assignment.

So on an earlier slide, Shawn had something under session planning and was talking about concurrent. So not only do we have student acceptance um by measured by those views, but we also have very particular and purposeful alignment of videos assigned by faculty.

So during the first week of class in all three of these years, there were videos on learning science assigned as part of orientation.

Um the peaks that you see that are not marked by arrows in March of 2013 for that um this, this orange line. Um And then similarly, um you know, similar other places they really do correspond with places in the curriculum where you had faculty very specifically utilizing videos uh within uh within the curriculum next slide.

So um some anecdotal tips for I'm gonna call it dosing prework, right? There's often questions uh and conversations that, that we have around how much is the right amount in the flipped classroom, how much is the right amount of video, how much is the right amount of reading?

Um So I talk about that as as dosing, right? Uh anecdotally, what, what we hear is that about one hour um of prework per class seems to be the optimal amount for students to be able to really focus on.

Um I would also uh say that we've, we've seen lessons about frontloading content, right? To not make it so heavy at the beginning of the week that students can't get to us.

So to sort of spread those assignments out. Um, another thing is if you are alone in flipping the classroom and the rest of the coursework your students have is not flipped.

We consider this total student contact time and their workload across the week. Right.

Um, we don't wanna give students a huge playlist of osmosis videos and recorded lectures and readings to do for Tuesday morning.

If they also have, um, a whole day of lecture on Monday for other classes, right? That they will, they will fall behind.

Um Another point that was a little bit nested into Sean's. Uh one of Shawn's earlier slides is deciding whether you will measure um that prework and how you will measure that and, and are we going to educationally value it?

Right. Are we on the honor system um where we just expect students to do this work because we've told them and they're adult learners with the agency.

Uh We have some programs we work with who encourage students to do this required flipped classroom prework, whatever it is by giving them participation points for doing it.

Um back to that congruence, right? Suiting the prework to the objectives and the desired outcomes for your session.

Oftentimes we say, you know, if you uh if you want students to come, as Jill Cummings mentioned in the chat, you want students to come just prepared to build on their learning.

Maybe we just give them one or two videos to watch to to prime the learning, right? So students can ask better questions or have a framework and not be lost.

Um In other cases, may we may want to completely replace what might have been a lecture and then do hands on activities.

Um But uh to end this, I think the the one really important thing about prework and the flip classroom being um having the results we want is to clearly connect that prework to the class work and to what we're measuring, right, to ensure students understand the relevance, they're, they're very astute students at knowing what you're going to test them on.

Um and what really matters at the end of the day. Um And I think it's a really nice opportunity to uh to really connect things meaningfully for them throughout the, throughout the class and throughout the curriculum.

But we thank you for joining us today. Um Our intention is to have another variation of this presentation in early 2023 where we will invite some other um uh faculty who have published on the topic of uh videos and education and we'll have a round table um panel as well, a panel invited event.

So look out for an invitation for that in uh in probably February of 2022. And until then, uh stay safe, stay well.

And thank you for your attention today.