How supporting Wikipedia editors is helping improve trust in science

Um So today we'll talk about how supporting Wikipedia editors is helping to improve the trust in science. So, um next slide um I'm Catherine, uh Catherine Johnson, I am Director of Educational Strategy and academic engagement at um Elvir.

So I came back, I came over from uh osmosis where I've worked for 3.5 years as the uh company was acquired by Elvir. So, very excited to be able to more positively and more broadly impact medical education and patient outcomes through education.

So, my background is in instructional design and publishing and I have been doing this for 30 years in some capacity and I love medical education.

So for today, you are both already muted. So lovely and your cameras are on also lovely.

Um If you have questions, it's a small group, you can drop it into chat and I will queue Amine um or uh uh raise your virtual hand.

We'll, we'll call on you and uh um Amine and I are pretty seasoned as you must be as well at uh running these webinar uh meetings.

OK. Oh, yeah, that's right.

I'm not needed. OK, great.

So, um uh so I'm very excited to, to, to share with you both and to get to know your thanks for your intros. Um Here's mine.

So, um I've learned through the years that doing a personal disclosure as well as an academic uh industry disclosure is a good idea.

Um So there uh Andres is my time in your beautiful country with that soccer jersey uh from Colombia. And um and uh uh we used to love much more international travel, but now domestic travel during the pandemic and maybe once again as we begin to emerge, some international travel again.

Uh So I'm in a, in a two physician couple. I'm privileged to have two healthy Children.

Uh They are now 16 and 12 and my wife has a does uh does, has a public health background and does work in childhood obesity.

Um And here's my academic uh and industry disclosure. So I've been an academic psychiatrist my entire career at the University of California, San Francisco where I still active clinically.

Um And um I also teach in Berkeley and at Samuel Merritt University, which is an All Health Sciences University uh in the simulation center.

And the bottom row is my academic, a non academic disclosure. So, um I'm excited to remain affiliated with osmosis.

Um I've been around with osmosis for six years as a faculty consultant. Um and the work here that I'm gonna share with you around a uh Wikipedia is part of what brought me to osmosis.

Um And fundamentally, I really believe in that bottom right corner that we need each other. Um And so having the access to these industry created resources allows us to be more effective educators at our academic institutions.

So with that, I'll do my land acknowledgement. I'm here in the San Francisco Bay area.

I currently am in Oakland and I wanna acknowledge the peoples from the Olo Muma and Confederate villages of LJA. The link on the web on the here I think is really cool.

It actually um shows you native peoples from around the world and you see these uh overlaid on like the Google map. So you can really get a good sense of learning more, not just about the names of the peoples that were um the the first peoples in the places where you are, but also uh more about uh the overlap uh across different groups of individuals.

So I acknowledge those people who were here before me. All right.

So Wikipedian health, I've done this for 10 years now. So this is I'm gonna talk fast.

I I've just uh please interrupt me at any time with questions, but I'm gonna make six key points today. The first thing we're gonna do is say like any good scientists, we should use our librarian colleagues like Jenny to teach us how to find these papers and then you find the manuscript you need, that proves what you're asking the question here.

How much do people read Wikipedia for health information? So here's a study by a couple of colleagues, one Canadian physician and a colleague of his and they actually just counted it up.

So um uh at the end of 2013, Wikipedia's health content was about 100 and 55,000 articles and 1 billion bites of text, whatever a bite is.

And that content was supported by over 950,000 references. So, uh and they just counted how many eyeballs read that and it was viewed 4.88 billion times in the 2013 year.

So they argued it made it one of if not the most viewed medical resource globally. Now, any good science should be replicated.

So here's the repeat study several years later. Now, a larger group, international group, same thing just counted up and they said, OK, Wikipedia was born in 2001 and now as of 2017, it was 44 million articles across 295 language Wikipedia.

Uh and there were 5.4 million were in English. And of that 5.4 million in March 2017, there were 30,000 articles on health related topics in English.

And another 164,000 health related articles on in the other 294 language Wikipedia. Collectively, they were read over 10 million times per day per day, not a typo per day.

So if it wasn't in 2013, the most widely health related uh in information source, it sure was by 2017. OK.

So here's one example, uh here's the English language Hepatitis Wikipedia page. And on the side, there's some special tools you can use to look up things like traffic statistics.

So here's that page, here's a snapshot in time, a two month window in 2019. And you can see that Hepatitis in English, Wikipedia has been read 100 and almost 100 and 11,000 times and the astute faculty member of a medical school or will note a five peak two valley pattern, saw tooth pattern here.

Uh Chantal. What is this pattern?

Why do we have this pattern? Any guesses?

It's we good. Oh yeah, on weekends and weekdays.

Yes, exactly. Why is that?

Because in general hepatitis is diagnosed on weekdays. So what do patients do we give them the diagnosis?

They go read about it at home after the clinic visit. Uh Sure it's not zero on weekends cause sometimes you get diagnosed uh with hepatitis on a weekend.

But this is human behavior right here for us. Same pattern now, but now different disease.

Here's influenza, a full calendar year over a million views. And we say the same pattern with the five peak two valley pattern with a seasonal variation because a lot of the human populist uh is reading about this in the what we'll call the global north.

Um, so, um, and there, there's this one valley right here. And, uh, and Jenny, do you wanna guess what that is?

Uh, that would be Christmas break? Yes, it is.

Exactly because nobody wants to read about influenza when they're holiday, vacationing with their family members. Right.

So, again, human behavior just wonderful to see it in, in a sort of pattern. So it turns out that we can look at those traffic statistics and see how much people are reading health related information on Wikipedia.

And in March 2020 here was the top 22 or so most heavily viewed health related pages on English, Wikipedia. No surprises, right?

Corona, Corona, Corona Spanish flu Corona virus, Corona virus, Corona black death pandemic, like what we would have expected.

And you can see the daily average the Coronavirus pandemic, a million views per day, 300,000 views of the Coronavirus 2019 page, et cetera, et cetera.

So, OK, so key point number one, the world uses Wikipedia for health information. Uh We kind of knew that but now we see the numbers.

OK. Second point.

Have this, have you guys heard of wiki projects? Andres Jenny uh Chantelle.

No. Yes, no good.

Uh If you haven't like anybody else when I don't know something, I Google it wiki project. And the first hit of uh the Google search is the Wikipedia Wiki project page that page describes and defines a wiki project as a group of contributors who want to work together as a team to improve Wikipedia.

So here's some examples. Here is the Wiki Project Food and Drink Wikipedia page.

So if you're passionate about the accuracy of information about carrots, you can be a part of this team. Here's the wiki project architecture page.

Again, if you're passionate about the quality and accuracy of information about the Taj Mahal on Wikipedia, you can be a member of this group, here's Wiki Project Plants, here's Wookie Project Birds, here's wiki project physics.

And I've done this deliberately from a playful to a more scholarly domain. Here's wiki project mathematics.

And then finally, here's wiki project medicine. So uh this wiki project as a group of volunteers committed to improving health related content on Wikipedia, many of them are health professionals, many of them are not, they're all dedicated to high quality accurate information on Wikipedia that's health related.

And when you scroll through the wiki project page, you'll see some great resources. So on the bottom left, uh those blue boxes sources of reliable health information to search for, to c a manual of style for how to write for Wikipedia, how to edit complex of interest, scroll further and you'll see some tasks and tools um scroll further and you'll see that wiki project partners with the Cochran initiative and they have their own journal.

There's even a program here at the bottom which you'll hear a lot more about which is um embedding wiki editing in health professional universities.

Um So these individuals have taken the roughly 45,000 English language Wikipedia health related articles and they've ranked them by importance.

And that important scale takes into account two very uh different factors. One is the number of eyeballs, reading those pages.

The second one is the global burden of disease. So I like to point out that uh human sexuality is top importance because everybody reads about sex on Wikipedia.

But so too is Dengue fever even though it's not endemic to continental United States. So, all right.

So uh that important scale is just artificially divided into four buckets, top high, mid and low. And you can see the necessary criteria in order to be landing in one of those buckets.

Um And so it won't surprise you that things like TB and cancer are top priority. Now by definition, top priority articles cap at 100 articles.

So it's going to be less than a 1% of health related topics. But you can see kind of the the the criteria listed for the other uh to land in the other buckets.

So um now I want to point out that these mid and low priority buckets are still extremely important to patients and their loved ones.

So there's no, no judgment here about that. It's just that fewer people are reading those pages on Wikipedia.

OK. So here's the same thing with Wiki Project Pharmacology.

A group of individuals committed to improving pharmacology and medicine related topic on Wikipedia. They have also rank ordered the roughly 12,000 English language Wikipedia page by importance, same scale different articles top high, mid low and you can see uh the criteria necessary to land in those buckets and you can see uh the examples at the bottom.

So in the top priority category, I'll just point out it's not single drugs, it's drug classes. But you can see again some other drugs that land in, in the other buckets.

So um so key point number two is that volunteers to write for Wikipedia aggregate around what are called wiki projects. All right.

Yes, you say. But isn't all of Wikipedia trash?

As uh many academics often say, it turns out that's not true anymore. So all of Wikipedia is graded for em uh quality.

So look at the bottom, we start as a stub article if somebody throws in a few sentences and then it moves up as people add content to a start quality to see quality to B and up the scale.

If you were to click on one of these show, uh things from the Wiki Project medicine page, you'll see the more detailed criteria necessary to earn that grade on Wikipedia.

So ac article is more substantial than start but not as good enough to be B and the B article criteria has six specific criteria necessary to be earning, earning the grade of B as you see listed there.

Um and on up the scale, the highest quality is what are called featured articles, like many web pages, they just are featured when you go to Wikipedia directly rather than via a Google search or something.

And you can see that the the very clear cri grading criteria in order to be deemed deemed featured article the best of the best.

So when you put those two scales together, the important scale and the quality scale, you stratify all Wikipedia health related pages by importance and quality.

So effectively, you have a roadmap of Wikipedia's equality. Um And uh here's the list I showed you earlier, the top 20 most heavily trafficked Wikipedia pages that are health related in 2020 of March and here's their grades.

So as you look through them, you can see that many of them were deemed be uh be quality or s equality. Uh So some work to do to improve them.

Uh But certainly they were a high priority um at the time that uh because of all the heavy traffic, um some of them not as high.

So here is an update for September 2022. So here are the top 20 most heavily trafficked Wikipedia pages then.

Um and um and here are the grades then. So I'll make a couple of points here.

So we would expect that the COVID pandemic remained hi hi, highly um important and highly trafficked. Um but if you look on that right side, you're gonna see some that were very um not present on the left side.

So, b uh Elizabeth Holmes as number two and blunt trauma, number one, sexual intercourse at number five. The thing I'll point out here is that uh Wikipedia viewership is very much influenced by social media and what's trending um in, in the news at the time, right?

So, um but there's also some things that sort of sadly remain active no matter what. So suicide methods comes in at number nine.

And as a psychiatrist, I'll just point out that that's a public health challenge that we need to address. People are looking for ways to kill themselves through Wikipedia and we wanna make sure that we try to, to sort of uh mitigate those risks.

Ok. So where are the grades?

So we all read the Wikipedia pages themselves. If you look at the top of the page, you'll see that there's a talk tab, the article tab is the text of the Wikipedia page.

The talk tab is where Wikipedian talk to each other. So if you click on that, that's where you'll see the grades and you'll see uh as you scroll through there, the dialogue between Wikipedian.

So here we can see that the Schizophrenia Wikipedia page is part of several different wiki projects, all of which have graded it featured article quality, but it's variable importance depending upon um how closely aligned is with the core of that wiki project community's efforts.

OK. When you scroll up further, you'll see the dialogue between Wikipedian.

So here's a post from a Wikipedian saying that the first sentence of this article says schizophrenia as a failure to understand reality is unnecessarily stigmatic.

And they make the argument why they feel that way. And you can see the dialogue with a reviewer, I mean, a a separate Wikipedian saying you're correct.

This is over simplistic, stigmatizing and incorrect. So if you want to see dialogue between Wikipedia as they debate, the content, click on the talk page and you'll see it right there.

So Keypoint number three, Wikipedia is graded for quality and you should use that scale in your personal professional lives.

Re if you're gonna read Wikipedia for health related information, just go look at the grade. Uh So it's super simple.

One click to see it and you can even move the grades to on your device to be right there on Wikipedia. So you don't need to sort of find it behind the scenes.

All right. And I teach my patients the same stuff.

So if we drill a little deeper, now we look at Wiki Project medicine and pharmacology on these scales, you can click on any one of those cells and it'll give you that list of articles.

So here are the top important s quality grades, uh articles from Wiki Project Medicine. And I wanted and you can see when they were graded as top importance and when they were graded quality.

And uh if we um uh look at those dates, I wanna acknowledge a couple of things on the important scale. These are pretty old dates, right?

Uh But it doesn't matter because they haven't gotten any less important since they were graded as top importance. So I don't, I don't care about how old those dates are that call.

On the other hand, the quality grades uh do matter and how long ago they were graded matters. Um And now why are they so old?

Because it's hard enough to get people to write for Wikipedia. It's that much harder to get people to grade Wikipedia's quality to get volunteers to do that.

So, um so I'm gonna suggest that these grades may actually be below the current quality of the pages. We just haven't re graded them.

Ok. Now that's a hand waving thing.

I can't really say that with certainty, but I uh that is a supposition. I propose to you.

So um let's talk about some other amazing initiatives that are happening with Wiki Project medicine. A couple of them are on the wiki page.

One of them is internet in a box. Think about this as basically a local uh like device that you can take somewhere where you don't have access to the internet.

Um And it has all uh all the health related content on Wikipedia. So you can download it locally, think rural clinics, uh, or places where, um, internet is, um, not available.

Um, in addition, there's an app for everything. Here's the Wiki Project medicine app, uh, of the health related content on Wikipedia.

And, um, and that app, um, uh, also available in, uh, 10 different languages. So snapshot in time health related content on Wikipedia, uh, take it with you wherever you are.

As I oftentimes tell students if you're on a plane and you're worried as a medical student about your health knowledge. Um Make sure you download this app before you no longer have wifi on the plane.

OK. So um useful tools especially in parts of the world where internet is uh controlled or heavily uh monitored.

And we can think about countries like that. For example, can I ask you a question?

I'm, I'm dying to know. So does the World Health Organization endorse the wiki in a box as a solution?

Do they know about it? II don't know the exact answer to that question.

I do know that there are what we will call Wikipedian in residents at nonprofit organizations that can help sort of translate the content from those organizations to the Wikipedia pages.

So that's an example of where I know that there at least a while back used to be a Wikipedian in residents at Wh O That's amazing.

I'm just thinking about the global health goal around um portability of information and wifi in rural places, as you noted is um it, you know, it is a challenge.

It's so powerful. Thank you for sharing that.

You bet. And there's a video later I can show you about a collaboration with Columbia University and uh countries in the Caribbean where um they're trying to do exactly that kind of a and in that video, if we have time, I'll play it.

But in the video, one of the speakers says the world, sometimes people ask, how fast can you get broadband to rural um under serve?

And the answer is not fast enough. And so these interim solutions can certainly um address the health care needs while we're working on getting broadband to those places.

So um ok, so there's this great collaboration chain waiting for us to participate. Uh So, so I got excited about it.

So uh in 2013, I created the world's first medical school uh course for students to Wikipedia for academic credit, Jenny has hinted at some of these uh uh edit Toons and we've done that kind of stuff too.

Um But I'm excited because this is actually academic credit. The only way we can pay our students is with academic credit rather than salary and edit Aon are just volunteerism.

There's nothing wrong with that. It's just that it, it's uh relegated to volunteerism.

And so in this case, we happen to create the first course in the world. So I want to show you some resources uh that uh demonstrate what our students have done.

So this is a March 2016 cycle of my course, I'll talk about what this dashboard is in a little bit. But here those students worked on the English language Hepatitis page.

And there's this beautiful tool that allows me to see as faculty, which words my students wrote. So here is the uh is a subsection of the Wikipedia page and you can see my students weren't just fixing periods and commas, they were rewriting the whole thing prac practically.

So, so here's a few quotes from students and II always like to make this interactive. So Catherine, can you be Caroline and unmute and read that aloud?

I can indeed, I learned that Wikipedia has become a much more reliable source of source over the years and could really be a useful source for many of our patients.

Andres. Would you be Phoebe for us?

Sure. Uh Beyond the Oh, thank you.

And then Jenny, let's be f you can be Frederick. I think our job is to be the person in this global community who can share very complicated things in very simplistic terms.

And I think Wikipedia is a great resource for that because it is free and a lot of people do use it. Thank you guys.

Um Words of truth from my students directly from their mouths. Uh You may have heard a little bit about vaccines in the news recently.

Um here's the Wikipedia Vaccination page. Back in 2019, 1 of my students, Derek Smith uh decided he was uh excited about and um struggling with the uh anti vaccine backlash in this country around uh sort of uh concerns to not uh vaccinate Children.

So he decided to uh research that anti vaccine movement and he joined an anti vaccine Facebook group to understand the sources of misinformation.

Based on that, he decided to edit the Wikipedia Vaccination page um and wrote out the safety section and did not experience any pushback from anyone on either side of the debate.

So while Derek was working on it, that page was viewed 33,100 and 52 times during the month uh while he was working on it in my course.

So um here's his work again, using that same tool. Here's the whole new safety section.

Uh He added uh a table with some safety monitoring from some or government organizations. He added stuff on side effects, some mild, moderate, severe side effects um and addressed ingredients commonly of concern.

So I created this section because the the sources of misinformation were citing aluminum and mercury concerns, for example.

So um so these tools that have shown you are from the Wiki Education Foundation, I'm gonna go into that in a minute. And so key point number five is there's this fabulous tools available for us as, as faculty.

Um We are just latecomers to this party. OK.

So um so I wanna show you what we've done at UCSF and then show you some other universities. So um every year I offer this as 1/4 year elective for medical students to take it uh as an immersive, fourth year rotation, uh full time, one month, they do nothing else but edit Wikipedia for health related uh health related content for med school credit.

And you can see that every year I have a small handful of students that do it, what they work on how much they've added uh references and cumulative uh article views.

So I'm pretty proud of that. My students, they do amazing work and they've done stuff to really improve Wikipedia as a public health information source.

And we're stuck in this unhappy marriage still uh right now because um uh we all keep telling our students not to go to Wikipedia and they're doing it anyway because they love it.

It's written in a way they can understand it's easy access, it's high um high yield. Um And so I think the problem here is that we are pretending like this problem is gonna go away when it won't.

And so the couple's counselor for the unhappy marriage is the Wikipedia Education Program because basically what it is is educators and students around the world contributing to Wikipedia informal academic settings rather than fighting each other.

So, um so I'll show you some screenshots of Wiki Education's Wikipedia pages and these are all on Wikipedia all free. And it's basically guidelines for faculty to think about how to design courses or assignments for students out at Wikipedia.

Here's some content uh resources for students. So the Wikipedia Education program is global and in North America, we're lucky and privileged to have not only that but a very specific nonprofit called the wiki uh wiki edu, who um whose sole mission is to uh be this bridge between AKI Pia and uh AAA Pmia and Wikipedia.

So they're working to help us uh design courses and assignments. So this actually happens to be me and my bunch of my med students.

Um But here's the nonprofit, they are tiny, they are based out of San Francisco and the Presidio. Um This was the staff after they cut staff after the pandemic.

Um So they used to be twice as big, but this is a tiny team, but here's their impact. They have resources for us as instructors and for students.

And that dashboard I shown you. Um And here's what they've done.

So 95 million words have been added to Wikipedia by students since 20 10/5800 classes, mostly undergraduate courses in universities.

100 and 14,000 students have contributed um a couple of points to highlight here, a lot of on the bottom there. A lot of people have talked about the Wiki Pedia gender gap that most Wikipedian or people who write for Wikipedia are men and they might write about wars and sports.

Um So, uh it turns out that roughly 22% of Wikipedia editors health identify as female. In contrast in the, in the student program here in wiki education, 70% are women and non binary.

So you're addressing the Wikipedia gender gap by bringing Wikipedia editing into the classroom setting. Um So I'm gonna propose to you and this is uh so Catherine knows that years ago, I don't know if you know this, but II asked for osmosis illustrators to draw this for me.

I was so I just have to do the animation again. I was so happy to sort of be like, can you guys just draw these faces and make them happy?

Uh And so they did, I wish I knew which illustrator drew it so I could give her credit. I'm pretty sure it was one of our female uh illustrators.

But um in any case, I'm gonna propose to you that maybe we can start having a healthier relationship between Wikipedia and academia, at least here at our health universities.

Um And I'm gonna show you that in fact, there uh that many other schools have embraced this movement in other health professions, including dental, physical therapy, pharmacy, nursing, and public health.

Uh The Wiki Education Foundation aggregates courses into what are called a campaign. So this is the campaign for students in the health professionals courses.

So you can see, there have been 100 and 48 courses, 2400 students that have added 2.42 million words to health related pages.

Um and you know, 29,000 articles to pages that have been viewed 88.2 million times. So there is a movement here which I'm pretty excited about.

Uh Thanks for the comments. You're totally right, Jenny about I pe um and um and let me show you some examples because now what I wanna do is tickle your brains to be thinking about how would I do this locally?

OK. So I'm gonna show you some exemplars around the world.

We're gonna start in Israel, Shane. Evans is a Basic Science faculty member at a med school in um in Tel Aviv University.

And uh right around the same time I started, she created a Wikipedia editing course for uh medical students and graduate students.

And this was uh Wikipedia editing course on the Hebrew Wikipedia page uh pages. So these are some of her early students uh learning how to write uh for um health related content for her course, semester long course.

In 2014, the econ School of Medicine at Mount Sinai in New York City started having Wikipedia editing uh electives. And so this is their Wikipedia page and you scroll up further and you'll see that they have different academic years and different uh courses.

These are non credit bearing courses, but sponsored by a faculty member and L co led by students. But so they had some example um cycles of their course, a couple of things that I thought they did really wonderfully, one is they situated it in um scholarship.

So here's uh a student doing a, a poster about it uh at a local sort of one day education conference in uh at, at uh Mount Sinai.

And that poster after the conference is now available on Wikipedia as well. The other thing I think that we, that the folks there have done really well is the same thing like Jenny, we talked earlier about edit Aon.

In this case, you can see an example, edit Aon about race and gender bias in medicine. So in this case, anyone from the public can come and edit alongside uh uh uh uh Columbia affiliates and use the library resources in the work of writing for Wikipedia.

Um OK. 2016, uh Catherine her, she is a uh Basic science faculty member, teaches research for medical students at the University of Notre Dame Australia.

So uh Catherine originally embedded uh Wikipedia editing as a service learning component um arguing that many of the students that work there or learn there are not uh from the Aboriginal communities uh and she but would be serving those populations of patients.

And so she said, why don't you learn about the people you're gonna serve? And then while you're learning, write about it on Wikipedia.

So it's more accurate. And then she got huge pushback from many of her students who said I came to med school to be a doc not to write for Wikipedia.

So undeterred, she decided to embed it in the research track. So, so this is from her syllabus and so you can see that she's embedded Wikipedia editing as one of three key threads in thinking about systematic research inquiry.

So you've got um journal article review. Uh and I infer to do and create and while you're at it, why don't you write for Wikipedia?

So, um so that's what she's been up to through the years here in the US. We've had several medical schools that have embraced uh and uh Wikipedia editing many times as 1/4 year elective.

And what I'm showing you here is the year that they started which school every year, one or two students, uh one or two schools embraces it brings it on.

Um And uh and then just continues it. The one example where a faculty member stopped doing it is Rush.

And that's basically just because she moved to a new university. And when she went to the new university, she started Wikipedia editing there too because students love it and she loves it and it does good in the world.

OK. Here's an example.

2018, many people tell me, oh OK. Fourth year they, they've learned enough so they can be med students to write for Wikipedia, but first years are too dumb and stupid and naive to do it.

And I say hogwash as does Melissa Chely Heed. She's a librarian at the University of Hawaii John, a burns school of medicine and she's had medical students, first year med students editing Wikipedia since 2018.

In this case, this is from her syllabus. Um she and I paraphrase a bit here but she believes that medical students want to contribute to Wikipedia and know how to do it even as first year med students, if you guide them a little bit.

So, so these are sort of the, the, the sort of learning objectives and tasks OK. To uh college uh here.

Now we're gonna broaden beyond medicine. So this is physical therapy students and uh Tuo has had physical therapy students editing Wikipedia alongside their faculty since 2016.

Um In this example, they do it as a as sort of like in class during a three hour window where they're all editing in this uh classroom setting with alongside faculty and what the folks at to have done is scholarship as well.

So they've looked at what the students have done and published it. So here in from this journal article are several articles that students uh physical therapy students edited and you know, the page views and the four during the course and four months out, these are what we'd expect.

Chronic pain, joint dislocation, cerebral palsy tele rehabilitation. Um So point being here, this is not medicine specific but can be across health professions.

Um OK. My dental, our dental colleagues in the UK uh since 2016, here's uh the University of Dundee editing Wikipedia.

That's uh Noguez at the front, a faculty member in the school of Dentistry and what she's done, which I think is so cool is they formed a consortium of UK dental schools to sort of like divide and conquer.

So they basically you're not accidentally leaving parts of the dental content untouched and undoubled dipping in other domains.

So this consortium of UK dental schools sort of figures out what pages need, what work and who's doing what? OK.

Um Since 2014, my pharmacy colleagues at UCSF created a Wikipedia editing assignment in the School of Pharmacy and they embedded it into a required health policy course.

So anything below the line is my students in the medicine elective and anything above is pharmacy students. I'm showing you numbers of students now.

So you can see that every year I offer my rotation once or twice. And we have a small handful of students that will take the course depending upon their other priorities.

But on the other hand, when you embedded in a required course, boom, boom, boom, you suddenly have huge numbers of students that are editing Wikipedia as opposed to tiny fractions.

So they very quickly eclipsed my work with my students. The good news, I'm happy to report is that based on this.

We have now finally seen the light and we now have Wikipedia editing required at UCSF for all medical students in the second year course.

Um, as a, uh, part of what we have as a required inquiry thread. So suddenly much bigger numbers.

Ok. Here's the Columbia University.

This is the video I was telling you about earlier. I'm gonna skip the video but we'll come back to a time permitting since 2017, they've been uh partnering with uh rural clinics in several countries in the Caribbean uh to uh bring that offline uh um device to those clinics and understand what content would you want on internet in a box.

And since 2018, the University of Edinburgh School of Public Health um has been embedded in Wikipedia editing. So this is a blog post that I found from the faculty member there who said, look, Wikipedia is too big to ignore our students.

Go there all the time. First port of entry.

Why don't we partner with our students to write public health content on Wikipedia? And so this is just some screenshots from that blog post.

Um And again, I agree with this, a faculty member, like, why are we fighting our students? OK.

I'm gonna broaden beyond universities, talk about uh uh organizations. So here's Cochrane, I told you had uh wiki uh Wikipedia partners with them.

So this is a Cochrane initiative that ha that takes uh health related content and divides it up by, by domain of human health.

Um and tracks similar to student courses, it tracks Wikipedia editing. So here are several different campaigns with Cochran um where you've divided up.

Uh oh I told you, Catherine about wh O and the Wikipedian in re so here's an example there. Um So what they do because this is not time based on a semester or course, we call it a, an open ended campaign.

So you can see the different sort of sub campaigns and scrolling further. You see some other campaigns and other domains of human health and just picking one at random here, the Skin Wikipedia initiative, you'll see again, sort of those numbers of editors and content scroll up further and you'll see the pages that they're working on and those same tools I showed you earlier to click on and see the sort of who wrote what OK.

So um I'm uh really privileged to work with osmosis uh and with medical students around the world as part of osmosis. So one thing I did a couple summers ago and continue to do is offer a global course for medical students around the world, for our medical education, uh fellows and nursing students ambassadors.

Uh And basically, uh that year, we had 11 student editors from three different countries. Um And then um last year, I think it was nine countries and I don't remember the number of students off hand.

But here's a quote from one of those students, uh to, I've not had you read, you read. So sure.

One thing that really struck me was how friends and family responded when I told them I was in a course for medical students to improve health related articles on Wikipedia universally, I encountered surprise, followed by a thoughtful moment and then a statement along the lines of, well, that just makes sense or about time someone thought to do this kind of thing.

I really, really enjoy working with these students because they all have that same like save the world medical student mentality, but they're also around the world.

And so they learn and grow alongside each other um through this global community. So got offer it again this summer, I have to get off my butt and pick the dates.

So OK, so this is the kind of stuff that is uh finally academia and sort of journals catching up to. So academic medicine top tier journal in our field.

Here's an article published I think in 2019 teaching evidence based medicine to medical students using Wikipedia as the platform for teaching.

OK, over the last several years, um at Uppsala University considered one of the top 100 universities worldwide. This is in Sweden, here's a faculty member there that uh that I corresponded with who teaches phd students in medicine.

And at the end of seven years of dissertation and work. He says, why don't you now that, you know, all this science, why don't you give back to the viewership?

Um Very few people will read your dissertation, but a lot of people will read Wikipedia. So how, how about learn how to write for Wikipedia?

So he points out that a small fraction of his students say yes. But for those students, here's his course intro to scientific research.

Um And here are some of the pages that they edit. So, you know, um so these are other domains of science and I, you know, I don't really know what a Lacko lith is, but I know that somebody who knows a hell of a lot about Lalith is writing about it on Wikipedia.

OK. So some of our, not some of our professional organizations have seen this bandwagon.

So here's the Society of Family Planning. Um And uh Wiki Ed sponsors a training course to teach fully trained members of this organization to write for Wikipedia.

So now we're going beyond student populations um to the rest of our careers. And so here's a, a blog that talks about the pages that fa I mean that uh physician members of this organization wrote uh or edit it.

So vaginal Bleeding Osmotic dilators, tubal ligation things we would expect for an organization called Society, Family Planning, but sort of teach them how to write for Wikipedia and then give your knowledge.

Uh finally, the Center for Disease Control the National Institute of Occupational Safety and Health. They've had some wiki editing for uh campaigns for a while.

Back in 2019, they awarded uh the uh their award uh for educating the public through Wikipedia to this initiative. So here's some exa another blog post showing um the power of, of uh adding through Wikipedia, the wiki for World Health Hearing Day 2019.

Here again, several different universities, several different uh sort of edit of thought equivalent um F focusing on different uh hearing um occupational hazards in uh loud uh work environments.

Um So, key point number six, there's already this movement of health professionals uh schools and I'm gonna in invite you to consider being next.

Um I'm gonna close with a couple of musings and then we'll do lots of Q and A or whatever, wherever you guys want to take the conversation in putting this talk together or updating this for, for you all.

Um I wanted to add some uh musings rather than sort of data driven uh stuff. And so, so these musings are just stuff over the years that I've come to think about.

The first one is what I like to think of as the continuum of science. Um We all tend to put uh sort of uh uh top tier closed access peer reviewed journals in one end of the spectrum.

We know it to be scientifically truth as, as truth as is known now because it's been vetted and written by experts in their field.

At the other end is Wikipedia, which we all think is many people think is trash because anyone can edit. OK, open access for journals is pulling this pole of science towards the Wikipedia poll which is mastered open and initiatives like this where you get professionals contributing to Wikipedia students in development, contributing to Wikipedia is bringing Wikipedia science content towards the closed access pole.

And I really believe that in all areas of human science, we are going to find ourselves somewhere in the middle of that continuum.

We just haven't negotiated where just yet, how close to open access or how far from, from science expertise. Um uh But so I think that that at least in science, we're gonna get somewhere there in the middle, we just haven't figured it out yet, at least in science.

OK. Second thing is really hot off the presses like as my thoughts are evolving now about a generative A I.

Um And so uh uh so many people have heard about it. Chat GPT is the sort of front out of the gate barred is Google's equivalent.

They're both sort of generating um content based on our queries as humans. Um And uh obviously generative A I is gonna radically transform stuff.

We've already heard how it passes the USM licensing exams and the light the bar exam. And even this morning, I was reading an article about um generative A I in simulation in my sort of one of my other hats.

So, um it is an interesting question, how much we can use generative A I to write for Wikipedia versus rough draft content for Wikipedia, which then gets vetted for accuracy by humans.

Um So I'm just gonna leave that as an open question because I don't know yet either. Um Then uh wearing my educator hat, I'm gonna ask us to think about which is more important, something being easy or something being effective for what we learn or what we try to claim to teach our students, right?

So it turns out that you need to introduce some cognitive difficulty for learning to be retained and sustained in the long term.

That's just the science of learning. But um but are many times we want that easy shortcut, we think that that is actually better for us.

But if we teach our students that learning requires effort and writing for Wikipedia is effortful, but then you're gonna learn something deeply and retain it longer.

Maybe just maybe they'll be, they'll see value in doing that effort because they too want that knowledge to be retained rather than grab my phone and look it up.

I'm not sure I'll convince the younger generations of that necessarily. Um But I think we can at least frame the question for them so they can grapple with it or in partnership with us.

OK. So um the, the next meeting is really just to get to is something that I put here.

But I haven't really sort of like discussed with students. But I know our students, they wanna do good in the world.

They are medical students, nursing students, health professionals by definition. So we know they want high quality health information.

And so I think it's reasonable to say to them since you love and use Wikipedia, uh since you love and use social media, we know how much misinformation is being propagated on those platforms.

How about we be on the side of human health and, and meet our patients where they are. Um So maybe you wanna be propagating high quality health information on Wikipedia and then post it on your social media platform.

So you can tell people your followers, hey, I just improved health and content on, on schizophrenia or whatever. OK.

So, um and then lastly, uh thank you Catherine again for the reminder. Um I worked with a global consortium of osmosis affiliated medical students to create a Wikipedia editing course on the teachable platform and um really proud of it.

It's a very easy course that we could embed in any medical school globally. Um And uh any it could be used as a supplement or complement to the formal content that's on Wiki Ed.

Um But we have that course ready to go beautiful and we just need people that want to use it. So, so more than happy to explore uh providing that course to you all uh as you explore whether or not this might be um embed in your courses.

OK? I'll close with this final comment and whatever thoughts you guys have.

Jimmy Wales is quoted as saying, imagine a world in which every single person on the planet is given free access to the sum of all human knowledge.

And that's the sort of big Harry audacious goal of Wikipedia. And so I like to paraphrase him and say, imagine a world where all people have access to high quality health information clearly presented in their own language and that's the movement.

And so I'm really, really uh excited to hear your thoughts and reaction. Um I'll stop sharing screen and look at the chat and look at your questions.

So, all right. So um Chantelle, do I know it's a tiny community here?

So we can just look and see, I can look at it. You don't need to queue me up.

So, um Catherine, as has anyone looked overlap of med and Farm project? Yes.

Um There is a relationship sometimes you'll see on a Wikipedia page. Um um I'll look for it may be able to share it, but on the talk page, you can see when it's been a part of a health related assignment.

So there's a little cap and gown that shows you that this page was a part of a wiki ed course assignment in some year and so, and so you'll be able to see when a course when a page has been edited by more than one community of students.

I was thinking about your overlap of the, um, the top, as, uh as each working group identified the top things. And I was thinking about curriculum design.

Uh, um, right. And thinking, you know, about like Belmont's and New Med school.

Right. And, and, and what interesting things can you learn about where those, where those overlaps are around?

What the pharmacology people think are the most interesting drugs. Right.

And what the most interesting topics and II suspect those relationships are symbiotic, but I don't know what it tells us other than that there's probably, you know, I'm a hazard to guess that, um, like the, I noticed medical abortion was on, you know, was, was working group.

I imagine that's really high hits right now. Um, in the United States.

And, um, um, so anyhow, I just, I, it's, it's data but I don't know. II don't know if it's actionable data or what we would do with that to identify areas where we might need to do better with formal curricula.

Right. Uh, and my, my response to your thought there is this happens all the time outside of academia too.

Right. So, what are we teaching in the neurology course?

What are we teaching in the psychiatry? Oh, wait a minute.

There's overlap. Um, maybe we should coordinate across the course directors to make sure that we're not double dipping and teaching one thing and skipping something else in the med school content.

So it's totally reasonable to try to build those collaborations in a professional is a perfect example. So I agree.

Um the the task is human, human driven. That's the problem is we're all busy professionals and sometimes we don't pay attention outside of our, our courses or silos, right?

So um Andres says, do you know if there are any members, numbers methods in measuring clinical impact decisions? OK.

There are um and um I'll have to fish to find them. Um But um our Wikipedia Health and Sciences, you posted a link for us or a picture.

Do you want to describe what you said? Acknowledging the audio was a little choppy for you, so just speak uh clearly.

Can you hear me well? Yeah.

So, so this is uh just funny story that I hear about one year ago at the Radio Pia uh annual Congress is virtual and they were describing this funny story about a medical student who edited a Wikipedia article.

He put some of the radiation of the optic nerve uh nerve, his name and then this uh main impact on further medical text and articles afterwards.

So the the the the man was kind of discussing how to try to distinguish what is real and what is not from Wikipedia. But the thing that I was impressed about is the power because even a student who was actually was in India doing a short editing about the Wikipedia article got to change or even name uh and radiation, which we have not a name before.

So as you have, show us the numbers potential and I'm aware of these two. But as any students, lifelong students that we are, I'm sure that we learn a lot more about teaching something that from reading and memorizing it.

So I believe this is on the, on the, on the same, uh, on the same wave or in the same line that we want our students to, to these efforts and at the end, they will have love long life uh uh rewards because they were influencing the world.

Oh, yeah. Thank you.

Totally. Totally agree with you.

Yeah. I'd like to think that many times my students in the fourth year elective sign up for it because of the travel friendly nature of the elective.

They can go on interview season while they're in course in my course, which is great. It's not by accident.

I park it in the time of year. I do.

Um, but they broaden their thinking about what it means to be a physician in a net, socially networked era. And, um, and, and you know, what more could I ask for?

They're, they've, they've opened their minds more by the end of the course. And that's just fulfilling as an educator as well.

I'll share Um So the question about clinical thinking, I'll just oops wrong desktop stuff, share this desktop. Um So here um I posted a link to this box uh folder.

So this is uh what I call my wiki literature folder. Um on drive, anything that starts with a letter is not peer reviewed and anything that starts with a number are peer reviewed articles.

So um in the journal literature. So and somebody you asked about um uh so I'll just show you as an example.

Here's a plus one article about Wikipedia being a gateway for biomedical research and the distributions of citations in the English language, Wikipedia.

Um And um you know, uh journal of ethics talking about whether Wikipedia should be used in health sciences teaching. Um and then there was uh I think it's a literature review of um curricular inventory of Wikipedia editing in health professional schools.

Summer one of these, I forgot which number it is um looks at exactly whether clinicians are going to Wikipedia. Oh, here it is informal self education for clinicians, clinical decision making in medical practice.

So this, this was addressing the question you asked um uh Jenny, if you think about designing a course, uh the letter, the one that starts with an f um here are some course, learning objectives from different medical schools that have created courses.

So um so if you want to see the words that they're using in there, um you know, to create a course um and tying it to your local learn, you know, graduation competencies, learning level objectives.

Um These are some faculty that donated their learning objectives for anybody else that's interested in creating similar courses.

So uh we have a couple of minutes if we have other thoughts or questions, active courses for international learners. Yeah.

So um absolutely. Uh there are, I'm gonna run another course this summer, but my, my goal less than me doing it is to convince you to create a course at your school or an editing assignment at your school because then it's faster, broader, sustainable.

Um I consider me as an ally. I'm more than happy to coach you through designing courses or thinking of about the pros and cons of different strategies.

Um uh But II think I feel like I never dreamt that I would become the evangelist for Wikipedia, editing in medical schools and nursing schools globally and help press schools go away.

But that has been what I have become. It's the biggest thing I'm doing in my career and I just, I can't help but be excited about it and I want you guys to be excited about it too.

Yes, of course. I will share the slides.

Oh OK. Um Last minute I would love II as an educator, II force my students every time to do this.

I'm gonna put in the chat. Uh W I would like each of us to go around the room and share one thing I learned today or one thing I remain curious about or remain curious about or uh one thing I'll carry forward into my practice.

Um It can be small, it can be the same thing someone else said. But this way we're maximizing learning of our time together, Jenny, you were here first.

So I'm gonna invite you to go first. So uh ii know a couple of things.

I'm excited about this as a way to teach information literacy and information seeking skills to medical students and other health science students.

Um One thing, uh y you know, I, I've uh you know, an interprofessional standalone elective course perhaps, but um embedding into the curriculum probably is always the best bet to reach so many students.

So we are developing a health system science course and I'm thinking maybe that might be the place, but, you know, I'm excited to read a lot, but also feel like I'm gonna encounter a lot of skepticism from the medical faculty.

Um because, you know, this is not gonna be a test question on the U SE, which is what they are all about. That was a lot for my wife.

Oh, I love it. It's great.

It's great. Thank you.

Uh Andres. So I confirmed like my, my, the first impression I had a year ago about this too and I was kind of skeptic about the use of we, we make clinical decisions, but it's clear you, you have shown it with numbers that the powers maybe beyond the places that we, we practice are far away from our, our understanding.

And uh II sort of similar uh situation uh any because they are like this uh r uh um uh programs for students which I cannot modify.

But I'm sure II have a group of students which would be very interested to follow the course and maybe start editing Wikipedia.

Oh, thank you. Uh Give Catherine my colleague and friend the last word to wrap up today.

Oh my gosh. I'm so excited about the um the Wikipedia in a box.

Um And the power to transform learning in places that um you said broadband can't reach and I learned about the app. So did Jenny who knew?

Um So now I'm gonna ask students if they, they know about it. I think it's, I, you know, I think it's uh it's, it's really, it's really interesting.

There's so many choices that we have and how we consume and search for medical information and um um like you, I believe in doing good and I'm really thankful for you today and the time that you took him in um to, to talk with us and I'm thankful for Andres and Jenny for joining us and all of those who will be listening to this recording later.

Um So II invite everyone to keep the dialogue open. Thank you guys for coming.

Thank you. So much.

Thanks everyone. You bye-bye.