Leaders in Medical Education

Dr. Eric Warm, Director of the Internal Medicine Training Program at University of Cincinnati

Jeremie Lever
Oct 14, 2015

Dr. Eric Warm is Professor of Medicine at University of Cincinnati and Director of the Internal Medicine Trianing Program. He and the Department of Medicine at University of Cincinnati are demonstrating creativity and leadership in the area of medical resident training through the conception of Observable Practice Activities. A new resident assessment program was developed based on competency, allowing collection of discreet data on many aspects of their training program and continuous quality improvement. They have proactively and innovatively incorporated ACGME’s Next Accreditation System requirements, positioning Cincinnati to improve quality of care as well as efficacy in medical education.

How did you first become interested in medicine? Medical education?
My father thought medicine was a good profession, and I liked trying to understand how biological systems worked, so after getting a degree in biology, medicine seemed like a good fit (another way to put this is: I didn’t have a lot of other choices). I believe this path is more common than admitted. I had no true exposure to what physicians do, just an idealized notion, and so I went to medical school. The reality of medicine is complex, both clear and nuanced. I think students should be more aware of their choices than I was. Luckily for me things have worked out, but in my experience as a program director, watching hundreds of students become physicians, it isn’t always so.

As far as medical education, when I was a resident the best part of any day for me was when I was teaching something to somebody – be it a patient or a resident. The latin root for doctor is docere, which means to instruct, teach, or point out. We are all learners – it depends on what position we happen to be in at the time. The skills you use to teach a patient about how to take their medication correctly are the same skills you use to teach an intern how to manage diabetes. It’s a beautiful echo. After residency, I was selected as chief resident. A mentor of mine at the time told me you need to have a niche to be successful in academic medicine. Mine was finding better ways to learn and teach.

Can you share your background on how you got to where you are right now?
After chief residency, I joined the faculty as an associate program director in the residency program, practicing general internal medicine in the clinic and on the wards. Several years after that, I became director of the resident ambulatory practice, and along with my fabulous team, we joined several national collaboratives such as the AAMC’s Academic Chronic Care Collaborative, and the ACGME’s Education Innovations Project. We developed many innovative programs together and reshaped our residency. Six years ago I became the program director, and we’ve continued to develop our wonderful education team.

Please tell us about your program in Internal Medicine resident assessment at University of Cincinnati College of Medicine. Why was it needed? How does it address current challenges faced in resident training in Internal Medicine and other fields?
A few years ago we realized we were in the assessment business. There are many tasks and challenges every residency program has, but the base job is to ensure all graduates are good enough to practice independently. That might sound like a simple self-evident idea, but it wasn’t to us. Our assessment system was poor. It could tell us who the best and worst performers were, but not what these people could do, or not do, nor could it tell us who was really ready for unsupervised practice. People typically describe assessment in terms of formative (helping people get better in the moment), and summative (letting the world know how good someone is at a certain point in time). Our previous system did both things poorly. We read the works of Olle ten Cate, and Cees Van der Vleuten, and it changed how we thought about assessment. ten Cate’s work focuses on using entrustment of specific skills over time as an assessment framework. Van der Vleuten emphasizes that the most important function of assessment is assessment FOR learning, and not assessment OF learning (i.e. almost all assessment should be formative, to help people improve performance). Van der Vleuten also emphasizes a ‘program of assessment’ made up of many sources and types of data, where the stakes of any decision should be directly related to the number of assessments that make up that decision (in other words, the bigger the decision the larger the data set that should be used to make it). Our team took these ideas and tried to operationalize them on a large scale for an internal medicine residency. We created a system of Observable Practice Activities (OPAs) mapped to ACGME milestones, and deployed this throughout the medical center. We use entrustment over time as the assessment framework. Residents in our system now get thousands of assessment data points of many types over the course of a residency, and we can measure progressive entrustment over time. We can also set an entrustment threshold a resident must meet in order to practice unsupervised. If a resident struggles, we can give specific directed feedback to help them improve. We’ve now collected nearly 400,000 data points, and we are learning a lot on how to use these assessment points FOR learning.

What are 2-3 changes you would like to see in the current medical education system? Healthcare system?
Medical education should not be as divided as it is now. UME, GME, fellowship, practice – all are separated by bright lines. We need to develop a way to connect these entities, to study and measure progress and maintenance of competence over time in a continuum. People progress at different paces, and have different needs. We act as if 3 or 4 years in one place somehow fits all comers. To borrow a medical term, training physicians relies on a sensitive, not specific model. If we assume that all people are ready when a given amount of years is up, then it must be assumed that this is long enough that even the slowest will be fully prepared in that time. Of course this is not true. We need a more specific system – one in which we know people are ready to move to the next stage when they are truly ready. And this information should be carried forward from one stage to the next.

Do you have any final thoughts regarding the medical profession as a whole?
Remember – it’s not about whether teachers teach, it’s about whether learners learn.