Skip to Main Content
NYMC Library Banner
Ask a Librarian

SOM Faculty Development

2023 Teaching and Learning Symposium for Faculty Educators

Video Time Stamps

"The Impact of Recent and Pending Decisions of the U.S. Supreme Court on Medical Education" - 1:56

"2023 Best Practices: Medical Education Techniques" - 26:43

"Clinical Teaching in the Setting of Burnout" - 1:29.30

"Feedback and Evaluation: When, Why, How" - 2:18.06

"How to Deliver Constructive Feedback and Navigate Resistance" - 302.45

Classroom Pedagogy

Checklist for Assessing Bias in Health Professions Education Content

This webinar features Dr. Caruso Brown, the co-author of the Checklist for Assessing Bias in Health Professions Education Content, discussing how medical education content suffers from implicit, and sometimes explicit, bias with regard to race, ethnicity, gender and other characteristics.

Bloom’s Taxonomy? 

Bloom's Taxonomy is a model of education learning designed to help educators understand how information is acquired and processed. Bloom's Taxonomy provides a framework to help us understand what we want students to do with information, and it defines knowledge as a series of levels, from simple recall up to the ability to create knowledge itself. It is one of many tools that can be used as an instructor develops courses and identifies strategies to help engage students. 

Bloom's Taxonomy can be used to develop: 

  • Learning outcomes. 

  • Formative and summative assessments. 

  • Items that are aligned with the program and course-level objectives.

 

Bloom's Taxonomy

 

Learning Outcomes

Learning outcomes define what students will be able to do by the end of a unit or course. 

Learning outcomes: 

  • Begin with an action verb. 

  • Are observable and measurable. 

  • Focus on a student deliverable, such as a project or presentation. 

 

Writing Strong Learning Outcomes 

Now, that we've had a chance to learn what learning outcomes are, let's create some.

Step 1: Describe what students will be able to do by the end of the assignment. 

Example: project, presentation 

Step 2: Use Bloom's Wheel to identify an action verb to describe the outcome. 

  • Use only one action verb for each outcome. 

  • The action verb should refer to the activity not the process. 

  • Avoid general verbs that cannot be directly observed, like "know" and "understand" because the goal is to create active learning experiences. 

Example: By the end of this lesson, learners will be able to demonstrate... 

Step 3: Follow the action verb with the skill or knowledge area that learners will be asked to demonstrate. 

  • Make sure that the phrase captures an activity that is observable, measurable, and meaningful. 

Example: By the end of this lesson, students will be able to demonstrate how to select a research sample. 

 

Resources: 

Using Bloom’s Taxonomy to Write Learning Objectives

Bloom’s Taxonomy Teacher Planning Kit 

Backward Design

Backward design asks:

  • What do I want my students to know, learn, understand or be able to do?
  • How will I know if my students learned what I wanted them to know, learn, understand or be able to do? What kinds of assessments, evidence or performance tasks should I administer to determine this?
  • What learning experiences or activities will help them know, learn, understand or be able to do the things we want them to learn?

Backward Design Template

Use the following backward design template to craft your lesson:

STEP 1: Desired Results

‘Students will be able to...’

STEP 2: Assessment Evidence/Performance Tasks

‘Students will read and write about...’

‘Students will talk about...’

‘Students will be able to perform a...’

‘Students will answer questions about...’

‘Students will ask about...’

STEP 3: Learning Experiences and Activities

This can be a detailed list of active learning experiences and activities.

Writing Quality Multiple-choice, or ‘One-Best-Answer,’ Test Items 

A good test question must satisfy two basic criteria: 

  1. The test question must address important content. 

  1. Items must be well-structured and avoid flaws and complicated options. 

Item Templates 

The overall structure of an item can be depicted by an item template. You can typically generate many items using the same template. For example, the following template could be used to generate a series of questions related to gross anatomy. 

A (patient description) is unable to (functional disability). Which of the following is most likely to have been injured. 

This is a question that could be written using this template: 

A 65-year-old man has difficulty rising from a seated position and straightening his trunk, but he has no difficulty flexing his leg. Which of the following muscles is most likely to have been injured? 

  1. Gluteus maximus* 

  1. Gluteus minimus 

  1. Hamstrings 

  1. Iliopsoas 

  1. Obturator internus 

Many questions can be presented with the context of a patient vignette. The stem should consist of a single, clearly formulated problem. The lead-in of the stem must pose a clear question, that the student can pose an answer without looking at the options (National Board of Medical Examiners, p. 56, 2002). The patient vignette may include some, or all, of the following components: 

Age, Gender (e.g., A 45-year-old man) 

Site of Care (e.g., comes to the emergency department) 

Presenting Complaint (e.g., because of a headache) 

Duration (e.g., that has continued for 2 days.) 

Patient History (with Family History?) 

Physical Findings 

+/- Results of Diagnostic Studies 

+/- Initial Treatment, Subsequent Findings, etc. 

Ref: (National Board of Medical Examiners, p. 38, 2002) 

 

The Basic Rules for One-Best-Answer Items 

Use the following six rules to ‘test’ your items. If a question passes all six, it is probably well-phrased and focused on an appropriate topic. 

  • Each item should focus on an important concept.  

  • Focus on problems that would be encountered in real life.  

  • Avoid trivial, ‘tricky’ or overly complex questions.  

  • Each item should assess the application of knowledge, not the recall of an isolated fact. 

  • Clinical vignettes provide a good basis for a question. 

  • For the clinical sciences, each question should begin with the patient’s presenting problem, followed by the history (including duration of signs and symptoms), physical findings, results of diagnostic studies, initial treatment, subsequent findings, etc. Vignettes include only a subset of this information, but the information should be provided in this specified order. 

  • For the basic sciences, patient vignettes may be very brief. 

  • Laboratory vignettes are also appropriate. 

  • The stem of the item must pose a clear question, and it should be possible to arrive at an answer with the options provided. 

  • To determine if the question is focused, cover up the options and see if the question is clear and if the examinees can pose an answer based only on the stem. 

  • Rewrite the stem and/or option if they could not. 

  • All distractors, i.e., incorrect options, should be homogenous. They should fall into the same category as the correct answer, e.g., diagnoses, tests, treatments, prognoses, disposition alternatives. 

  • Rewrite any dissimilar distractors. 

  • Avoid using ‘double options,’ e.g., do W and X; do Y because of Z, unless the correct answer and all distractors are double options. 

  • Rewrite double options to focus on a single point. 

  • All distractors should be plausible, grammatically consistent, logically comparable and of the same (relative) lengths as the correct answer. 

  • Order the options in logical order, e.g., numeric, or in alphabetical order. 

  • Avoid technical item flaws that provide special benefit to test-wise examinees or that pose irrelevant difficulty. 

  • Do NOT write any questions of the form

  • ‘Which of the following statements is correct?’ 

  • ‘Each of the following statements is correct EXCEPT.’ 

 

Ref: (National Board of Medical Examiners, p. 33, 2002)  

 

 

Resources 

National Board of Medical Examiners (2002). Constructing Written Test Questions for the Basic and Clinical Sciences (3rd edition)  

National Board of Medical Examiners (2020). National Board of Medical Examiners Item Writing Guide 

Authentic Assessments

In addition to traditional exams (e.g., multiple-choice texts), there are many ways that instructors can assess their students using alternative assessments.

 

 

 

Alternatives assessments include:

  • Quick-writes
  • Portfolios
  • Presentations.
  • Discussion boards.
  • Journals, essays.
  • Demonstrations.
  • Exhibitions.
  • Role-playing.
  • Infographics. (See example to the right.)
  • How-to manuals.
  • Pamphlets/brochures.
  • Persuasive letters.
  • Multimedia/interactive posters.

 

Selecting an Authentic Assessment

When you choose an authentic assessment, begin by thinking about a lesson you share with your students. What kind of authentic assessment(s) might be suitable?

Example:

Content: Suppose your students are learning how to deliver difficult news to a patient.

Authentic Assessment: You might consider asking students to role-play the exchange between themselves and their patient.

Note: Whenever you administer an (authentic) assessment, it is critical to share the assignment parameters and your expectations with your students. Knowing what you are expecting from them will help facilitate a more fruitful learning experience.

Teaching Tip

Consider dividing larger projects and assignments into smaller, lower-stakes components. This will give you a chance to provide feedback to your students as they build toward the larger, graded deliverable.

Grading vs. Assessment

It is important to recognize that grading and assessment are not the same thing.

Grading is a way of ‘scoring’ students’ learning and performance, while assessments are used to help students learn and can include ungraded tasks.

  •  

Inclusive Teaching

Many of us have heard the terms diversity and inclusion, which are often conflated. They are, however, two distinct terms that are critical to classroom practice. Diversity references the similarities and differences between individuals, such as one’s personality and individual identity. It implies a variety of characteristics like race, sex or age. Inclusion refers to the efforts used to embrace those differences.

Inclusivity is defined as, ‘The practice of including people across difference. Inclusivity implies an intentional practice of recognizing and working to mitigate biases that lead to marginalization or exclusion of some people” (https://lse.ascb.org/evidence-based-teaching-guides/inclusive-teaching/philosophy/).

Inclusive teaching is committed to serving the needs of all students, regardless of background, identity or ability. It serves to support student learning, as well as the interactions with those from whom they wish to learn.

Inclusive teaching:

  • Honors diversity by recognizing the value that each member of our (learning) community brings to our collective experience.
  • Actively cultivates inclusions by inviting our students to share their rich and unique experiences.
  • Cultivates student belonging and a just climate where students feel free and safe to contribute.
  • Creates learning activities and experiences that are accessible to all types of learners.

 

Strategies for Inclusive Teaching

In order to teach inclusively, it is important to recognize how to make pedagogical choices that promote the intellectual, social and emotional well-being of all students.

 

Think about what inclusive teaching will look like in your classroom. Think about the strategies for inclusive teaching that you already use and the ones you might add or modify. Following are some strategies to consider:

  1. Foster a safe environment that cultivates students’ sense of belonging within which students are inspired to contribute and are empowered to share their rich and unique perspectives.
  2. Fashion activities and learning experiences that empower students to connect them to their own rich and unique perspectives.
  3. Fashion activities and learning experiences that empower students to share their perspectives with others.
  4. Review your course content and use readings, videos, activities, etc., that are respect diversity and promote belonging.
  5. Use examples in your teaching that promote inclusivity and offer culturally diverse exemplars, such as Dr. Percy Julian (chemist who pioneered the chemical synthesis of medicinal drugs and Dr. Alexa Canady (neurosurgeon).
  6. Use the first few classes to learn your students’ names. Ask students to introduce themselves and share an interest that starts with the first letter of their name, e.g., ‘My name is ‘Daiki, and I like diving.’ For larger classes, consider asking students to write their name on a name tent or share their name before sharing a comment or question.
  7. Invite students to visit you during office hours. This can also help you learn their names and learn about their interests and needs.
  8. Learn about your students’ backgrounds, personal and professional experiences and interests and customize your approaches accordingly.
  9. Use readings, video, activities, etc., that respect diversity and promote belonging.
  10. Make course materials accessible in multiple formats, such as audio and visual.
  11. Incorporate group work and other activities. Then, create space and time for students to reflect and discuss course topics with their peers.
  12. Identify alternatives to traditional assessments, such as videos or webpages.
  13. Objectively identify students who might be struggling with the material and offer them help.

 

Resources:

Atlases on diseases in different skin tones

https://lse.ascb.org

Universal Design for Learning (UDL)

Universal Design for Learning (UDL) is a a research-based approach to teaching and learning that focuses on designing courses that are accessible, inclusive and supportive of the students’ diverse needs and lived experiences.

UDL Guidelines

The UDL guidelines offer a set of suggestions that can be used in any domain to ensure that all learners can access and participate in meaningful, challenging learning opportunities (CAST, 2018).

How can I engage all my students?

Provide multiple means of engagement to tap individual learners’ interests, to challenge them appropriately and to motivate them to learn. Examples of engagement include:

  • Incorporating real-life examples.
  • Connecting topics to current events.
  • Creating experiential learning activities.
  • Asking for student feedback.
  • Including frequent instructor feedback.

How can I represent information?

Provide multiple means of representation to give students various ways to acquire, process and integrate knowledge and information. Examples of representation include:

  • Giving learners resources, such as animations and simulations, that they can control in sound and speed and videos that student can pause and playback at their own pace.
  • Providing transcripts/captions for video clips.
  • Recording the class, so students can go back and listen at their own pace.
  • Chunking lectures and long readings into smaller sections to help learners develop their knowledge.
  • Providing links to resources to help students better understand certain key terms.
  • Ensuring learning resources are accessible.

How can I invite students to demonstrate what they’ve learned?

Provide learners with multiple means of ACTION & EXPRESSION. This particular UDL approach offers diverse ways for learners to be involved with course content, their peers and the instructor. By building in different engagement opportunities, you can help learners see the relevance of disciplinary knowledge in their academic, professional and personal lives. Examples of demonstration include:

  • Building in ungraded practice.
  • Incorporating rubrics.
  • Providing opportunities for student reflection.

 

Resources:

UDL Guidelines (CAST, 2018)

Rubrics

Rubrics:

  • Can help instructors communicate their expectations to students.
  • Can help instructors assess student work fairly and efficiently.
  • Can provide students with a ‘roadmap’ for the assignment or assessment.
  • Can provide students with informative feedback.
  • Can prompt students to reflect on their own work.

Rubrics are ideal for complex projects, assignments or assessments that require the mastery of many skills. The following rubric demonstrates the various performance levels across a range of criteria:

 

Creating a Rubric

There is a set of basic steps to creating a rubric:

  • Define the purpose of the assignment or assessment.
  • Decide what type of rubric you will be using, i.e., analytic, holistic.
  • Define the criteria.
  • Design the rating scale.
  • Write descriptions for each level of the rating scale.
  • Create your rubric.

Ref: https://www.brown.edu/sheridan/teaching-learning-resources/teaching-resources/course-design/classroom-assessment/grading-criteria/designing-rubrics

 

Selecting a Rubric

Creating a rubric requires time and skill. You may wish, instead, to select an existing rubric that you can use as is or modify, if necessary. Following is a set of links with rubrics:

https://www.csu.edu/CTRE/pdf/rubricexamples-all.pdf

https://www.cmu.edu/teaching/designteach/teach/rubrics.html

https://www.rcampus.com/rubricshellc.cfm?mode=gallery&sms=publicrub&glid=5&

https://www.aacu.org/initiatives/value-initiative/value-rubrics

https://templatelab.com/rubric-templates/

https://www.wordtemplatesonline.net/40-free-rubric-templates-assessment-examples-word-pdf/

Med Ed Grand Rounds

Podcasts

Medical Podcasts

American Medical Association EdHub

Issues in Science and Technology

Let Me Ask You Something: Philosophy and Medical Education

New England Journal of Medicine AI Grand Rounds

On Becoming a Healer

  • Directly and Covertly Observing Care: How it Can Transform Medical Education and Improve Clinical Practice. Direct, covert observation of health care is a novel and underutilized tool to assess and provide feedback to health care trainees. In this episode of the On Becoming a Healer podcast, experts talk about two such approaches: the unannounced standardized patient and patient-collected audio. In the former, standardized patients are sent incognito into practice settings, and in the latter real patients volunteer to record their visits for the benefit of trainees and experienced practitioners alike.  Both approaches address the question, “How are our learners and clinicians performing in the real world?” They also identify those who do may do well on simulations (e.g., with SPs and OSCEs) but underperform in the clinical setting. As one of the guests observed, “If McDonalds is using secret shoppers to improve services, shouldn’t we be doing the same in health care (but with a lot more rigor) where the stakes are so much higher?”

  • Dire Consequences:" When Students do not Receive appropriate Accommodations on the USMLE Examinations. Medical students are frequently denied accommodations on the USMLE examinations, administered by the NBME, despite receiving accommodations at their medical school following documentation of a disability. Perhaps the most common request that is denied is extra time, and yet the last episode of the On Becoming a Healer podcast featured insight from a testing expert that time-limited tests are less valid and less equitable. In this episode, Kristina Petersen, PhD, a biochemistry professor at New York Medical College who received a 2022 award for Excellence in Scholarly Work from the Coalition for Disability Access in Health Science Education and the AAMC/Alpha Omega Alpha Robert J. Glaser Distinguished Teaching Award discusses a national survey that she and her colleagues conducted of USMLE Step-1 accommodation denial. The podcast concludes with a discussion about how the NBME could make the test fair and valid for everyone.
  • Why It's Time to Remove Time Limits on Tests, Like the USMLE Exams. There is a widely held perception that being able to complete a test quickly is an indication of mastery when compared with those who need more time. As a result, it is often difficult to obtain accommodations on high stakes examinations, including the USMLE exams.  Many students who request extra time because of a disability are denied accommodations and many other students who need it aren't eligible (e.g., English is a second language) or are inhibited from applying (e.g., Veterans, older students, students from certain cultural backgrounds). But what does the evidence show?
  • Running the Gauntlet: My Journey into Medicine with a Learning Disability sets the stage for two podcasts that will follow with experts on the science of student learning assessment and its implications for the USMLE examinations. These will address questions such as: Does struggling with multiple-choice tests under time pressure predict anything about future performance in the clinical setting? Do time limits make tests more or less valid and reliable?  What are implications of denying students accommodations on the USMLE examinations?  And, most importantly, what can we do about the documented perverse effects of our current system of assessment on equity and inclusion and, ultimately on the quality and diversity of our physician workforce? 
  • Why are Doctors Turning to ChatGPT for Help Relating to Patients? is in response to a recent article in the New York Times, "When Doctors Use a Chatbot to Improve Their Bedside Manner," which reports on the growing numbers of physicians turning to AI to tell them what to say to their patients, particularly during difficult conversations.
  • The episode titled Prescription Opioid Reductions and Suicide: What Should Caring Physicians Do in the Face of Uncertainty explores the dilemma of caring for patients with prescription opioid dependence when the physician feels it might be in the patient’s best interest to taper the medication and the patient is reluctant and fearful of withdrawal symptoms and suffering. What does the research show?  And what happens when the data is conflicting? How do you apply uncertain information to the care of individual patients?
  • In the episode titled My Patient’s in Shackles: Can We Take these Off? the guest describes a patient who is intubated, sedated, and paralyzed in the ICU, and still in shackles – and asks “why”? In a recent publication, he writes: “Those of us in positions of power may disregard the shackle, or not question its purpose, or even propose that it is justified.” We explore questions of the physician’s roles and responsibilities.
  • “As a healing profession, why do we harm our own?” “What downstream effects is it having on future physicians?” and “What can we do about it?” This is a complex multifaceted problem that has stubbornly persisted, even at institutions that have invested considerable effort and resources to change. What will it take?

****************************************************

Pedagogical Podcasts

The Faculty Chronicles (Touro University)

Modern Campus

Tea for Teaching

Teaching for Student Success

****************************************************

General Podcasts

In Reality

****************************************************

This list of podcasts is a living document that captures a set of curated recommendations from national and international faculty-developers. We hope you find one or two you like!

Suggested Reading

>   MedEd Pearls from the Harvard Macy Institute