Video Time Stamps
"The Impact of Recent and Pending Decisions of the U.S. Supreme Court on Medical Education" - 1:56
"2023 Best Practices: Medical Education Techniques" - 26:43
"Clinical Teaching in the Setting of Burnout" - 1:29.30
"Feedback and Evaluation: When, Why, How" - 2:18.06
"How to Deliver Constructive Feedback and Navigate Resistance" - 302.45
This webinar features Dr. Caruso Brown, the co-author of the Checklist for Assessing Bias in Health Professions Education Content, discussing how medical education content suffers from implicit, and sometimes explicit, bias with regard to race, ethnicity, gender and other characteristics.
Bloom’s Taxonomy?
Bloom's Taxonomy is a model of education learning designed to help educators understand how information is acquired and processed. Bloom's Taxonomy provides a framework to help us understand what we want students to do with information, and it defines knowledge as a series of levels, from simple recall up to the ability to create knowledge itself. It is one of many tools that can be used as an instructor develops courses and identifies strategies to help engage students.
Bloom's Taxonomy can be used to develop:
Learning outcomes.
Formative and summative assessments.
Items that are aligned with the program and course-level objectives.
Learning Outcomes
Learning outcomes define what students will be able to do by the end of a unit or course.
Learning outcomes:
Begin with an action verb.
Are observable and measurable.
Focus on a student deliverable, such as a project or presentation.
Writing Strong Learning Outcomes
Now, that we've had a chance to learn what learning outcomes are, let's create some.
Step 1: Describe what students will be able to do by the end of the assignment.
Example: project, presentation
Step 2: Use Bloom's Wheel to identify an action verb to describe the outcome.
Use only one action verb for each outcome.
The action verb should refer to the activity not the process.
Avoid general verbs that cannot be directly observed, like "know" and "understand" because the goal is to create active learning experiences.
Example: By the end of this lesson, learners will be able to demonstrate...
Step 3: Follow the action verb with the skill or knowledge area that learners will be asked to demonstrate.
Make sure that the phrase captures an activity that is observable, measurable, and meaningful.
Example: By the end of this lesson, students will be able to demonstrate how to select a research sample.
Resources:
Using Bloom’s Taxonomy to Write Learning Objectives
Bloom’s Taxonomy Teacher Planning Kit
Backward Design
Backward design asks:
What learning experiences or activities will help them know, learn, understand or be able to do the things we want them to learn?
Backward Design Template
Use the following backward design template to craft your lesson:
STEP 1: Desired Results
‘Students will be able to...’
STEP 2: Assessment Evidence/Performance Tasks
‘Students will read and write about...’
‘Students will talk about...’
‘Students will be able to perform a...’
‘Students will answer questions about...’
‘Students will ask about...’
STEP 3: Learning Experiences and Activities
This can be a detailed list of active learning experiences and activities.
Writing Quality Multiple-choice, or ‘One-Best-Answer,’ Test Items
A good test question must satisfy two basic criteria:
The test question must address important content.
Items must be well-structured and avoid flaws and complicated options.
Item Templates
The overall structure of an item can be depicted by an item template. You can typically generate many items using the same template. For example, the following template could be used to generate a series of questions related to gross anatomy.
A (patient description) is unable to (functional disability). Which of the following is most likely to have been injured.
This is a question that could be written using this template:
A 65-year-old man has difficulty rising from a seated position and straightening his trunk, but he has no difficulty flexing his leg. Which of the following muscles is most likely to have been injured?
Gluteus maximus*
Gluteus minimus
Hamstrings
Iliopsoas
Obturator internus
Many questions can be presented with the context of a patient vignette. The stem should consist of a single, clearly formulated problem. The lead-in of the stem must pose a clear question, that the student can pose an answer without looking at the options (National Board of Medical Examiners, p. 56, 2002). The patient vignette may include some, or all, of the following components:
Age, Gender (e.g., A 45-year-old man)
Site of Care (e.g., comes to the emergency department)
Presenting Complaint (e.g., because of a headache)
Duration (e.g., that has continued for 2 days.)
Patient History (with Family History?)
Physical Findings
+/- Results of Diagnostic Studies
+/- Initial Treatment, Subsequent Findings, etc.
Ref: (National Board of Medical Examiners, p. 38, 2002)
The Basic Rules for One-Best-Answer Items
Use the following six rules to ‘test’ your items. If a question passes all six, it is probably well-phrased and focused on an appropriate topic.
Each item should focus on an important concept.
Focus on problems that would be encountered in real life.
Avoid trivial, ‘tricky’ or overly complex questions.
Each item should assess the application of knowledge, not the recall of an isolated fact.
Clinical vignettes provide a good basis for a question.
For the clinical sciences, each question should begin with the patient’s presenting problem, followed by the history (including duration of signs and symptoms), physical findings, results of diagnostic studies, initial treatment, subsequent findings, etc. Vignettes include only a subset of this information, but the information should be provided in this specified order.
For the basic sciences, patient vignettes may be very brief.
Laboratory vignettes are also appropriate.
The stem of the item must pose a clear question, and it should be possible to arrive at an answer with the options provided.
To determine if the question is focused, cover up the options and see if the question is clear and if the examinees can pose an answer based only on the stem.
Rewrite the stem and/or option if they could not.
All distractors, i.e., incorrect options, should be homogenous. They should fall into the same category as the correct answer, e.g., diagnoses, tests, treatments, prognoses, disposition alternatives.
Rewrite any dissimilar distractors.
Avoid using ‘double options,’ e.g., do W and X; do Y because of Z, unless the correct answer and all distractors are double options.
Rewrite double options to focus on a single point.
All distractors should be plausible, grammatically consistent, logically comparable and of the same (relative) lengths as the correct answer.
Order the options in logical order, e.g., numeric, or in alphabetical order.
Avoid technical item flaws that provide special benefit to test-wise examinees or that pose irrelevant difficulty.
Do NOT write any questions of the form:
‘Which of the following statements is correct?’
‘Each of the following statements is correct EXCEPT.’
Ref: (National Board of Medical Examiners, p. 33, 2002)
Resources
National Board of Medical Examiners (2002). Constructing Written Test Questions for the Basic and Clinical Sciences (3rd edition)
National Board of Medical Examiners (2020). National Board of Medical Examiners Item Writing Guide
Authentic Assessments
In addition to traditional exams (e.g., multiple-choice texts), there are many ways that instructors can assess their students using alternative assessments.
Alternatives assessments include:
|
![]() |
Selecting an Authentic Assessment
When you choose an authentic assessment, begin by thinking about a lesson you share with your students. What kind of authentic assessment(s) might be suitable?
Example:
Content: Suppose your students are learning how to deliver difficult news to a patient.
Authentic Assessment: You might consider asking students to role-play the exchange between themselves and their patient.
Note: Whenever you administer an (authentic) assessment, it is critical to share the assignment parameters and your expectations with your students. Knowing what you are expecting from them will help facilitate a more fruitful learning experience.
Teaching Tip
Consider dividing larger projects and assignments into smaller, lower-stakes components. This will give you a chance to provide feedback to your students as they build toward the larger, graded deliverable.
Grading vs. Assessment
It is important to recognize that grading and assessment are not the same thing.
Grading is a way of ‘scoring’ students’ learning and performance, while assessments are used to help students learn and can include ungraded tasks.
Inclusive Teaching
Many of us have heard the terms diversity and inclusion, which are often conflated. They are, however, two distinct terms that are critical to classroom practice. Diversity references the similarities and differences between individuals, such as one’s personality and individual identity. It implies a variety of characteristics like race, sex or age. Inclusion refers to the efforts used to embrace those differences.
Inclusivity is defined as, ‘The practice of including people across difference. Inclusivity implies an intentional practice of recognizing and working to mitigate biases that lead to marginalization or exclusion of some people” (https://lse.ascb.org/evidence-based-teaching-guides/inclusive-teaching/philosophy/).
Inclusive teaching is committed to serving the needs of all students, regardless of background, identity or ability. It serves to support student learning, as well as the interactions with those from whom they wish to learn.
Inclusive teaching:
Strategies for Inclusive Teaching
In order to teach inclusively, it is important to recognize how to make pedagogical choices that promote the intellectual, social and emotional well-being of all students.
Think about what inclusive teaching will look like in your classroom. Think about the strategies for inclusive teaching that you already use and the ones you might add or modify. Following are some strategies to consider:
Resources:
Atlases on diseases in different skin tones
https://lse.ascb.org
Universal Design for Learning (UDL)
Universal Design for Learning (UDL) is a a research-based approach to teaching and learning that focuses on designing courses that are accessible, inclusive and supportive of the students’ diverse needs and lived experiences.
UDL Guidelines
The UDL guidelines offer a set of suggestions that can be used in any domain to ensure that all learners can access and participate in meaningful, challenging learning opportunities (CAST, 2018).
How can I engage all my students?
Provide multiple means of engagement to tap individual learners’ interests, to challenge them appropriately and to motivate them to learn. Examples of engagement include:
How can I represent information?
Provide multiple means of representation to give students various ways to acquire, process and integrate knowledge and information. Examples of representation include:
How can I invite students to demonstrate what they’ve learned?
Provide learners with multiple means of ACTION & EXPRESSION. This particular UDL approach offers diverse ways for learners to be involved with course content, their peers and the instructor. By building in different engagement opportunities, you can help learners see the relevance of disciplinary knowledge in their academic, professional and personal lives. Examples of demonstration include:
Resources:
Rubrics
Rubrics:
Rubrics are ideal for complex projects, assignments or assessments that require the mastery of many skills. The following rubric demonstrates the various performance levels across a range of criteria:
Creating a Rubric
There is a set of basic steps to creating a rubric:
Ref: https://www.brown.edu/sheridan/teaching-learning-resources/teaching-resources/course-design/classroom-assessment/grading-criteria/designing-rubrics
Selecting a Rubric
Creating a rubric requires time and skill. You may wish, instead, to select an existing rubric that you can use as is or modify, if necessary. Following is a set of links with rubrics:
https://www.csu.edu/CTRE/pdf/rubricexamples-all.pdf
https://www.cmu.edu/teaching/designteach/teach/rubrics.html
https://www.rcampus.com/rubricshellc.cfm?mode=gallery&sms=publicrub&glid=5&
https://www.aacu.org/initiatives/value-initiative/value-rubrics
https://templatelab.com/rubric-templates/
https://www.wordtemplatesonline.net/40-free-rubric-templates-assessment-examples-word-pdf/
Part I: Student Accommodations: Myth or Miracle
Passcode: ks1W38
Medical Podcasts
American Medical Association EdHub
Issues in Science and Technology
Let Me Ask You Something: Philosophy and Medical Education
New England Journal of Medicine AI Grand Rounds
Directly and Covertly Observing Care: How it Can Transform Medical Education and Improve Clinical Practice. Direct, covert observation of health care is a novel and underutilized tool to assess and provide feedback to health care trainees. In this episode of the On Becoming a Healer podcast, experts talk about two such approaches: the unannounced standardized patient and patient-collected audio. In the former, standardized patients are sent incognito into practice settings, and in the latter real patients volunteer to record their visits for the benefit of trainees and experienced practitioners alike. Both approaches address the question, “How are our learners and clinicians performing in the real world?” They also identify those who do may do well on simulations (e.g., with SPs and OSCEs) but underperform in the clinical setting. As one of the guests observed, “If McDonalds is using secret shoppers to improve services, shouldn’t we be doing the same in health care (but with a lot more rigor) where the stakes are so much higher?”
****************************************************
Pedagogical Podcasts
The Faculty Chronicles (Touro University)
****************************************************
General Podcasts
****************************************************
This list of podcasts is a living document that captures a set of curated recommendations from national and international faculty-developers. We hope you find one or two you like!
NOTICE OF NONDISCRIMINATORY POLICY AS TO STUDENTS
The New York Medical College admits students of any race, color, national and ethnic origin to all the rights, privileges, programs, and activities generally accorded or made available to students at the college. It does not discriminate on the basis of race, color, national and ethnic origin in administration of its educational policies, admissions policies, scholarship and loan programs, and athletic and other school-administered programs. See full non-discrimination statement with contact info.