Five years ago, a medical school dean made a casual request for a “one-page handout” to address persistent biases in the MD program curriculum related to many social and structural determinants of health. In response, Dr. Caruso Brown developed the Upstate Bias Checklist1, which was the subject of a NCEAS webinar2 in 2019. The Checklist is a free, publicly available tool that anyone can use when developing or reviewing content for learners at any level in the health professions. It is designed to avoid burdening learners with the responsibility to call attention to biased material, although it can and has been used by learners to provide feedback on content and educational experiences. It is expressly not intended to be punitive toward educators, but instead to promote self-reflection, faculty development and quality improvement in education, while also preventing the harm that comes when biased content reaches learners—harm that not only impacts our learners but also impacts their and our future patients.3
Use of the bias checklist to review curricular content in the MD program has been required at SUNY Upstate Medical University since June of 2020. In October of 2020, together with Dr. Lauren Germain in the Office of Evaluation, Assessment and Research at SUNY Upstate and Dr. Maria Alonso Luaces and Dr. Natabhona Mabachi in the Office of Diversity and Inclusion at the University of Kansas School of Medicine (KU), we formed the Bias Checklist Collaborative, a group with members from more than fifteen institutions, all committed to address bias and integrating health equity into medical and health professions education.
Recently, we began a review of bias checklist data from Upstate and KU, led by Hannah Connolly, an MD/PhD/MPH student at SUNY Upstate and Syracuse University. We analyzed 373 bias checklists completed between February 2020 until December 2022. We first examined the characteristics of those who completed the bias checklists and the frequency of content being flagged as “at risk for bias” within a given domain (e.g., race, disability status, etc.). Then we examined the relationship between the probability of affirming an intent to modify the curricular content (after having the content flagged) and checklist user characteristics.
Checklist users were most likely to identify as White (79%) and male (57%); the largest single age group represented was 55-64 (28%). Approximately 66% of the survey respondents reported completing some type of training around using the checklist; 60% of respondents reported that they were evaluating their own content (a “content creator”; the checklist can also be used by others, including course directors evaluating externally created materials, colleagues providing peer review, learners evaluating their teachers’ work, and administrators interested in content at their institutions).
Our most interesting preliminary finding related to the comparison between the two institutions. KU and SUNY Upstate are situated in states which are demographically and politically very different; however, they have some commonalities. Both are public, urban medical schools in comparably sized cities and with similarly sized student bodies. At Upstate, where the checklist is originated, we have taken a “top-down” approach, with checklist use required by the Dean and integrated into the annual course review process. KU’s approach—spearheaded by Dr. Lucas, the Director of Diversity and Inclusion, and Dr. Mabachi, now the Director of Evaluation at the American Academy of Family Physicians—emphasized building buy-in among faculty from the ground up.
Overall, 27% of respondents who were prompted to consider changing content reported doing so or planning to do so. (It is important to note that the target isn’t 100%, as the prompt is might to encourage reflection and is not a guarantee that content is inappropriate.) After controlling for age, sex, and race, Upstate-affiliated users reported changing content only once for every three times a KU-affiliated user reported changing content (OR=0.35, p<0.01). To better understand this result, we included participation in training and identification as a “content creator” in the model. (KU users were more likely to have received training and to be content creators, given KU’s approach.) However, we found that these two variables only explained a fraction of the relationship (4%). This leads us to the conclusion that there are as-yet unexplored individual and institutional that both promote and prevent change—the focus of our next study.
Table 1. Characteristics of Analytic Sample.
| Institution | |||
| Total (n=373) | SUNY Upstate(n=181) | University of Kansas(n=192) | |
| Content | |||
| % changed any content | 26.81 | 14.38 | 34.38 |
| % Race | 46.81 | 37.50 (n=24) | 43.48 (n=23) |
| % Image | 29.03 | 23.26 (n=43) | 32.10 (n=81) |
| % Vignette | 40.00 | 50.00 (n=6) | 33.33 (n=9) |
| % Gender | 21.00 | 22.78 (n=79) | 20.00 (n=140) |
| % Sexual Orientation | 13.33 | 14.29 (n=15) | 12.5 (n=24) |
| % Disability | 19.05 | 20.00 (n=15) | 16.67 (n=6) |
| % Substance Use | 41.38 | 20.00 (n=10) | 52.63 (n=19) |
| % BMI | 13.13 | 14.29 (n=35) | 12.5 (n=64) |
| % Immigration | 10.81 | 5.88 (n=34) | 66.67 (n=3) |
| % Poverty | 27.27 | 37.50 (n=8) | 0.00 (n=3) |
| % Age | 33.33 | 37.50 (n=8) | 31.25 (n=16) |
| % Religion | 33.33 | 40.00 (n=5) | 0.00 (n=1) |
| % Prison | 30.00 | 33.33 (n=9) | 0 (n=1) |
| % Rural | 8.81 | 0.00 (n=7) | 9.14 (n=186) |
| % IPE | 6.43 | 7.41 (n=54) | 5.98 (n=117) |
Confounders
| %Female | 42.90 | 44.75 | 41.15 |
| Age | |||
| %18-24 | 2.14 | 3.87 | 0.52 |
| %25-34 | 12.33 | 23.76 | 15.63 |
| %35-44 | 23.59 | 19.89 | 27.08 |
| %45-54 | 18.50 | 17.13 | 19.79 |
| %55-64 | 27.61 | 22.65 | 32.29 |
| %65+ | 8.85 | 7.73 | 9.90 |
| %Prefer not to say | 6.97 | 4.97 | 8.85 |
Race
| %White | 78.02 | 75.69 | 80.21 |
| % Black | 2.95 | 2.21 | 3.65 |
| % Asian | 9.65 | 14.36 | 5.21 |
Professional Factors
| % Content Creator | 60.05 | 49.72 | 69.79 |
Checklist Factors
| % Trained | 66.49 | 43.65 | 88.02 |
Table 2. Odds Ratios Estimated from Logistic Regression Models Predicting Content Change from Institution, Including Confounders.
| Model 1 | Model 2 | |
| Institution (Upstate=1) | 0.44***(0.27, 0.71) | 0.35***(0.20, 0.61) |
| Age | ||
| 18-24 | – | – |
| 25-34 | 0.80 (0.16, 4.01) | |
| 35-44 | 0.51 (0.10, 2.56) | |
| 45-54 | 0.38 (0.07, 1.91) | |
| 55-64 | 0.45 (0.09, 2.23) | |
| 65+ | 0.24 (0.04, 1.45) | |
| Prefer not to say | 0.53 (0.09, 3.03) | |
| Sex(female=1) | 1.40(0.85, 2.32 | |
| Race (White=1) | 0.65(0.34, 1.22) | |
| Adjusted R2 | 0.03 | 0.05 |
Note: ***p<0.01
References:
- The Bias Checklist. Available online at: https://tinyurl.com/upstatebiaschecklist
- Caruso Brown AE. De-biasing medical education: A checklist methodology. Available online at: https://sdoheducation.org/community-hub/de-biasing-medical-education-a-checklist-methodology/
- Caruso Brown AE Hobart TR, Botash AS, Germain LJ. Can a checklist ameliorate implicit bias in medical education? Medical Education. 2019 Mar 11;53(5):510-.
-Hannah Connolly, MD/PhD/MPH student
-Amy Caruso Brown, MD, MSc, MSCS, HEC-C
Hannah Connolly is a MD/PhD/MPH student at SUNY Upstate Medical University Norton College of Medicine and completing a PhD in Social Science at Syracuse University. She is currently conducting an ethnographic study of medical student professional identity formation, paying particular attention to their relationship to social justice and the social/structural determinants of health.
Amy Caruso Brown, MD, MSc, MSCS, HEC-C is the Interim Chair of the Center for Bioethics and Humanities and an Associate Professor of Bioethics and Humanities and of Pediatrics at SUNY Upstate Medical University, where she also co-directs the health systems science curriculum for medical students. She is the recipient of a 2022 President’s Grant from the Josiah Macy, Jr. Foundation in support of her work on the Bias Checklist.