How Medical School Applicant Race, Ethnicity, and Socioeconomic

Downloaded from http://journals.lww.com/academicmedicine by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3i3D0OdRyi7TvSFl4Cf3VC1y0abggQZXdgGj2MwlZLeI= on 12/13/2020

Academic Medicine, Vol. 90, No. 12 / December 2015

1667

Research Report

The Multiple Mini-Interview (MMI)

is replacing traditional interviews at

medical schools.

1–5

In the MMI, trained

raters evaluate applicants in a series of

brief, timed, structured stations, and

station ratings are pooled to yield a

summary score. Stations are designed to

assess skills that are difﬁcult to ascertain

from medical school applications, such as

interpersonal communication, teamwork,

ability to handle stress, problem solving,

and integrity/ethics. The MMI is well

accepted by applicants, reasonably

reliable, and predictive of medical school

and subsequent performance.

1–3

Little studied is how underrepresented

racial/ethnic minority (URM) and lower

socioeconomic status (SES) applicants

may be affected by adoption of the MMI.

This is a key issue given that U.S. medical

schools admit disproportionately few

URM and lower SES individuals.

6–8

URM

admissions declined at many schools

in the wake of legislation restricting

consideration of race and ethnicity in

acceptance decisions,

9,10

a trend that may

accelerate following a Supreme Court

decision upholding the constitutionality

of such restrictions.

Concurrently,

medical education and health policy

groups call for a more diverse physician

workforce.

12–16

In such a climate, it is

necessary to evaluate how evolving

admissions trends like MMI adoption

may inﬂuence the diversity of medical

school classes.

A long-recognized problem with

traditional nonstructured interviews

is vulnerability to interviewer biases

triggered by various applicant

characteristics.

17–22

Implicit (i.e.,

unconscious) biases disfavoring racial/

ethnic minority and lower SES persons

are common in U.S. society,

including

among physicians.

The effects of

bias during interviews can be reduced

by increasing structure (removing

ambiguity and, therefore, the tendency

to rely on stereotype-driven judgments)

and pooling evaluations from multiple

raters (potentially diluting or offsetting

individual biases).

20,25–27

In being

structured and incorporating multiple

raters’ perspectives, the MMI may

thus be less susceptible to implicit bias

effects than traditional medical school

interviews.

Only three studies to our knowledge

have explored the associations of medical

school applicants’ racial/ethnic minority

status or SES with MMI performance.

In one study, involving six Canadian

schools, aboriginal status was negatively

correlated with MMI scores, whereas

family income level was not signiﬁcantly

associated with MMI performance.

The analyses included few aboriginal

participants (< 3%) and did not explore

other race/ethnicity categories. Further,

a robust indicator of SES would consider

factors beyond income, such as parental

Abstract

Purpose

To examine associations of medical

school applicant underrepresented

minority (URM) status and socioeconomic

status (SES) with Multiple Mini-Interview

(MMI) invitation and performance and

acceptance recommendation.

Method

The authors conducted a correlational

study of applicants submitting secondary

applications to the University of

California, Davis, School of Medicine,

2011–2013. URM applicants were black,

Southeast Asian, Native American,

Paciﬁc Islander, and/or Hispanic. SES

from eight application variables was

modeled (0–1 score, higher score =

lower SES). Regression analyses

examined associations of URM status

and SES with MMI invitation (yes/no),

MMI score (mean of 10 station ratings,

range 0–3), and admission committee

recommendation (accept versus not),

adjusting for age, sex, and academic

performance.

Results

Of 7,964 secondary-application

applicants, 19.7% were URM and 15.1%

self-designated disadvantaged; 1,420

(17.8%) participated in the MMI and

were evaluated for acceptance. URM

status was not associated with MMI

invitation (OR 1.14; 95% CI 0.98 to

1.33), MMI score (0.00-point difference,

CI −0.08 to 0.08), or acceptance

recommendation (OR 1.08; CI 0.69 to

1.68). Lower SES applicants were more

likely to be invited to an MMI (OR 5.95;

CI 4.76 to 7.44) and recommended for

acceptance (OR 3.28; CI 1.79 to 6.00),

but had lower MMI scores (−0.12 points,

CI −0.23 to −0.01).

Conclusions

MMI-based admissions did not disfavor

URM applicants. Lower SES applicants

had lower MMI scores but were

more likely to be invited to an MMI

and recommended for acceptance.

Multischool collaborations should

examine how MMI-based admissions

affect URM and lower SES applicants.

Acad Med. 2015;90:1667–1674.

First published online May 20, 2015

doi: 10.1097/ACM.0000000000000766

Please see the end of this article for information

about the authors.

Correspondence should be addressed to Anthony

Jerant, Department of Family and Community

Medicine, University of California, Davis, School of

Medicine, 4860 Y Street, Suite 2300, Sacramento,

CA 95817; telephone: (916) 734-7081; e-mail:

[email protected].

How Medical School Applicant Race, Ethnicity,

and Socioeconomic Status Relate to Multiple

Mini-Interview–Based Admissions Outcomes:

Findings From One Medical School

Anthony Jerant, MD, Tonya Fancher, MD, MPH, Joshua J. Fenton, MD, MPH,

Kevin Fiscella, MD, MPH, Francis Sousa, MD, Peter Franks, MD, and Mark Henderson, MD

Research Report

Academic Medicine, Vol. 90, No. 12 / December 2015

1668

education.

A single-school U.S. study

using a limited dichotomous (yes/no)

single-item self-report indicator of

disadvantaged status found no evidence

of an association between disadvantaged

status and MMI performance.

Similarly,

a single-school United Kingdom study,

using a geographic area-based measure

of deprivation, found no association

between applicants’ deprivation score

and MMI performance.

However,

ecological measures such as geographic

area deprivation scores have signiﬁcant

limitations.

To our knowledge, no studies have

examined whether applicants’ race/

ethnicity inﬂuences acceptance following

MMI participation, or whether race/

ethnicity or SES inﬂuences the likelihood

of being invited to an MMI. Such

outcomes are likely to be more strongly

inﬂuenced by parochial concerns (e.g.,

institutional mission focus) than the

MMI process,

12,33

which is relatively

similar across schools.

34,35

Nonetheless, it

is important to consider MMI invitation

and acceptance decisions to provide

context for evaluating the MMI-based

admissions process.

We examined the associations of

applicants’ URM status and SES with

MMI invitation, MMI performance, and

post-MMI acceptance recommendation

among applicants to the University

of California, Davis (UCD), School

of Medicine (SOM) in Sacramento,

California, over three admission cycles

(2011–2013), adjusting for other

applicant demographic characteristics and

postsecondary academic performance.

Method

We employed data collected as part of

the routine admissions processes during

the 2011, 2012, and 2013 application

cycles. The admissions ofﬁce provided

relevant application data in an electronic

spreadsheet with personal identiﬁers

removed. The study was conducted

from April 18, 2014, through the end

of August 2014. The UCD institutional

review board reviewed the protocol and

determined it was exempt.

Application, screening, and MMI

invitation and scheduling

Applicants initially applied to the

UCD SOM via the American Medical

College Application Service (AMCAS).

Following initial screening based on

cumulative grade point average (GPA)

and Medical College Admission Test

(MCAT) scores, admissions committee

members reviewed all applications and

invited a subset of applicants to submit a

secondary application. Faculty evaluated

secondary applications for invitation

to an MMI based on cumulative

GPA and MCAT scores, personal

statements, extracurricular activities,

recommendation letters, and other

characteristics that could contribute to

fulﬁlling the educational and service

missions of the school. Invited applicants

self-scheduled their MMI sessions via an

online portal.

MMI process and scoring

The MMI consisted of 10 individual

10-minute stations. At each station,

applicants had 2 minutes to read a brief

set of instructions, and 8 minutes to

address the assigned tasks on entering

the room. Nine stations assessed skills

in the following domains: integrity/

ethics, professionalism, interpersonal

communication, diversity/cultural

awareness, teamwork, ability to handle

stress, and problem solving. An additional

station asked applicants to explain their

choice to pursue a career in medicine.

Most stations were adapted from content

developed at McMaster University and

marketed by ProFitHR.

A single trained rater, blinded to

participants’ AMCAS application

information, attended each station. In

some stations, raters interacted directly

with applicants. At others, raters observed

applicant interactions with actors or

other applicants. There were 216 different

raters during the study period; the mean

number of MMI stations that each

evaluated was 104 (standard deviation

[SD] 61.9; range 8–276). Women made

up 61% of raters. Rater professional

backgrounds were as follows: physicians,

31%; medical students, 15%; other

clinicians (e.g., nurses), 11%; basic

science faculty, 6%; patients, 2%; and

various nonclinician leaders (e.g., deans),

professionals (e.g., lawyers), and high-

level administrative staff (e.g., curriculum

manager), 35%. The range of rater

backgrounds reﬂected the conviction

that diverse perspectives are helpful in

selecting future physicians who will be

able to work effectively with people from

all walks of life. Mandatory rater training

included a one-hour course reviewing the

admissions process, rater roles and duties,

and the need to avoid pursuing protected

class issues (e.g., race/ethnicity, gender).

At each station, raters scored overall

applicant performance using an

anchored four-point scale: 0, < 25th

percentile performance (relative to other

applicants); 1, 25th–50th percentile;

2, 51st–75th percentile; or 3, > 75th

percentile. Raters were instructed

to consider both the applicant’s

communication abilities and the content

(e.g., comprehensiveness) of their

statements in assigning ratings. The

total MMI score was the mean of each

applicant’s individual station scores.

Scale internal consistency (Cronbach

alpha = 0.67) was comparable to that

observed in other MMI studies.

2,18,37–41

Acceptance recommendation

The admissions committee met weekly

during the admission cycle to review each

participant’s MMI performance, AMCAS

application, and secondary application.

Subsequently, the committee made one

of the following recommendations:

reject, low waitlist, high waitlist, or offer

acceptance. For the current analyses, we

dichotomized the recommendation (offer

acceptance versus not).

URM status

We determined URM status (URM

[black, Southeast Asian, Native

American, or Paciﬁc Islander race and/

or Hispanic ethnicity] versus not [all

other responses]) from self-reported

race/ethnicity information in the

AMCAS application. These groups

remain underrepresented in the medical

profession relative to the general

population.

Socioeconomic disadvantage

We developed a composite measure of

SES using self-reported information

in the AMCAS application, screening

candidate indicators for inclusion using

logistic regression analyses and based

on their contribution to predicting

applicants’ self-designated disadvantaged

status. The following predictors (yes/

no items except where indicated) were

signiﬁcant and maximized the area under

the receiver operating characteristic curve

(0.95): fee assistance received for medical

school application (yes/no); childhood

spent in an underserved area; family

recipients of family assistance program;

Research Report

Academic Medicine, Vol. 90, No. 12 / December 2015

1669

Table 1

Characteristics of 7,964 Applicants to the University of California, Davis, School of Medicine

Submitting Secondary Applications, by MMI Participation Status, From a Study of Applicant

Race and Ethnicity, Socioeconomic Status, and MMI-Based Admissions Outcomes, 2011–2013

Characteristic

Not invited to an

MMI, n = 6,389

Participated

in an MMI, n = 1,420

All screened applicants,

N = 7,964

Demographics

Age category (years), no. (%)

< 22 1,113 (17.4) 213 (15.0) 1,363 (17.1)

22 1,664 (26.0) 306 (21.5) 2,011 (25.3)

23 1,388 (21.7) 284 (20.0) 1,699 (21.3)

> 24 2,224 (34.8) 617 (43.5) 2,891 (36.3)

Female, no. (%) 2,884 (45.1) 715 (50.4) 3,678 (46.2)

Underrepresented minority, no. (%)

1,172 (18.3) 368 (25.9) 1,570 (19.7)

Socioeconomic factors

Self-designated disadvantaged, no. (%) 742 (11.6) 425 (29.9) 1,202 (15.1)

Fee assistance, no. (%) 627 (9.8) 274 (19.3) 927 (11.6)

Underserved childhood, no. (%) 662 (10.4) 329 (23.2) 1,019 (12.8)

Family on assistance, no. (%) 981 (15.4) 403 (28.4) 1,416 (17.8)

Any need-based scholarship, no. (%) 1,406 (22.0) 493 (34.7) 1,941 (24.4)

Highest parental education level, no. (%)

Less than high school 61 (1.0) 39 (2.7) 105 (1.3)

High school graduate 259 (4.1) 117 (8.2) 386 (4.8)

Some college 440 (6.9) 115 (8.1) 563 (7.1)

College graduate 5,629 (88.1) 1,149 (80.9) 6,910 (86.8)

Family income category, no. (%)

< $25,000 303 (4.7) 163 (11.5) 480 (6.0)

$25,000 to < $50,000 531 (8.3) 190 (13.4) 735 (9.2)

$50,000 to < $75,000 530 (8.3) 132 (9.3) 670 (8.4)

≥ $75,000 5,025 (78.7) 935 (65.8) 6,079 (76.3)

Contributed to family income, no. (%) 436 (6.8) 220 (15.5) 656 (8.4)

% Family contributed to college costs, mean (SD) 29.1 (39.1) 22.7 (35.1) 27.8 (38.4)

SES score, mean (SD)

0.12 (0.23) 0.25 (0.34) 0.14 (0.26)

Academic factors

Cumulative GPA category, no. (%)

< 3.4 1,070 (16.7) 201 (14.2) 1,283 (16.1)

3.4 to 3.6 1,827 (28.6) 293 (20.6) 2,146 (26.9)

> 3.6 to 3.8 2,060 (32.2) 404 (28.5) 2,510 (31.5)

> 3.8 1,432 (22.4) 522 (36.8) 2,025 (25.4)

Total MCAT score category, no. (%)

19 to 26 277 (4.3) 136 (9.6) 416 (5.2)

27 to 30 1,818 (28.5) 305 (21.5) 2,145 (26.9)

31 to 32 1,517 (23.7) 262 (18.5) 1,797 (22.6)

33 to 34 1,370 (21.4) 246 (17.3) 1,644 (20.6)

35 to 44 1,407 (22.0) 471 (33.2) 1,962 (24.6)

Abbreviations: MMI indicates Multiple Mini-Interview; SD, standard deviation; SES, socioeconomic status; GPA, grade point

average; MCAT, Medical College Admission Test.

For all of the characteristics listed, P < .001 for the difference between those not invited to an MMI and those who

participated in an MMI (chi-square test for categorical variables and t test for continuous variables).

Of the 15,844 people who applied to the medical school during the three-year study period, 8,933 (56.4%) were invited

to submit secondary applications. Of these invitees, 7,964 (89.2%) submitted secondary applications, all of which were

individually screened by the admissions committee to determine MMI invitations.

Applicants indicating black, Southeast Asian, Native American, or Paciﬁc Islander race and/or Hispanic ethnicity.

Scaled 0–1.0, higher score meaning lower SES. Derived using logistic regression predicting self-designated disadvantaged status

based on responses to the following application items: fee assistance received for application (yes/no); underserved childhood

(yes/no); family on assistance (yes/no); contributed to family income as child (yes/no); total family income (in dollars); parents’

highest level of education (< high school, high school graduate, some college, college graduate); received ﬁnancial-need-based

scholarship(s) in college (yes/no); and percentage of college costs contributed by family. See Method for details.

Research Report

Academic Medicine, Vol. 90, No. 12 / December 2015

1670

income level category of applicant’s

family (< $25,000; $25,000 to < $50,000;

$50,000 to < $75,000; or > $75,000);

applicant contributed to family income;

any ﬁnancial-need-based scholarship(s)

in paying for postsecondary education;

percentage of postsecondary education

costs contributed by the family; and

parents’ highest level of educational

attainment (< high school, high school

graduate, some college, or college

graduate). The model yielded a predicted

probability of being self-designated

disadvantaged, ranging continuously

from 0 to 1.0 (higher predicted

probability = lower SES). We employed

the score, which correlated 0.91 with a

factor-analytic-derived score, as the study

measure of applicant SES. The SES score

was preferable to alternatives

because

the continuous scale acknowledges

that socioeconomic disadvantage is

not a binary characteristic and reduces

misclassiﬁcation (false positives and

negatives).

Other characteristics

The admissions ofﬁce also provided

information from AMCAS regarding

applicant age, sex, cumulative

postsecondary GPA, and total MCAT score.

Data analysis

We analyzed the data using Stata version

13.1 (Stata Corporation Inc., College

Station, Texas). We modeled MMI

invitation (yes/no) and medical school

acceptance recommendation (accept

versus not) using logistic regression (a

separate regression for each dependent

variable). We modeled total MMI score

using linear regression. All models

included the following characteristics:

age category (< 22 [reference], 22, 23,

or > 24 years); female gender (yes/no);

URM status (versus not); cumulative

GPA category (< 3.4, 3.4–3.6, > 3.6–3.8,

or > 3.8 [reference]); total MCAT

score category (19–26, 27–30, 31–32,

33–34, or > 34 [reference]); SES (0–1.0

continuous score), and application year

(2011, 2012, or 2013). The acceptance

recommendation model additionally

included the total MMI score (0–3).

Results

During the three study application

cycles, 15,844 people applied, and

8,933 (56.4%) were invited to submit

secondary applications. Of the invitees,

7,964 (89.2%) submitted secondary

applications, and 1,575 (19.8%) were

invited to an MMI. Of the MMI invitees,

1,420 (90.2%) attended an MMI.

Applicant characteristics

Table 1 summarizes the characteristics

of screened applicants. Compared with

those not invited to an MMI, those who

participated in an MMI were older and

more likely to be female, from an URM

group, disadvantaged (based on both self-

designation and the continuous SES score),

and had higher GPAs and MCAT scores.

MMI invitation

Although adjusted URM status was not

associated with MMI invitation, lower

SES was associated with receiving an

MMI invitation, as were older age, female

gender, and higher GPA and MCAT score

(Table 2).

MMI score

Mean MMI score was 1.30 (SD 0.62). In

adjusted analyses, URM status was not

associated with MMI score, but lower SES

was associated with lower MMI scores

(Table 3). Older age, female gender, and

lower GPA were also associated with

MMI score (Table 3).

Acceptance recommendation

Of the 1,420 MMI participants, 334

(23.5%) were recommended for acceptance.

Lower SES was associated with being

recommended for acceptance, whereas

URM status was not (Table 4). Of other

factors examined, mean MMI score was

the most strongly associated with being

recommended for acceptance. Older age,

female gender, and higher GPA and MCAT

scores were also associated with being

recommended for acceptance (Table 4).

Discussion

In a diverse sample of applicants over

three admission cycles at UCD, superior

MMI performance was strongly linked

Table 2

Results of a Logistic Regression Model Examining Associations of Characteristics

of 7,964 Applicants to the University of California, Davis, School of Medicine

Submitting a Secondary Application With MMI Invitation, From a Study of

Applicant Race and Ethnicity, Socioeconomic Status, and MMI-Based Admissions

Outcomes, 2011–2013

Variable

Adjusted OR

(95% CI)

P value

Age category (reference < 22)

22 1.10 (0.91, 1.33) .30

23 1.33 (1.09, 1.62) < .01

24+ 1.87 (1.56, 2.24) < .001

Female

1.36 (1.21, 1.53) < .001

Underrepresented minority

1.14 (0.98, 1.33) .08

SES score

5.95 (4.76, 7.44) < .001

GPA (reference > 3.8)

< 3.4 0.32 (0.26, 0.40) < .001

3.4–3.6 0.38 (0.32, 0.45) < .001

> 3.6–3.8 0.52 (0.45, 0.60) < .001

MCAT (reference ≥ 35)

< 27 0.69 (0.53, 0.90) < .01

27–30 0.37 (0.31, 0.44) < .001

31–32 0.43 (0.36, 0.51) < .001

33–34 0.50 (0.43, 0.60) < .001

Abbreviations: MMI indicates Multiple Mini-Interview; OR, odds ratio; CI, conﬁdence interval; SES,

socioeconomic status; GPA, grade point average; MCAT, Medical College Admission Test.

The logistic regression model also included admission year (2011, 2012, or 2013).

Applicants indicating black, Southeast Asian, Native American, or Paciﬁc Islander race and/or Hispanic ethnicity.

Scaled 0–1.0, higher score meaning lower SES. Derived using logistic regression predicting self-designated

disadvantaged status based on responses to the following application items: fee assistance received for

application (yes/no); underserved childhood (yes/no); family on assistance (yes/no); contributed to family income

as child (yes/no); total family income (in dollars); parents’ highest level of education (< high school, high school

graduate, some college, college graduate); received ﬁnancial-need-based scholarship(s) in college (yes/no); and

percentage of college costs contributed by family. See Method for details.

Research Report

Academic Medicine, Vol. 90, No. 12 / December 2015

1671

with being recommended for acceptance.

Further, URM applicants were no less

likely than non-URM applicants to

receive an MMI invitation, performed

similarly on the MMI, and were just as

likely to be recommended for acceptance.

In the only prior study to exploring

this issue, MMI performance at six

Canadian medical schools was worse

for the < 3% of applicants with self-

reported aboriginal status, whereas

income was not signiﬁcantly associated

with MMI scores.

Our ﬁndings provide

some reassurance that adoption of the

MMI-based admissions process at U.S.

medical schools need not adversely affect

admission prospects for URM applicants.

The similar MMI scores for URM

and non-URM participants support

the notion that structured interview

processes that incorporate the

perspectives of multiple evaluators like

the MMI may be less vulnerable to the

effects of individual evaluator implicit

biases.

20,25–27

Although we did not measure

rater implicit biases regarding racial/

ethnic minorities, such biases have been

documented to be pervasive in U.S.

society, including among physicians and

other professionals,

23,24

and can affect the

outcomes of employment interviews in

various ﬁelds including medicine.

17,19–22

Thus, it is likely that implicit biases were

present among our raters; however, they

did not exert a signiﬁcant net inﬂuence,

given that mean MMI scores did not

differ between URM and non-URM

applicants. Because lack of URMs in

medicine is a widely acknowledged

problem,

6,7,13,33,42–44

it is possible that

biases against URM applicants were

offset by ratings biased in favor of URM

applicants, made by raters seeking to

address limited racial/ethnic diversity in

the physician workforce.

Our ﬁndings regarding MMI

performance may have the broadest

applicability, given the relatively high

standardization of the MMI across

institutions.

34,35

By contrast, MMI

invitation and acceptance decisions

are shaped more by parochial concerns

such as institutional mission and local

workforce needs.

12,33

In this context, our

ﬁnding that lower SES applicants had

worse adjusted MMI performance may

be cause for concern. Although three

prior studies reported no association of

applicant SES with MMI performance,

all relied on less robust measures of SES.

Nonetheless, the decrement in MMI

performance with decreasing SES in

our study was small: The MMI score

(scale of 0–3 points) declined by a mean

of 0.12 points across the 0–1 range of

the SES score. Further, the lower MMI

scores among lower SES applicants

were more than offset by their greater

likelihood of being invited to an MMI

and recommended for acceptance. These

ﬁndings may reﬂect the ongoing shift

from a purely metric-based applicant

review process toward the more holistic

process advocated by the Association of

American Medical Colleges.

12,15

Although the reasons for the lower MMI

performance among lower SES applicants

are unclear, poorer postsecondary

academic preparation and performance

are unlikely explanations because we

adjusted for GPA and MCAT score.

Lower SES applicants may have fewer

life experiences bolstering skills assessed

by the MMI. Similar reasoning has

been suggested to explain the lower

MCAT scores among such applicants.

Although less afﬂuent applicants are

more likely to report paid employment

during postsecondary education, their

ﬁnancial circumstances may require

taking jobs that do not require MMI-type

preemployment screening. Lack of prior

experience with MMI-type screening

may be a disadvantage in the medical

school MMI because prior experience

with a particular interview format is

associated with better future performance

with that format.

Lower-level jobs

also may not facilitate the higher-level

communication, critical thinking, and

problem-solving skills the MMI assesses,

and the time required for such jobs may

limit participation in pursuits that build

such skills (e.g., scholarly presentations,

volunteer clinic work).

Rater implicit bias could also help to

explain the lower MMI scores among lower

SES applicants. Raters were not provided

any information about participants,

and only one MMI station afforded

Table 3

Results of a Linear Regression Model Examining Associations of Characteristics

of 1,420 Applicants to the University of California, Davis, School of Medicine

Completing an MMI With Total MMI Score, From a Study of Applicant Race and

Ethnicity, Socioeconomic Status, and MMI-Based Admissions Outcomes, 2011–2013

Variable

Adjusted PE

(95% CI)

P value

Age category (reference < 22)

22 0.15 (0.05, 0.25) < .01

23 0.14 (0.04, 0.25) < .01

24+ 0.28 (0.18, 0.38) < .001

Female

0.14 (0.08, 0.20) < .001

Underrepresented minority

0.00 (−0.08, 0.08) .97

SES score

−0.12 (−0.23, −0.01) .03

GPA (reference > 3.8)

< 3.4 0.07 (−0.05, 0.18) .25

3.4–3.6 0.15 (0.05, 0.24) < .01

> 3.6–3.8 0.06 (−0.02, 0.14) .12

MCAT (reference ≥ 35)

< 27 0.00 (−0.13, 0.14) .96

27–30 0.03 (−0.07, 0.13) .56

31–32 0.03 (−0.06, 0.12) .54

33–34 0.05 (−0.04, 0.14) .31

Abbreviations: MMI indicates Multiple Mini-Interview; PE, parameter estimate; CI, conﬁdence interval; SES,

socioeconomic status; GPA, grade point average; MCAT, Medical College Admission Test.

The linear regression model also included admission year (2011, 2012, or 2013).

Applicants indicating black, Southeast Asian, Native American, or Paciﬁc Islander race and/or Hispanic ethnicity.

Scaled 0–1.0, higher score meaning lower SES. Derived using logistic regression predicting self-designated

disadvantaged status based on responses to the following application items: fee assistance received for

application (yes/no); underserved childhood (yes/no); family on assistance (yes/no); contributed to family income

as child (yes/no); total family income (in dollars); parents’ highest level of education (< high school, high school

graduate, some college, college graduate); received ﬁnancial-need-based scholarship(s) in college (yes/no); and

percentage of college costs contributed by family. See Method for details.

Research Report

Academic Medicine, Vol. 90, No. 12 / December 2015

1672

applicants the opportunity to describe

their backgrounds. Nonetheless, subtle

information apparent to raters could have

led to implicitly biased ratings, possibly

triggered by the generally relatively

large social distance between lower SES

applicants and the typical rater.

47,48

Most

of our MMI raters were well educated and

relatively afﬂuent. Prior work indicates that

applicant factors such as use of language

unfamiliar to the typical rater could trigger

a biased low rating.

20,21

Applicants’ verbal

skills have been shown to determine

immediate interviewer impressions and,

in turn, ﬁnal appraisals.

The issue of

SES-based physician workforce disparities

has received less attention than race/

ethnicity-based disparities.

Thus, it is less

likely that raters consciously biased their

evaluations in favor of lower SES applicants

to address SES-based physician workforce

disparities. If our ﬁndings are replicated,

it would suggest the need to consider rater

training to minimize the inﬂuence of

SES-based biases.

We found that older applicants and

women performed better than their

younger and male counterparts both in the

MMI (consistent with prior studies

28,37

)

and throughout the admissions process.

Older applicants are more likely to have

had life experiences requiring effective

communication. Women, more than men,

have been noted to communicate in ways

that quickly build rapport in novel social

situations such as the MMI. Previously

underrepresented in medicine, women

now constitute well over half of all U.S.

medical students.

Given the increasing

adoption of the MMI and strong inﬂuence

of MMI performance on medical school

acceptance, our ﬁndings suggest the

potential for a disparity to develop

disfavoring male applicants, and warrant

monitoring.

Limitations

Our study was correlational, limiting

causal inferences. Our data were derived

from applicants at a single school. How

the ﬁndings generalize to other schools

is unclear. We lacked sociodemographic

information regarding MMI raters

and admissions committee personnel,

precluding examination of how such

characteristics may have inﬂuenced the

study outcomes. We employed a novel

composite measure of SES, albeit one

with theoretical advantages over other

indicators.

Because schools vary widely

in institutional mission,

applicant

pools, admissions personnel, and other

attributes, multi-institution studies

are needed to better gauge the impact

of MMI-based admissions processes

on URM and socioeconomically

disadvantaged applicants. Studies ideally

should account for the characteristics of

admissions personnel and applicants, and

examine the relative utility of different

SES indicators. It is also unknown

whether URM or lower SES applicants

would have fared differently in one-

on-one interview-based admissions

processes. Ideally, randomized trials

would compare traditional interview

processes with MMI-based processes,

examining their impact on the prospects

of URM and lower SES applicants.

Conclusion

In conclusion, in analyses of data from

one California medical school, an

MMI-based admissions process did not

disfavor racial/ethnic minority groups

underrepresented in the physician

workforce. Applicants from lower SES

backgrounds, also underrepresented in

medicine, had lower MMI scores but

were more likely to receive an MMI

invitation and be recommended for

acceptance. Multischool collaborations

are needed to further evaluate the

impact of MMI-based medical school

admissions processes on URM and lower

SES applicants.

Funding/Support: None reported.

Other disclosures: None reported.

Ethical approval: On April 18, 2014, the University

of California Davis (UCD) institutional review

board (IRB) reviewed the study protocol (UCD

IRB reference number 599704-1) and concluded

that it is not human subjects research, based on

the use of completely anonymous data that were

collected for a purpose other than the research,

and is therefore exempted from IRB approval and

oversight.

Table 4

Results of a Logistic Regression Model Examining Associations of Characteristics

of 1,420 Applicants to the University of California, Davis, School of Medicine

Completing an MMI With Admissions Committee Recommendation to Accept, From

a Study of Applicant Race and Ethnicity, Socioeconomic Status, and MMI-Based

Admissions Outcomes, 2011–2013

Variable

Adjusted OR

(95% CI)

P value

Age category (reference < 22)

22 1.14 (0.62, 2.11) .66

23 2.13 (1.15, 3.95) .02

24+ 1.78 (1.01, 3.15) .05

Female

1.42 (1.02, 1.99) .04

Underrepresented minority

1.08 (0.69, 1.68) .74

SES score

3.28 (1.79, 6.00) < .001

GPA (reference > 3.8)

< 3.4 0.14 (0.07, 0.29) < .001

3.4–3.6 0.51 (0.32, 0.83) < .01

> 3.6–3.8 0.84 (0.56, 1.26) .39

MCAT (reference > 35)

< 27 0.25 (0.12, 0.54) < .001

27–30 0.28 (0.16, 0.48) < .001

31–32 0.69 (0.43, 1.11) .12

33–34 0.56 (0.34, 0.90) .02

MMI score

35.24 (23.14, 53.67) < .001