"Students of the game": Using big data to assess internal medicine resident experiences

Researchers at the Temerty Faculty of Medicine have found that where and when internal medicine residents trained could lead to significant differences in the kinds of patients they saw.

The study, published in JAMA Network Open, showed that between residents, there was substantial variation in case volume, demographics, breadth, acuity, medical complexity and social determinants.

According to the study, some residents saw over three times as many cases of heart failure as their peers training at other hospitals. Other key diagnoses included in the study were pneumonia, urinary tract infections, chronic obstructive heart disease and neurocognitive disorders. As well, some residents saw almost four times as many admissions that required critical care transfer than other residents.

“You can’t know what you [haven’t] seen,” says co-lead author Brandon Tang, an assistant professor of internal medicine.  “If we see these differences empirically, we can think about how to ensure we prepare learners for the full range of what they’ll see in practice.”

Tang, who is also a general internist at Unity Health Toronto’s St. Michael’s Hospital, and co-lead author and internal medicine resident Andrew Lam tapped into GEMINI-MedED, an internal medicine database established by professors Fahad Razak, Shiphra Ginsburg, Brian Wong and Amol Verma that combines clinical and educational data from five Temerty Medicine-affiliated hospitals.

The researchers zeroed in on a data set that included nearly 800 senior medical resident physicians across five academic hospitals and more than 143,000 overnight admissions from July 2010 through December 2019.

The study measured a range of factors including the number of patients admitted per shift, the ages and sex of patients admitted, variety of diagnoses at discharge, risk of in-hospital death, proportion of admissions that required admission to the intensive care unit, how frail the patients were, and how many people were readmitted to hospital within a month.

Lam and Tang also looked at social determinants of health, including disability, neighbourhood income and visible minority status. 

The authors note that hospital-level differences, such as differing admission guidelines and local patient populations, likely contributed to case variation between sites.

In addition to differences between training sites, the study also showed that over the ten-year study period, residents managed increased patient volume and complexity despite working a similar number of shifts per year.

Residents who began their training toward the end of the study period in 2019 encountered higher patient volumes and more complex cases than those who began their residency earlier in the decade.

The researchers note that this phenomenon likely reflects population-level health trends driven by an ageing population, advancements in prescription drugs, and growing social challenges such as homelessness, substance use and psychiatric illness.

Lam and Tang were in part inspired by the world of professional sports, where athletes, known as ‘students of the game,’ study previous matches to improve their performance and teams embrace analytics to improve their overall performance. They wondered if medical training could benefit from a similar analytics-driven approach.  

“We wanted to create similar statistics for medical education to better understand the range of cases internal medicine residents encounter during their training,” says Lam, who is also the chief medical resident at Toronto Western Hospital. “By measuring things like how many complex cases trainees see during residency, we can use data to better prepare learners for the future.’

Lam and Tang say their findings show that electronic health records can be used to ensure residents are exposed to the breadth of cases required to prepare them for independent practice. Further, the pair say the data can help identify gaps in training and guide training requirement standards.