Simple screening questionnaires better for identifying social needs compared to advanced machine learning, study determines

Simple patient screening questionnaires addressing five different health-related social needs (HRSNs) such as food insecurity, housing instability, financial strain, transportation barriers and legal issues performed better than advanced machine learning methods at tracking HRSNs. This is according to a new study led by researchers at the Fairbanks School and Regenstrief Institute. However, authors emphasized that there was no perfect method; each one had gaps. 

The study examined data from 1,252 adult patients between ages 18 and 99 from two health systems in Indianapolis from January 2022 to June 2023 and used four different measurement approaches to track HRSNs. Those measurements were electronic health record (EHR) screening questionnaires, natural language processing (NLP) of clinical notes, rule-based algorithms and machine learning models (MLMs).

The EHR screening questions had the strongest overall performance of any measurement approach for food insecurity, housing instability, transportation barriers and legal problems, but all measures struggled with measuring financial strain. The NLP performed poorly for all five HRSNs and exhibited higher specificity than sensitivity for all HRSNs. The algorithm performed no better than coin-flips for food insecurity, housing instability, transportation barriers and legal problems. While ML classification for financial strain was poor, it outperformed all other methods.

Overall, screening questionnaires performed best but compared with reference standard measures, none of the measurement approaches performed well for every HRSN. Authors suggest that researchers who rely on a single method to collect HRSN data will likely underestimate patients’ true social needs burdens.

“Health systems increasingly recognize that social factors are as important to health as medical care, but it’s not always clear how to identify patients with those needs in routine practice,” explained Joshua Vest, PhD, professor of health policy and management at the Fairbanks School, a research scientist at the Regenstrief Institute and lead author of the study. “Our study shows that while questionnaires are a strong starting point, we cannot rely on any single method to capture the full range of patients’ social needs.”

Social determinants of health — conditions in the environments where people are born, live, learn, work, play, worship and age that affect a wide range of health, functioning and quality-of-life outcomes and risks — increasingly have been linked to medical and healthcare well-being and outcomes among older adults.