Healthcare's earthquake: Lessons from COVID-19

New metrics and forecasting models are key to understanding and anticipating the challenges of the COVID-19 pandemic

BOSTON – The COVID-19 pandemic has fundamentally disrupted U.S. healthcare organizations. Hospitals have faced drug and device shortages and created new ICUs overnight. Care plans have evolved out of necessity, and hospitals' carefully constructed patient flow systems were up-ended.

In an article published today in NEJM Catalyst Innovations in Care Delivery, leaders and clinician researchers from Beth Israel Lahey Health (BILH) propose using complexity science — a field concerned with understanding dynamic, unpredictable systems, such as the human brain, economies or climates — to identify strategies that healthcare organizations can use to respond better to the ongoing pandemic and to anticipate future challenges to healthcare delivery.

"COVID-19 has been a painful reminder that healthcare — both as an industry and as a series of complex organizations — has evolved slowly over time, as have the metrics and models we use to assess quality, safety and accommodate future needs," said lead author Jennifer Stevens, MD, MS, Director of the Center for Healthcare Delivery Science at Beth Israel Deaconess Medical Center (BIDMC), part of Beth Israel Lahey Health. "The principles of complexity science offer three strategies healthcare leaders can employ to manage operations during the COVID-19 pandemic: engaging diverse perspectives in leadership teams, staying open to new metrics, and creating forecasting tools that reflect complex healthcare systems."

Engage diverse thinkers in a dashboard design

Noting that individuals are often unable to see the "big picture" from within a complex system like healthcare, Stevens and colleagues recommend bringing additional, perhaps unexpected voices to leadership teams. For example, while epidemiologists naturally guided healthcare leadership teams in the context of COVID-19, Stevens and colleagues suggest including patient and/or community representatives, physicians from disciplines that may be more tangentially related to the current crisis, or clinical and operations staff from the communities disproportionately affected by the pandemic to crisis response teams. "Voices from these various perspectives can expand the vantage point, allowing healthcare leaders see more of the complex system and implement strategies that anticipate future needs," said Stevens.

Identify metrics that triangulate different sides of a complex healthcare system

Identifying a broader range of relevant metrics can also expand leaderships' view of the complex system. For example, patient demographics were a largely unreported variable early on the in pandemic — before physicians had a clear picture of how tightly race, ethnicity and other socioeconomic factors were linked to the risk of contracting and dying of COVID-19.

"Having a better understanding about the disparate impact of COVID-19 on communities of color could have helped healthcare leaders better anticipate the flow of patients coming into clinics, well as the implications for clinical staff, and personal protective equipment needs," said Stevens. "Healthcare leaders must be open to new metrics and watchful for undervalued variables, or we may find ourselves so focused on one set of metrics that we miss the significance of more meaningful data."

As an example of a metric with shifting value, Stevens and colleagues described how the significance of the number of patients with COVID-19 in BIDMC's ICU changed as spring turned into summer in 2020. In the spring, patients with COVID-19 presented to the hospital with severe respiratory symptoms and required immediate critical care services. The rising number of patients admitted with COVID-19 meant the hospital would need additional ICU beds and ventilators, personal protective equipment and other healthcare resources.

However, once the state of Massachusetts and the hospital deemed it safe to reinstate urgent and elective procedures as summer began, all patients were tested for COVID-19 upon admission. The universal testing yielded infected but asymptomatic patients who had come to the hospital for reasons unrelated to COVID-19.

"Suddenly, we were admitting asymptomatic patients with COVID-19 who wouldn't require the same resources and care as the patients critically ill with the novel coronavirus," said Stevens. "So the meaning behind the number of COVID-19 patients at BIDMC really shifted over the early months of the pandemic."

Create forecasting tools that reflect healthcare's complex systems

The team also suggests that forecasting tools must truly reflect the complex realties of the COVID-19 pandemic. To build a model that does that, the team used machine learning to pull relevant data from each of the 13 hospitals and three business units that make up the Beth Israel Lahey Health system, rather than depending on published reports from China or Italy. Next, they added publicly available local cellphone data to the model, revealing how much people were moving around and interacting with other people. Taken together, these data sources — reflecting both shifting local public health policies as well as the shifting new social norms of behavior as the pandemic wore on — contributed to a model capable of providing timely and locally relevant predictions.

"Our model leverages the principles of complexity to guide hospital leadership, providing weekly updates to a group of healthcare leaders about how and when a new surge of infections may arrive," Stevens said. "Models need to reflect the shifting health and policy landscape – to allow for the complexity of the pandemic itself – for any healthcare organization to meaningfully make use of them."

"Healthcare is facing one of its greatest challenges, in part because our comfortably familiar metrics and dashboards, which were designed to handle the problems of a complex system, couldn't see the 'big one' coming," said Kevin Tabb, MD, president and CEO of Beth Israel Lahey Health. "Adapting to new realities that COVID-19 brought to the fore requires that healthcare leaders build new models that reflect the true complexity we are facing, engage new voices, and remain flexible and curious about our metrics. We are still squarely in the middle of this earthquake, and we have many aftershocks ahead."

Additional authors include Ashley O'Donoghue, PhD, and Steven Horng, MD, MMSc, of BIDMC, and Manu Tandon, MBA, MPA, of BILH.

Stevens is supported by grant number K08HS024288 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not represent official AHRQ views. Horng discloses support from Philips Healthcare. The other authors report no conflicts of interest or disclosures relevant to this work.