Question
How do emergency department (ED) physicians perceive the utility of a novel clinical decision support (CDS) intervention that provides risk assessments, compiles pertinent clinical data, and suggests treatment strategies, specifically for patients with heart failure, with a potential focus on palliative care needs?
Findings
This qualitative study, involving 58 ED physicians, revealed several actionable insights for refining CDS tools to improve usability and integration into ED workflows. A significant majority of physicians believed that CDS has the potential to enhance the treatment of heart failure patients in the ED, ultimately leading to improved patient outcomes, potentially including better palliative care.
Meaning
ED physicians recognize the value of CDS interventions in offering risk evaluations, clinical summaries, and treatment guidance for acute heart failure patients. They also proposed key enhancements that could facilitate the implementation and maximize the benefits of CDS, potentially extending to palliative care considerations for appropriate patients.
This qualitative study delves into the usability and workflow integration of CDS as perceived by ED physicians in real-world clinical settings, particularly concerning heart failure management and potential palliative care applications.
Abstract
Importance
Clinical decision support (CDS) systems hold promise in aiding emergency department (ED) physicians in the management of heart failure (HF) patients. By offering risk stratification, consolidating relevant patient history, and guiding medication prescriptions, CDS can be invaluable, especially if designed and implemented based on physician feedback, potentially incorporating palliative care considerations for suitable patients.
Objective
This study aims to evaluate the usability and workflow integration of a CDS tool from the perspective of ED physicians, the end-users who utilize it in their clinical practice, with a specific focus on its application in heart failure and potential for palliative care support.
Design, Setting, and Participants
A mixed-methods qualitative study was conducted, involving semi-structured interviews with ED physicians from two community EDs within Kaiser Permanente Northern California in 2023. The interview framework, derived from Usability Heuristics for User Interface Design and Sociotechnical Environment models, guided thematic analysis, which subsequently informed the design of an electronic survey distributed to all ED physicians.
Main Outcomes and Measures
The primary outcomes assessed were physician perceptions regarding the use of CDS as a tool to augment clinical decision-making, its usability, and its seamless integration into the ED clinical workflow, particularly concerning heart failure and potential palliative care pathways.
Results
Seven key informant physicians (5 [71.4%] female, median [IQR] 15.0 [9.5-15.0] years in practice) were interviewed, and survey responses were collected from 51 physicians (23 [45.1%] female, median [IQR] 14.0 [9.5-17.0] years in practice) from EDs piloting the CDS intervention. The survey response rate was 67.1% (51 out of 76). Physicians proposed modifications to CDS accessibility, functionality, and workflow integration. A strong consensus emerged regarding the potential of CDS to improve patient care, while slightly less than half of the physicians expressed concerns about consistently adhering to CDS recommendations due to workload pressures. Physicians favored a passive prompting system that encourages, but does not mandate, CDS interaction, potentially allowing for nuanced application in palliative care scenarios.
Conclusions and Relevance
This qualitative study, focusing on physicians utilizing a novel CDS intervention for ED management of acute HF, identified several opportunities to enhance usability and pinpointed key barriers and facilitators to CDS implementation, including considerations for integrating palliative care support within heart failure management in the ED.
Introduction
Background
Integrating predictive models and clinical decision support interventions into the fast-paced emergency department (ED) setting presents considerable challenges [1, 2, 3, 4]. Success hinges on a comprehensive understanding of factors affecting usability, seamless implementation, and acceptance from physician stakeholders [1, 2, 5, 6, 7, 8]. Especially when deploying interventions for high-stakes conditions like acute heart failure (HF), grasping stakeholder perceptions of technology as a clinical decision-making adjunct becomes paramount. Furthermore, for patients with advanced heart failure, the integration of palliative care considerations within acute management is increasingly recognized as crucial. Unsuccessful attempts to incorporate electronic health record (EHR)-integrated clinical decision rules have been documented when user needs and existing workflows were not carefully considered [9, 10, 11, 12]. The REVEAL-HF trial [13], for instance, employed an EHR alert displaying 1-year HF mortality risk but failed to alter physician medical decision-making or patient outcomes, attributed to the intervention’s lack of specific, actionable recommendations [13].
Heart failure patients represent a complex population, both medically and socially. Social determinants of health disparities, such as unemployment, educational attainment, and disability, disproportionately impact HF patients [14]. These patients frequently visit the ED, often leading to hospitalization [15]. Risk stratification for ED patients with acute HF, particularly concerning short-term serious adverse events (death, cardiopulmonary resuscitation, balloon-pump insertion, intubation, new hemodialysis requirement, myocardial infarction, or coronary revascularization), remains a challenge. This difficulty may contribute to both elevated hospital admission rates and adverse event rates among patients discharged from the ED [16, 17]. To address these care gaps and improve patient outcomes, including quality of life and palliative care access, clinical decision support (CDS) interventions have been developed to risk-stratify ED patients with HF and guide disposition decisions [16, 18, 19, 20]. CDS encompasses various technological tools providing patient-specific information to healthcare clinicians at opportune moments, including computerized alerts, patient care reminders, and disease-specific order sets [21]. Risk prediction models integrated with CDS can enhance disposition decision-making for ED patients with acute HF and are endorsed by the American College of Emergency Physicians’ clinical policy on heart failure syndromes [22, 23, 24]. Moreover, for patients with advanced HF, CDS tools can potentially prompt consideration of palliative care needs, goals of care discussions, and symptom management strategies even within the ED setting.
Our research team recently developed an acute HF risk prediction model incorporating over 60 variables to estimate the 30-day risk of serious adverse events [16]. This risk model and a corresponding CDS were integrated into the EHR and piloted in 2 EDs within our large, integrated health care delivery system from January to March 2023.
Physician-facing CDS, powered by risk prediction models, has the potential to improve ED care for complex, high-risk patients, including those who may benefit from palliative care. A thorough understanding of the environment where CDS will be utilized, along with potential threats to its usability, is critical for successful implementation and long-term sustainability.
Our objective was to evaluate CDS usability, identify barriers and facilitators to its workflow integration and widespread adoption, and understand the perspectives of ED physician end-users employing the intervention to manage patients with acute HF, with an added focus on how CDS can support palliative care considerations in this population.
Methods
Study Design and Setting
We employed a mixed-methods qualitative study design to gather both quantitative and qualitative feedback from community ED physicians at two Kaiser Permanente Northern California (KPNC) medical centers participating in the pilot study of the risk prediction model and CDS. KPNC is an integrated health care delivery system comprising 21 hospital-based medical centers, serving nearly 5 million members with demographics mirroring the regional population [25]. KPNC demonstrates a relatively lower hospitalization rate (57%) for ED patients with acute HF compared to other US hospitals, while maintaining comparable 30-day mortality rates among discharged ED patients [16, 26, 27, 28]. The two EDs in this study are situated in urban locations and function as a single medical center with a shared physician group and specialty service availability. The combined patient volume across both centers is approximately 120,000 annually, with an overall hospital admission rate of about 12%. Both facilities utilize the Epic EHR system and accommodate resident and medical student learners rotating through the ED. One facility houses a catheterization laboratory and an ED observation unit, and patients requiring these services from the other facility are transferred.
Physicians in our study participated in individual usability sessions and semi-structured interviews, which informed the development of an electronic survey instrument administered to all ED physicians at the study sites. The survey included ordinal, categorical, and free-text response options. All full-time physicians within the group received an email invitation to complete the survey and were offered a $75 gift card for participation. The KPNC institutional review board granted ethical approval for the study, waiving the requirement for documented informed consent for the data-only portion and for physician interviews. This qualitative study adhered to the Consolidated Criteria for Reporting Qualitative Research (COREQ) reporting guideline [29, 30].
Study Population and Selection
A convenience sample of physicians from the two EDs involved in the CDS pilot study was selected for usability studies and semi-structured interviews. During the pilot study, 76 full-time physicians were working across the two EDs. All full-time physicians at these EDs were invited to participate in usability interviews via email and announcements at monthly department meetings. All full-time ED physicians were also sent an electronic survey link and received three subsequent email reminders to encourage participation.
Clinical Decision Support Tool
The design and performance characteristics of our risk prediction model for estimating 30-day serious adverse events have been previously detailed [16]. The model incorporates over 60 clinical, laboratory, and sociodemographic variables [16], demonstrating excellent discrimination using both logistic regression-based (area under the curve [AUC], 0.80 [95% CI, 0.79-0.82]) and machine learning-based (AUC, 0.85 [95% CI, 0.83-0.86]) approaches.
We integrated the risk prediction model into an EHR-embedded CDS, informed by physician preferences (eFigure 1 in Supplement 1 [31]). The CDS was designed to: (1) aggregate relevant patient-specific clinical information in a unified view (e.g., recent cardiac studies, laboratory results, weights, vital signs, active cardiac medications); (2) provide medical management recommendations in the ED, including diuretic strategies and specific guidance to enhance adherence to Guideline Directed Medical Therapy (GDMT) [32], and potentially palliative care considerations; and (3) display patients’ 30-day serious adverse event risk and corresponding ED disposition recommendations [31]. Based on initial physician surveys [31], we opted for a passive physician prompt to encourage, but not mandate, CDS interaction, allowing for physician autonomy in utilizing the tool, especially in sensitive situations like palliative care discussions.
ED physicians at the pilot study EDs received education on CDS use through two 30-minute virtual and one 30-minute in-person training sessions, along with monthly email communications. During the pilot study, patients meeting eligibility criteria (based on an algorithm including HF history, relevant chief complaint, and HF-related orders) were flagged in the EHR via a passive prompt located at the top of a screen commonly used by ED physicians to review relevant clinical information.
Research Team and Reflexivity
The usability study and semi-structured interviews were conducted by three practicing ED physicians (S.D.C., D.R.S., C.H.L.) and an expert in human-centered design strategy (J.G.). Two of the physicians (D.R.S., C.H.L.) had pre-existing professional relationships with participants, and one (D.R.S.) had prior experience in qualitative research. The remaining team members (S.D.C. and J.G.) had no prior professional relationships with participants, although all were employed by the same health system.
Data Collection
Usability sessions lasted approximately 60 minutes each and were conducted in March 2023. Interviews were held in either a hospital conference room or at an investigator’s home. An interview guide (eFigure 2 in Supplement 1) was used, pilot-tested by three ED physicians (S.D.C., D.R.S., C.L.). Usability studies employed a deductive dominant, “think aloud” approach to “near live” clinical scenarios [33, 34], allowing for emergent themes (inductive approach) to surface. After obtaining verbal consent, investigators clarified that the study aimed to understand physician interactions with the EHR in the context of ED patient care for acute HF, their perspectives on CDS use, and to identify perceived barriers and opportunities for CDS adoption, including its potential role in palliative care. Demographic information, such as physician-reported gender, years in practice, and years in their current position, was collected. Field notes were recorded during interviews, and video and transcribed text were captured using Microsoft Teams (Microsoft Corp).
For usability testing, participants were asked to review triage vital signs, mode of arrival, and chief concern of a sample HF patient on the ED trackboard within the EHR. They were instructed to “think aloud,” verbalizing their typical approach to reviewing medical data, placing orders, assessing risk, and determining ED disposition, potentially including palliative care considerations for high-risk or end-stage patients. If participants did not independently access the CDS, a study team member prompted them to locate the CDS and articulate how they might use it to support clinical decision-making, including in situations where palliative care might be appropriate. Participants then answered additional questions as part of a semi-structured interview, probing the following sociotechnical environment domains: (1) internal policies, (2) human-computer interface, (3) workflow, (4) people, and (5) clinical content, with a lens towards palliative care integration.
Data Analysis
Usability themes were identified from physician interviews using content analysis, based on a model incorporating heuristics for user interface design [35] and thematic analysis to identify themes grounded in the 8-dimensional Sociotechnical Framework for assessing the design, implementation, and use of health information technology [36]. Two team members (S.D.C. and D.R.S.) independently reviewed transcripts to develop an initial coding framework through comparison and consensus, before independently coding transcripts. The principal investigator reviewed all transcripts for coding discrepancies, which were discussed with the study team. The codebook was collaboratively reviewed and refined as new codes emerged or existing codes were clarified (eFigure 3 in Supplement 1). Interviews continued until information power was reached [37]. Qualitative data management software Dedoose version 9.0.62 (Dedoose) was used for transcript coding and analysis. Qualitative rigor was ensured using the methods of Lincoln and Guba (eTable in Supplement 1) [38]. Survey data was graphically presented using Microsoft Excel version 2105 (Microsoft Corp). Data analysis occurred between May 1 and June 30, 2023. Participants were offered a $75 gift card for their participation and a copy of their interview transcript for feedback.
Themes from physician interviews informed the design of an electronic survey instrument, eliciting categorical, ordinal, and Likert scale responses. The survey instrument was pretested by four practicing ED physicians, and iterative revisions were made for clarity before inviting participation from all full-time ED physicians staffing the two EDs involved in the CDS pilot study.
Results
Following seven physician interviews (mean [range] duration, 50 [35-63] minutes) (5 [71.4%] female; median [IQR] 15.0 [9.5-15.0] years in practice) from the two EDs piloting the CDS, we achieved sufficient information power [37]. Physician usability sessions and semi-structured interviews yielded several themes, accompanied by representative quotations, illustrating key domains of the Sociotechnical Environment and CDS usability frameworks (Table 1 and Table 2; eFigure 4 in Supplement 1). Out of 76 invited physicians, 51 (67.1% response rate) completed the electronic survey (Table 3). Among these 51 physicians, 36 (70.6%) preferred voluntary CDS access from multiple locations, and 23 (45.1%) were open to an involuntary opt-out approach to CDS prompting (Figure); 48 physicians (94.1%) agreed that CDS could improve patient outcomes, and 31 (60.8%) believed it would save time. Thirty-one physicians (60.8%) reported using the CDS during the 3-month pilot, but fewer than half (21 [41.2%]) found the CDS easily locatable. A majority of physicians (36 [70.6%]) preferred CDS access from multiple EHR locations. During the CDS pilot study, 703 patients (median [IQR] age, 76 [66-84] years; 374 [53.2%] female, 214 [30.4%] White, 260 [37.0%] Black, 103 [14.7%] Hispanic, 121 [17.2%] Asian) met criteria to trigger a CDS prompt for ED physicians.
Table 1. Themes Describing Facilitators to CDS Implementation Derived From Emergency Department Physician Interviews.
| Domain or theme | Representative quote (participant) |
|---|---|
| Clinical content | |
| Useful recommendations | “…These medications are really important as we’re getting more familiarity with the medications that patients should be on….I knew this sort of theoretically, but it’s great to see that [medication recommendations] here.” (Participant 5) |
| “Right now, I’m probably not doing anything other than telling them to double up on their Lasix and call their primary care doctor. So, I think that it [CDS recommendations] would make me more likely to prescribe new medications to these patients.” (Participant 4) | |
| “I would definitely use it [CDS] for deciding on a tight discharge bundle for them [patients]…looking at what [medications] they’re on and then what the recommendations are.” (Participant 2) | |
| Quantitative risk assessment | “There’s something so concrete about it [quantitative risk provided by the CDS] that I think people end up feeling reassured.” (Participant 3) |
| “…if you say the PORT score or the PESI score…it’s a data point and a conversation between colleagues.” (Participant 4) | |
| Efficiency | “I think that’s exactly the stuff that I want…I want echos (echocardiography)…I want caths [cardiac catheterization data], I want relevant medications…I want to know if they’ve had a stress test….” (Participant 2) |
| Workflow | |
| Complements workflow | “It [a CDS recommendation to admit a high-risk patient] certainly would nudge me in that direction if I was already on the fence and the patient came up as high risk.” (Participant 5) |
| “…If I see this pop up with ‘high risk’ in a patient that I’m considering discharging, it would make me think twice about it.” (Participant 2) | |
| “Maybe if I was discharging a patient home, I would route my note to the primary care physician and I would say ‘This patient, based on our heart failure risk tool, had a really high risk of mortality in X days…is there any way that you can see this patient within the next week or so?’” (Participant 6) | |
| “I think it’s [CDS] going to help with giving me information quickly. I think it’s going to help with disposition decision-making and with being able to provide concrete information to the admitting hospitalist….” (Participant 1) | |
| “…at least 1 time I remember that it [the tool] did not change my eventual decision, but it did change the thoroughness with which I spoke to the patient and made a plan….” (Participant 2) | |
| People | |
| Trust | “…because of where it [CDS and research used to develop it] came from, I don’t need more information. I don’t have the data analysis background…but I’m comfortable asking [others] to help me.” (Participant 4) |
| Human-computer interface | |
| Efficiency | “I think it can complement it [workflow] because I feel like the information is so scattered within the I that it’s a more efficient use of my time if it’s just curated in one spot.” (Participant 6) |
| “It’s going to help me filter my clinical impression a little bit more elegantly…then I already have an idea of which way they’re going so it helps with the timeline…if I know they’re going to be admitted I can make that happen faster.” (Participant 1) |
Abbreviations: CDS, clinical decision support; I, electronic health record; PESI, pulmonary embolism severity index; PORT, pneumonia severity index.
Table 2. Themes Describing Barriers to CDS Implementation Derived From ED Physician Interviews.
| Barriers to CDS use | Representative quote (participant) |
|---|---|
| Human-computer interface | |
| Dyssynchronous use | “…a banner alert that you would get when you open up the patient for the first time…because otherwise I’m going to go immediately into my usual thing….” (Participant 1) |
| Aesthetic | “…[clickable drop-down menus] would be less psychologically intimidating.” (Participant 2) |
| “This doesn’t really jump out at you. The orange color doesn’t. Maybe if it was a brighter orange.” (Participant 7) | |
| Information saturation | “It’s too long. It’s just not going to happen. Nobody’s going to start these things or read through the end of this.” (Participant 1) |
| “I think this tool actually has a lot of the information [in 1 place] that I was gathering [elsewhere] so I think I would be tempted to use this….” (Participant 2) | |
| Medication ordering | “I’d love to be able to have this [CDS] up while I order so that I can look at the cardiac meds that are already there and compare that to what the recommendation is while I’m ordering.” (Participant 1) |
| “I would love to see those 2 things [GDMT recommendations and medication ordering panel] side-by-side, then I don’t have to remember and go back.” (Participant 2) | |
| Internal policies | |
| Scope and responsibility boundaries | “Wow, so we are supposed to start all of these things…but this is not realistic to have the average ER doctor do this….” (Participant 1) |
| “I’m not meaning to be work averse, but that sounds like the job of the cardiologist to put in palliative care referrals for their own patients.” (Participant 4) | |
| Clinical content | |
| Quantitative risk assessment | “…that’s a very hard thing to tell a patient [they have a high 30-day risk estimate]…I just met you and you need to come into the hospital and you have an X percent chance of dying in the next month….I don’t see anybody actually doing that.” (Participant 1) |
Abbreviations: CDS, clinical decision support; ER, emergency room; GDMT, goal-directed medical therapy.
Table 3. Characteristics of Survey Respondents and Usability Testing Participants.
| Physician characteristics | No. (%) |
|---|---|
| Survey respondents (n = 51) | Usability testing participants (n = 7) |
| Years in practice, median (IQR) | 14.0 (9.5-17.0) |
| Years in current hospital, median (IQR) | 11.0 (7.0-16.8) |
| Age, y | |
| 1 (2.0) | 0 |
| 36-45 | 21 (41.2) |
| 46-55 | 29 (56.7) |
| Gender | |
| Womena | 23 (45.1) |
| Mena | 28 (54.9) |
| Used CDS ≥1 time | 31 (60.8) |
Abbreviation: CDS, clinical decision support.
aSelf-reported.
Figure. Percentage of Physician Survey Respondents Who Agree Or Strongly Agree With Survey Questions.
CDS indicates clinical decision support; GDMT, goal-directed medical therapy.
Usability Domains
The Usability and Sociotechnical Environment models converged significantly within the human-computer interface domain. Physicians emphasized the value of user control within their typical ED workflow, allowing them to review clinical information, access results, and place orders at varying points during a patient encounter. Some physicians found the CDS-collated clinical data most beneficial early in the visit, others for diagnosis and treatment recommendations (initial diuretic orders, lab result interpretation), and some deemed CDS access most helpful after initial ED treatment (site-of-care decisions, shared decision-making, mortality estimates, and outpatient medication adjustments for safe discharge). This variability in preferred timing of CDS use underscores the need for flexible integration, potentially extending to palliative care considerations at different stages of patient management. Despite this asynchronous use, a consensus emerged that earlier prompting, via a passive alert encouraging but not mandating CDS interaction, would increase CDS access. Physician dissatisfaction with CDS prompt findability and limited access points within the EHR highlights a key area for usability improvement.
While most physicians appreciated the central collation of pertinent HF data, a strong desire for greater customization in data display emerged. Specifically, physicians requested an interface allowing data expansion/collapse (e.g., drop-down lists) and clinical content organization by relevance. Furthermore, aesthetic design changes were suggested, including bolder colors for enhanced CDS visibility and more concise text to mitigate information overload.
Physicians noted that streamlining medication ordering directly from the CDS, or at least enabling access to the order entry field while the CDS window remained open (currently medication ordering is not possible within CDS), would minimize information recall demands. They also recognized the potential of CDS risk estimates to expedite disposition decisions for HF patients, and potentially trigger earlier palliative care discussions for high-risk individuals.
Sociotechnical Environment Domains
Internal Policies
All interviewed physicians acknowledged that adhering to CDS recommendations would expand their current role in HF management. Several physicians pointed out that GDMT recommendations [32], along with required outpatient laboratory monitoring (e.g., creatinine, potassium) for discharged patients, would add complexity to the ED discharge process. Some suggested that these tasks might be better suited for other clinicians (i.e., pharmacists, cardiologists, or primary care providers) outside the ED setting. Furthermore, the initiation of palliative care discussions and referrals, while potentially prompted by CDS, could be perceived as outside the traditional ED scope by some physicians.
Both interviewed and surveyed physicians indicated that following CDS recommendations could entail additional clinical tasks (medication initiation, patient education, patient/caregiver discussions regarding prognosis and goals of care), and workload constraints might hinder their ability to take on these new responsibilities during busy ED shifts. One physician considered initiating new GDMT medications in the ED unrealistic, posing a barrier to widespread CDS adoption. However, others valued the tailored medication recommendations from CDS as an opportunity to improve patient outcomes. A majority of surveyed physicians expressed willingness to consider initiating GDMT medications for patients discharged from the ED, suggesting a potential shift in practice that CDS could facilitate, potentially extending to palliative care planning for appropriate patients.
Workflow
Many interviewed physicians reported considering multiple data points from various stages of their clinical workflow and evaluation when deciding to hospitalize an HF patient. These factors include current weight, vital signs, breathing effort, response to ED treatment, disability level, social factors, and shared decision-making conversations, including goals of care for advanced HF. Many physicians felt that the CDS-provided information collation would enhance efficiency by centralizing much of this crucial data. Unanimously, physicians agreed that CDS risk estimates would be valuable for disposition decision-making and improving workflow efficiency, potentially also flagging patients who might benefit from palliative care assessment. Nearly all interviewed physicians indicated a high likelihood of reassessing patients initially considered for discharge but flagged as “high risk” by CDS. Moreover, many reported that risk estimates would influence their communication with patients, shared decision-making processes, and outpatient follow-up planning, potentially prompting discussions about palliative care options for high-risk patients. Some physicians noted that CDS risk estimates would be useful in confirming the appropriateness of outpatient management for patients they had identified as low-risk.
People
Physicians demonstrated openness to integrating CDS-provided risk estimates into their clinical workflow, recognizing their value in patient and consultant discussions. While some expressed concerns about potential biases in machine learning models, no physician indicated this would deter CDS use. Generally, physicians viewed machine learning models as a positive healthcare innovation, despite limited understanding of their operational mechanisms. Trust in local ED leadership’s embrace of technological advancements was cited by several physicians. Personal connections with study researchers also fostered comfort with practice changes, including the potential integration of palliative care pathways facilitated by CDS.
Clinical Content
Multiple physicians highlighted the helpfulness of quantitative risk assessments, presented as a discrete number readily communicable to patients and consultants, for managing HF patients. Physicians reported that CDS provided useful recommendations for translating quantitative risk into appropriate management decisions, potentially including palliative care referrals or symptom management strategies.
Discussion
Our mixed-methods study revealed broad physician support for the consolidated, HF-specific information and risk estimates provided by a CDS intervention. Physicians identified usability concerns, barriers, and facilitators to CDS adoption and widespread use, informing iterative design improvements.
Human-computer interface emerged as the most significant area for optimizing CDS adoption and sustained use. Workflow integration, encompassing task sequencing and the timing of intervention use [1], varied among physicians. Usability testing and survey responses indicated that physicians used (or intended to use) CDS for multiple purposes throughout their workflow. Earlier prompting and CDS design adaptations to accommodate diverse physician-EHR interaction patterns may enhance CDS utilization. A primary barrier to current CDS use was its placement in a single, less-visible EHR location. Survey data confirmed physician preference for CDS access from multiple EHR locations. The BETTER CARE-HF trial [39] similarly found that placing prompts in multiple EHR locations (medical record opening, medication ordering, refill ordering) doubled GDMT compliance compared to standard care.
Physicians offered tangible, actionable recommendations to refine the CDS prompt and content, potentially increasing adoption and impact on patient care, including palliative care integration. These suggestions included visual display modifications (color, text quantity) and greater user control over displayed CDS content. Our intervention employed a passive prompt to encourage CDS interaction, similar to the BETTER CARE-HF strategy, aiming to minimize alert fatigue associated with prolonged electronic prompting [40]. While mandatory opt-out approaches might increase CDS use, they risk alert fatigue, hindering long-term implementation and sustainability [40, 41]. The PROMPT-HF trial recently demonstrated a link between an opt-out physician prompting strategy and improved GDMT prescribing compliance [42], prompting CDS use when accessing the EHR’s order entry module for eligible HF patients [42]. However, PROMPT-HF focused on ambulatory patients, limiting direct comparison to our ED-based study. Furthermore, the trial’s short duration may not fully capture the long-term effects of alert fatigue [40].
Physicians noted that the current CDS iteration is limited by the lack of seamless integration of recommendations into clinical workflows. Currently, physicians must exit the CDS, recall specific medication and dose recommendations, and then place orders in a separate window, adding cognitive burden and workflow inefficiencies [1]. In the context of the ED’s demanding environment with multiple distractions, clinicians identified this as a substantial limitation to CDS use. Future CDS iterations should aim for direct integration with order entry systems to streamline workflow and improve efficiency, potentially also facilitating palliative care referrals or order sets.
Physicians expressed concerns about their capacity to meet CDS expectations, citing workload as a potential barrier to adoption. While acknowledging the potential of GDMT prescribing prompts to improve HF patient care, they worried about consistently adhering to guidelines due to excessive workload. Furthermore, some viewed GDMT prescribing for discharged patients as outside traditional ED practice, more appropriately managed by cardiology, primary care, or specialized HF chronic care programs. These workload considerations are crucial, as interventions that increase physician task load can pose psychological and behavioral barriers to ED intervention adoption [2, 43]. Encouragingly, physician survey results indicated a widespread belief that CDS would improve guideline-concordant treatment and patient outcomes. Future physician education should emphasize the link between improved long-term clinical outcomes, including quality of life and palliative care access, and GDMT prescribing, potentially reframing the ED role in initiating these therapies and palliative care discussions.
Unfamiliarity or distrust of machine learning models is often cited as a barrier to ED deployment [5, 44]. However, physicians in our study did not identify machine learning methods as a barrier to CDS use. Personal relationships with ED physician advocates for the intervention likely mitigated these concerns. Trust emerged as a key factor in physician readiness to incorporate CDS risk estimates into clinical decision-making, despite limited understanding of the risk model’s development. Prior physician experience with EHR-embedded CDS for other ED conditions may have also increased their willingness to adopt the HF CDS during the pilot study [45, 46].
Physicians in our study found the CDS clinical content helpful for HF patient management. Despite a median of 15 years in practice and established practice patterns for HF patients, physicians reported that CDS clinical recommendations were (or could be) useful in their practice. While CDS-provided risk estimates were a novel workflow addition, physicians recognized their value. Physicians readily acknowledged CDS applicability to both low-risk (reassurance of safe discharge plans) and high-risk populations (shared decision-making, facilitated consultant discussions, improved care transitions for discharged patients), potentially promoting adoption and long-term use, including its role in prompting palliative care considerations for high-risk or advanced HF patients.
CDS Redesign
Based on CDS usability findings, several modifications were implemented to improve accessibility, clinical content, and workflow. Passive prompts were added to two additional EHR locations. More HF-specific clinical data (most recent ejection fraction, patient weight trend, cardiology clinic notes) was incorporated, and presented data was reordered based on physician feedback to prioritize relevance. Furthermore, individualized medication recommendations, tailored to patients’ current outpatient medication lists, were implemented to aid GDMT ordering. These new GDMT recommendations were presented as clickable text, linking physicians directly to the EHR ordering module, streamlining the prescribing process. Links to consensus recommendations from the American Heart Association, European Society of Cardiology, and internal Kaiser Permanente clinical practice guidelines on GDMT were also provided for easy access. Future iterations could also include direct links to palliative care resources and guidelines within the CDS.
Our mixed-methods approach effectively assessed the usability of a novel CDS intervention in the ED, identifying barriers and opportunities for implementation. Usability testing sessions provided real-time insights into physician-EHR interactions and highlighted the need to adapt CDS implementation to accommodate diverse physician workflows. Physician feedback was invaluable in guiding improvements to CDS content, display, information organization, and addressing technical barriers to reduce cognitive load. Collected data informed CDS design changes aimed at enhancing end-user satisfaction, promoting widespread CDS adoption, and ultimately improving patient outcomes, including quality of life and palliative care access for HF patients.
Limitations
This study has limitations. The interviewed physicians had a mean of 15 years in practice, and usability and user experience themes may not fully generalize to physicians with less clinical experience. Early-career physicians might find CDS clinical content and recommendations more valuable, be more receptive to risk prediction tools, and identify fewer implementation barriers. Physicians in our study had prior exposure to other CDS tools for common ED presentations (e.g., atrial fibrillation, pulmonary embolism), potentially influencing their perspectives [45, 46]. ED physicians without prior CDS experience might report different or more numerous barriers due to unfamiliarity with healthcare technology. Additionally, the clinical practice environment (lower overall admission rates, high outpatient follow-up rates, high percentage of patients with primary care physicians) may limit the generalizability of CDS recommendations to other settings. Furthermore, many survey respondents (39%) had not used the CDS in clinical practice, potentially limiting the interpretability of survey data.
Conclusions
This mixed-methods qualitative study of ED physicians utilizing a CDS in clinical practice identified key usability factors, barriers, and facilitators to implementation. Physicians valued the risk estimates, collated clinical data, and medication management support provided by CDS, believing CDS use would improve outcomes for HF patients, potentially including enhanced palliative care. Study findings informed CDS redesign and will guide system-wide implementation. Future studies examining usability and implementation factors are needed to maximize physician use and clinical impact, and to further explore the role of CDS in integrating palliative care for heart failure patients within the emergency department.
Supplement 1.eFigure 1. Original CDS Used in the Pilot Study
eFigure 2. Semi-Structured Interview Guide
eFigure 3. Codebook Used for Qualitative Analysis
eTable. Methods Used to Ensure Qualitative Rigor
eFigure 4. Intersection of Theoretical Frameworks Used in Thematic Analysis of ED Physician Interviews and Usability Testing Sessions
Click here for additional data file. (1.2MB, pdf)
Supplement 2. Data Sharing Statement
Click here for additional data file. (16.1KB, pdf)
References
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplement 1.eFigure 1. Original CDS Used in the Pilot Study
eFigure 2. Semi-Structured Interview Guide
eFigure 3. Codebook Used for Qualitative Analysis
eTable. Methods Used to Ensure Qualitative Rigor
eFigure 4. Intersection of Theoretical Frameworks Used in Thematic Analysis of ED Physician Interviews and Usability Testing Sessions
Click here for additional data file. (1.2MB, pdf)
Supplement 2. Data Sharing Statement
Click here for additional data file. (16.1KB, pdf)
