INTRODUCTION
Patient engagement is increasingly recognized as a critical component in revolutionizing and improving healthcare outcomes. Considered a “blockbuster prescription,” it’s a priority for leading healthcare research institutions like the Patient-Centered Outcomes Research Institute (PCORI) and the Agency for Healthcare Research and Quality (AHRQ). Furthermore, the National Committee for Quality Assurance mandates patient engagement for patient-centered medical home (PCMH) accreditation, highlighting its essential role in modern healthcare models. Patient engagement spans various dimensions of healthcare, from direct care interactions to organizational design and policy development. Within organizational design, patient co-leadership in initiatives, particularly quality improvement projects, represents the highest level of engagement.
Research consistently demonstrates the positive impact of patient engagement on quality improvement. A systematic review highlighted that patient involvement leads to enhanced patient information resources and improved access to care. Moreover, AHRQ emphasizes that engaging patients in improvement efforts results in better quality, safety, and satisfaction for both patients and staff.
Primary care practices, especially PCMHs, are actively striving to engage patients across the spectrum of care, including research and practice enhancement. While initial findings suggest a positive perception of patient involvement in medical home transformation, genuine patient engagement in quality improvement remains limited. Surveys indicate that a minority of PCMHs actively involve patients in these crucial efforts, despite acknowledging the value of patient input in other areas. A significant barrier to wider patient engagement is the lack of clear, actionable methods for practices to implement.
Traditional methods for gathering patient input, such as suggestion boxes, surveys, and patient advisory boards, are common. Interviews and focus groups are also utilized, particularly in research settings, to organize patient involvement in practice improvements. However, these methods are often time-intensive, labor-heavy, and require specialized qualitative data analysis skills. Furthermore, the outputs are frequently dense textual documents that are difficult to translate into easily understandable visual representations of complex ideas and patterns. Given the undeniable importance of patient engagement, there is a pressing need for formalized, efficient methods to engage patients and measure the impact of their contributions. This article introduces group concept mapping, a structured conceptualization method, as a promising tool to effectively engage patients in primary care practice improvement. We will illustrate its practical application using a quality improvement project conducted in a large family medicine practice.
CONTEXT AND RESOURCES of Concept Mapping
Concept mapping, in its broader sense, is not a novel idea. It encompasses a range of knowledge representation theories and applications within educational and cognitive psychology. It visually represents concepts within a subject area and their interrelationships in a concept or knowledge map. The spatial arrangement of concepts indicates their relatedness, and connecting lines or arrows clarify the nature of these relationships.
The specific group concept mapping method we are focusing on was pioneered in the 1980s by William Trochim. Trochim expanded upon existing concept mapping theories by developing it into a group process. In this approach, participants collaboratively define a conceptual space by responding to a specific prompt and then determine the relatedness of ideas by sorting responses into clusters based on perceived similarity. This process culminates in a visual concept map reflecting the collective understanding of the topic. Participants collectively brainstorm, organize knowledge, and then interpret and apply the resulting maps. An early publication detailing the method’s development and statistical underpinnings is publicly available. Many group concept mapping projects utilize software specifically designed by Trochim to facilitate the process (software details in Supplemental Appendix, accessible at http://annfammed.org/content/14/4/370/suppl/DC1).
In the context of patient and consumer engagement, Trochim’s concept mapping has been predominantly applied in public health community-based participatory research. It has also seen limited use in medical studies exploring patient and caregiver experiences in chronic disease management. However, a review identified only a handful of studies utilizing concept mapping to engage patients in practice improvement, with just one conducted in primary care. Notably, none of these prior studies offered a detailed methodological framework for applying concept mapping specifically for primary care practice improvement.
METHODOLOGIC BASIS of Concept Mapping
Group concept mapping uniquely integrates qualitative techniques like brainstorming and thematic sorting with quantitative methods such as multidimensional scaling and cluster analysis. Multidimensional scaling is a statistical technique that uses aggregated similarity ratings from the sorting task to create a visual representation – a point map. In this map, each point represents an idea from the brainstorming session, and the distances between points indicate their relatedness. Cluster analysis then groups these points into clusters, minimizing distances within clusters to visually represent groupings of related ideas. Trochim and Linton provide statistical details and SAS programming language for these analyses. Supplemental Figures 1 and 2 (available at http://www.annfammed.org/content/14/4/370/suppl/DC1) illustrate the point and cluster maps from our quality improvement study. While these techniques are statistically robust, direct research on group concept mapping methodology is relatively recent. Rosas and Kane’s review of 69 group concept mapping studies concluded that the reproducibility and reliability of sorting and rating are remarkably high.
MAPPING PROCESS: A Step-by-Step Guide
The concept mapping process we present for practice improvement is a hybrid approach, blending research application features with program planning elements. This section details our process and key lessons learned from our quality improvement project, summarized in Table 1. A total of 41 individuals participated: 16 patients and 25 clinic staff (physicians, nurse practitioner, and quality improvement personnel). Patients were recruited from the clinic waiting area over six half-day periods between May and July 2014, and received a $10 gift card for their participation. Clinic staff were recruited from clinical research and quality improvement meetings. All participants provided verbal consent, and the study was approved by the university’s institutional review board. No demographic or identifying information was collected.
Table 1: Concept Mapping Steps and Lessons Learned
| Step | Description | Lessons Learned |
|---|---|---|
| Preparation | Identifying and recruiting stakeholders; creating and testing the focus prompt | Build support for project with leadership; include administrative stakeholders in process. Use a third-person prompt. Extensively pilot-test focus prompt. Recruit many participants; allow for attrition between steps. |
| Generation of statements (brainstorming) | Group, individual, or online brainstorming can be conducted simultaneously or sequentially | Allow brainstorming group process to reach saturation; use a mix of written and group brainstorming to encourage responsiveness by all members. Carefully screen output for redundancies and clear wording before using it for the sorting task. Solicit responses from a wide range of stakeholders, even if they may not be involved in further tasks. Brainstorming output can be used for other visual displays such as word clouds. |
| Structuring of statements (sorting and rating) | Individual sorting of statements into categories online or in person. Rating statements in 1–2 domains (impact, importance, feasibility, etc.) | Allow ample time and consider remuneration for sorting task. For practice-based implementation, provide a large table and quiet space for sorting. Consider wrapping process in a practice “engagement day.” |
| Representation (maps and other visual displays) | Creating a point map and cluster maps using multidimensional scaling and cluster analysis. Visual depiction of rated statements in clusters “go/no-go” and pattern matching graphical displays. | Not all projects will require all visual output types: choose output that satisfies task purpose. Point map as an intermediate step is not generally useful: cluster maps are more intuitive displays. Some projects may find cluster maps not useful and can move to the other visual displays. Pattern matching is especially useful for identifying differences in ratings between patients and clinicians/clinical staff. |
| Interpretation | Sharing output with participants for interpretation as a group with facilitation | Can generate maps with interpretative guidance from some stakeholders but not necessarily all. For our practice improvement purposes, interpretation step not emphasized. |
| Use | Brainstormed solutions, visual conception of “problem space,” graphical displays of concordance between groups in ideas can drive practice change | Consider presentation at staff meetings and faculty meetings, and across multiple stakeholders, including those not involved in the other steps. |
Our group concept mapping process comprised six key steps: preparation, statement generation, statement structuring, representation, interpretation, and utilization. Each step is detailed below.
Preparation: Setting the Stage for Concept Mapping
The initial preparation phase involves identifying relevant stakeholders and participants. This group should be composed of individuals connected through a shared involvement in a specific organization, process, or problem. The number of participants can vary, but it’s crucial to include a “wide variety of relevant stakeholders.” While large groups are manageable, typically 10 to 40 participants are sufficient to reach saturation during brainstorming. For projects comparing ratings across groups, a minimum of 10 participants per group is advisable.
Next, the participant group and project leaders collaboratively develop a focus prompt. This question or statement is crucial as it guides the breadth and direction of ideas generated during brainstorming. Careful attention to its design, including logical analysis of potential responses and pilot testing, is essential. Principles of effective question design, such as avoiding double-barreled and ambiguous questions, are highly relevant. Our initial focus prompt was: “What are some ways that patients can be involved at our practice in order to improve the care we provide?” We later recognized this was double-barreled, eliciting responses related to both general practice improvement and patient involvement specifically. Using a third-person stem (e.g., “What are some reasons people might not want to be involved…”) can be beneficial, allowing participants to offer generalizable ideas without personalizing opinions in a group setting.
Generation of Statements: Brainstorming Ideas for Improvement
In this phase, participants brainstorm responses to the focus prompt. Brainstorming can occur individually or in groups, in person or online, either concurrently or sequentially. While group brainstorming can stimulate idea generation, individual brainstorming can mitigate the risk of dominant voices overshadowing the process. Brainstorming literature emphasizes prioritizing the volume of ideas while suspending judgment on their quality to maximize creative output.
In our practice, we initiate staff brainstorming with private written responses to encourage the expression of diverse ideas, followed by group brainstorming. A skilled facilitator is crucial for guiding group brainstorming, managing group dynamics, maximizing participation, and determining when idea saturation is reached. Saturation, in this context, is similar to that in qualitative research – the point where no new themes emerge.
In our project, patients were invited to a conference room post-appointment to brainstorm in response to the focus prompt, facilitated by a study team member. Responses were recorded on a whiteboard, visible to all, creating a hybrid group brainstorming environment. Separate brainstorming sessions were conducted with primary care clinicians and staff in group meeting settings.
Table 2 provides examples of brainstormed ideas from both patients and staff, categorized by cluster. We collected data from 16 patients and 25 clinic staff, with roughly equal response volumes from each group, despite differing elicitation methods. We opted for a flexible approach, using different participant subsets for sequential tasks, rather than requiring the same participants for all phases. This enhanced feasibility without compromising participant engagement at each stage. It is generally accepted that brainstorming can involve a larger initial group, with subsequent tasks like sorting and rating performed by smaller subsets.
Table 2: Results of Statement Cluster Analysis With Examples of Idea Statements
| Cluster | Examples of Idea Statements |
|---|---|
| Access and navigation (12 statements) | Assist patients with navigating interactions with other departments. Identify patient transportation needs and concerns. Give more time for self-management support to the medical assistants. Access texting program to help remind patients about appointments, studies, medications, and referrals.a |
| Empowering patient proactivity and self-care (14 statements) | Patients could participate in creating an action plan for improving their health care.a Encourage patients to bring their medications to their visits. Examination rooms could have computers with health-related or disease-specific education modules for patients to view while they wait. Patients should have high expectations of our health care system, but they must also be patient: modern health care is complicated and frustrating for everyone at times. |
| Formalize patient involvement in the practice (14 statements) | (Patients could) work on developing standards of care in the practice. Provide more opportunities for patient feedback, committees, questionnaires, rating evaluations.a Patients could participate in a “run-through” of practice redesign ideas. Patients could draft materials like brochures for procedures. |
| Patient-provider communication (23 statements) | Allow patients to give feedback to their providers. Patients could write down questions before their appointment.a Provide a way for patients to give feedback to providers about how they are doing. Train the doctors to ask sensitive questions, for example, about sex or addiction. |
| Community resources (20 statements) | (Patients could) assist the practice in identifying community resources that can support health. (The practice could) provide help for patient caregivers. Offer more interactive health education opportunities.a Provide opportunities in the waiting room for things that patients could participate in. |
| Technology (18 statements) | Have a laptop or iPad at check in for patient health surveys and entering patient information. Increase opportunities to communicate with health professionals using portal or website. Have a method to help patients track their progress in managing their chronic diseases.a Help patients with computers in the waiting room. |
[a](#fn1) An idea that appeared in our go-zone diagram as having both high feasibility and high impact.
Structuring of Statements: Sorting and Rating for Organization
The next step involves sorting and rating statements, which can be done in person or online, individually or in groups (though sorting and rating are individual tasks, group settings may offer time efficiency). For sorting, participants categorize each statement from the combined brainstorming sessions, adhering to guidelines: (1) minimum of two statement groupings, (2) no “miscellaneous” or “other” groupings, (3) each grouping must contain at least two statements, and (4) each statement belongs to only one grouping. Sorting approximately 100 ideas, as in our study, can take 20 to 50 minutes per participant. In our project, patients sorted in person, while clinic staff used online concept mapping software. Manual sorting data can be entered into the software or a spreadsheet for analysis (Supplemental Appendix at http://annfammed.org/content/14/4/370/suppl/DC1).
Conducting sorting concurrently with patient recruitment in the waiting room resulted in a high completion rate in our study. Research on group concept mapping shows sorting and rating task completion rates slightly above 50%.
This stage also allows participants to rate each statement. We used two rating dimensions: “Impact if implemented” and “Feasibility of implementation,” each on a scale of 1 (lowest) to 5 (highest). Concept mapping software can visualize rating data as averages across clusters, within statements, or across groups. Demographic data allows for rating comparisons between groups, like patients versus staff. In our study, patients and staff prioritized clusters differently. For example, staff rated “Technology” and “Empowering Patient Proactivity and Self-Care” as most feasible, while patients prioritized “Better Patient-Provider Communication” and “Access and Navigation.” Rating data can be displayed in a “go-zone” diagram, a quadrant map showing statements rated high in both feasibility and impact. Pattern match diagrams can also visualize the degree of agreement between groups (Supplemental Figures 2 and 3).
Representation of Statements: Visualizing the Concept Map
After individual sorting, results are aggregated. First, a point map is generated using multidimensional scaling of aggregated sorting data. Then, cluster boundaries are drawn on the point map to create a cluster map. Statements are numbered for easy identification.
Cluster determination is iterative, requiring user judgment for meaningful clusters. Near outliers can be moved to adjacent clusters for better interpretability. This visual adjustment, engaging participants in the process, is important. A stress index can numerically guide cluster solution choice, indicating map accuracy in representing grouping data. The software suggests cluster names based on word repetition within clusters, which users can modify. Cluster maps can be finalized participatively, by a smaller group, or by an individual analyst. We opted for a six-cluster solution in our study (Supplemental Figure 1).
We used concept mapping software to overlay participant ratings onto cluster maps, visualizing impact and feasibility ratings within each cluster. Go-zone diagrams highlighted ideas with high feasibility and impact (Supplemental Figure 2). Pattern matches compared patient and staff ratings (Supplemental Figure 3).
For practice improvement, rating data visualization may be more critical than the maps themselves. For instance, identifying a “technology capacity” cluster was less impactful than understanding the feasibility and impact ratings of ideas within it, such as “Increase opportunities to communicate with health professionals using portal or website.”
Interpretation and Use of Maps: From Visualization to Action
In a fully participatory process, map interpretation occurs within the group. Final decisions on map and data utilization are guided by the project’s initial goals and desired outcomes. For program planning, feasibility and impact ratings are valuable for decision-making. In our project, the research team developed the final cluster solution, rating data (separate for clinicians and patients), pattern matches, and go-zone diagrams.
These findings were shared with our practice’s Patient and Family Advisory Council. The council responded positively, seeing concept mapping as a useful method for generating and prioritizing their activities, which they are now implementing. (Our council also participated in another concept mapping project on “whole-person orientation in primary care.”) A draft of this article was shared with our department chair, who disseminated it to faculty. We also adapted concept mapping for brainstorming and mapping faculty and staff ideas during our practice’s accountable care organization transition. These data were presented at a department-wide town hall meeting. Furthermore, concept mapping was utilized in our departmental strategic planning. These examples demonstrate concept mapping’s effectiveness in efficiently eliciting, organizing, and visualizing stakeholder ideas.
DISCUSSION: Concept Mapping as a Viable Tool
In summary, group concept mapping is a promising and versatile method with significant potential in primary care. As primary care practices increasingly integrate patients into practice improvement, concept mapping offers a feasible patient engagement technique. It effectively illustrates and quantifies both agreement and divergence between patient and stakeholder perspectives, highlights unique patient insights, and provides a foundation for patient-centered practice improvements. Concept mapping is also a powerful tool for stakeholder engagement in broader clinical research, demonstrating and quantifying the impact of patient involvement. Our experience demonstrates the feasibility and utility of concept mapping for engaging patients and clinicians in a busy urban primary care practice for practice improvement research. Despite needing multiple sessions to recruit sufficient patient participants, brainstorming yielded over 100 quality improvement ideas from both patients and staff. Pattern matching and go-zone diagrams are valuable for prioritizing improvement initiatives.
Group concept mapping is adaptable to various practice improvement and research objectives. Its flexibility allows users to select the most relevant and feasible steps and outputs. However, the cost of proprietary software or statistical programming expertise may be a barrier for some practices. In such cases, practices can opt for simpler approaches: brainstorming, paper-based statement ranking, and spreadsheet software for basic analysis and group comparisons (Supplemental Appendix at http://annfammed.org/content/14/4/370/suppl/DC1).
Regardless of the chosen approach, maintaining the participatory spirit of the methodology is crucial. Visual cluster analysis is recommended for organizing results into actionable plans.
Further research is needed to explore concept mapping processes and outcomes. Future studies could compare concept mapping with other patient engagement techniques in terms of efficacy and efficiency. Understanding patient acceptability is also important, considering the potentially greater time commitment compared to interviews or focus groups, balanced against the opportunity for more active, multi-step patient involvement. Group concept mapping, as presented here, appears to be a viable stakeholder engagement method, and research measuring engagement as an outcome of concept mapping is warranted.
Acknowledgments
We gratefully acknowledge Jennifer Fisher Wilson at the Writing Center at the Center for Teaching and Learning, Thomas Jefferson University, for her manuscript review.
Footnotes
Conflicts of interest: The authors declare no conflicts of interest.
Funding support: This project was partially funded by the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services (HHS) under grant D56HP20783, The Jefferson PCMH Predoctoral Education Project.
Disclaimer: The views and conclusions expressed are those of the authors and do not represent the official position or policy of HRSA, HHS, or the US Government.
Previous presentations: Portions of this article were presented at the 2015 North American Primary Care Research Group (NAPCRG) Annual Meeting.
Supplementary materials: Available at http://www.AnnFamMed.org/content/14/4/370/suppl/DC1/.
