Overview

This program at an academic medical center utilized a patient and family-facing app to improve patient safety and communication between patients and their caregivers and healthcare providers.

Department

Department of Medicine

Collaborators

Elizabeth Lilley
Frank Chang
Ann DeBord Smith
Jessica Cleveland
Awatef Ergai
Zachary Katsulis
James Benneyan
Esteban Gershanik
David W. Bates

Status/Stage of Development

Completed

Practice Setting

Academic Medical Center

National/Policy Context

• To make e-Health tools more user-friendly for patients and their caregivers, developers now focus on the design of patient portals and mHealth apps for patients to self-manage their care. 
• Incorporating new mHealth app design principles can lead to better engagement with patients, which is linked to better outcomes, shorter lengths of hospital stay, and decreased health costs when compared with those associated with less engaged patients.
• eHealth tools are also a new source of data on patients (both inpatient and ambulatory) and caregivers for researchers and healthcare systems.
• Current analysis of patient safety events are mainly from clinicians’ perspectives and could benefit from including patient and caregivers’ perspectives as well.

Patient Population Served and Payor Information

• Adult medical units (inpatients) 

Leadership

• David Bates was the overarching project lead 

Project Research + Planning

• In total, there were 11 individual usability sessions, 3 small group usability sessions, and over 250 interactions with patients and families during engagement rounds conducted to inform MSC app usability. 
  • Over 7 months, up to 286 participants interacted with the MSC App.
  • The innovators targeted clinical inpatient units including the medical intensive care unit (MICU), oncology unit, and intermediate vascular unit. 
  • Individual Scenario-based usability sessions were 15 to 20-minute sessions in which participants completed tasks in the MSC App for a set of predefined scenarios. 
    • Patients – and their care partners (caregivers), if present – who were chosen to participate had to be neurologically alert and clinically stable. 
    • 18 scenarios each consisted of two user tasks and were designed to reflect typical patient situations while allowing for systematic evaluation of specific features of the app. 
    • Two researchers were always present: one conducted the session and the other took notes and recorded completion time for tasks. 
      • Participants were asked to “think-aloud” as they completed scenarios on an iPad using a predefined template to maintain procedure consistency.
      • Researchers captured think-aloud comments, completion successes and errors, instances when assistance was required, and the amount of time required to complete the task. 
  • Group Usability Sessions consisted of hour-long interactive sessions conducted with the hospital’s Patient & Family Advisory Council (PFAC). 
    • Participants were shown the MSC App and given two scenarios to complete (all participants were given the same scenarios to facilitate group discussion afterwards). 
  • Engagement Rounds were conducted biweekly to patient rooms and family waiting room areas, during which MSC App Version 1 was available for use while development continued to inform refinements for Version 2. 
    • Engagement rounds informed patients and care partners of the MSC App availability and answered any additional questions they had. 
    • Any concerns that arose during these engagement rounds were noted by team members and triangulated with data from individual and group usability testing to further inform app development.

Tools or Products Developed

• MySafeCare Safety Reporting Application (MySafeCare [MSC] App).
  • The MSC App is a Web-based and mobile-enabled app facilitating real-time communication of medical concerns from hospitalized patients and care partners to their medical provider team. 
  • Software development was completed by OpenClinica, LLC.
  • The MSC App has two components: 
    • A Patient and Carepartner Facing Component is designed to run on a mobile device, tablet, or laptop computer. It provides users with the choice to remain anonymous, the option to enter the severity level of their concern, and the chance to further elaborate on their concern in their own words. 
    • It also includes a “Compliments” category for feedback about the staff involved or care received in their hospital stay. Figure 1 shows examples of screens that the patient would scroll through.
    • Categories of concern include:
      • Plan of Care
      • Medication
      • Medical Devices
      • Communication
      • Infection
      • Privacy
      • Pain
      • Waiting Time
      • Other
    • A Clinical Dashboard Component displays submissions to appropriate clinical staff members.

 

Training

• Users need to get familiar with qualitative analysis methods (thematic analysis) 

Tech Involved

• Desktop computer
• iPads
• Smartphone Application

Team Members Involved

• NPs
• PAs
• Physicians

Workflow Steps

• Patients arrive to medical appointments as usual and are provided with the link to the MSC application in their patient rooms. If they are concerned about any health, communication, treatment, or safety issues they can alert their care provider team of their concerns (providing as many details as they wish). Patients can access the mobile app on their own mobile devices or laptops/tablets.
• Care providers receive complaints in real time on their own devices (e.g. workstation computer, iPad, email, etc.) and are able to respond to them right away.
  • Typical concerns include changing medication doses, cleaning up spills, assisting the patient to the bathroom, reassessing treatments, etc.

Outcomes

• The MSC application was evaluated based on four heuristics:
  • Match between system and the real world
  • Consistency and standards
  • Flexibility and efficiency of use
  • Aesthetic and minimalist design
• These four heuristics were found to address end-users’ improved design aspects aligning with all of Nielsen’s heuristics (See Glossary).
  

  Fig 4. How Nielsen’s Heuristics line up with MSC’s

Future Outcomes

• In the future, more research will be conducted to determine the best guidelines for apps targeting patients who are acutely ill in hospitals, who may face different challenges when using the app compared to healthier patients or their caregivers.
• Future work also includes further optimization of the MSC app through implementation across different clinical units, allowing the collection of usability feedback from different patient populations. 
• To make hospitalized patients aware of new patient engagement initiatives, clinics can provide viewable information in patient rooms to notify patients/family when conducting daily engagement rounds.

Benefits

• MSC targets “near-miss” and “unsafe condition” cases, allowing incidents that are concerning to patients and care partners to be handled not only after they occur, but also at any point where a patient feels a situation is unsafe. 
• MSC is efficient and flexible in balancing a library of reliable, structured, and coded data while not completely preventing partial data collection functionalities (providing for a real-time response or continuous learning).
• Unlike with other mHealth applications, MSC gives users complete control of their answers by providing both structured questions for efficiency and also optional questions allowing for free-form text data entry.

Intervention-Specific Challenges

• Patient-reported safety concern systems have an inherent limitation in that interpreting safety issues is a complex process, as their causes are mostly multifactorial.
• Patients who have a higher degree of illness often do not have the energy to read the lengthy informed consent text required by the IRB (Institutional Review Board), which could affect patients in non-research contexts if the app was used in clinic. 
  • In response, MSC collaborated with the IRB to revise the consent text and create an educational video about the informed consent information. 
• Another concern was the time patients and care partners spent scrolling through long pages of questions, which was mitigated through separating different sections into different pages based on the types of questions.
  • Fewer questions per page minimized scrolling, and applying branching logic to the questions decreased user burden. 
• Because they wished to provide an anonymity option, MSC had to use a web-based platform for its application instead of having users to download a mobile app to a device.
• MSC’s major priority is to tailor phrasing and word choice of the app content to effectively describe the intent to the end-user. 
  • There is often a misunderstanding of the meaning of “safety concern” among patients, caregivers, and clinicians.
  • App developers must ensure that information and terminology is consistent and understandable across varying users. 
• For patients who are more acutely ill or have just undergone treatment, the challenges observed could be magnified. 

Glossary

• Nielsen’s Usability Heuristics: a set of ten guidelines for user interface design:
  • Visibility of system status: the system should always keep users informed through appropriate feedback within reasonable time. 
  • Match between system and the real world: the system should speak the users’ language using familiar words and concepts rather than system-oriented terms. 
  • User control and freedom: a clearly marked “emergency exit” should be available for users to leave unwanted states (system functions chosen by mistake) without having to go through extended dialogue. 
  • Consistency and standards: users should not have to wonder whether different situations or actions mean the same thing. 
  • Error prevention: a careful design should prevent problems from occurring in the first place. 
  • Recognition rather than recall: minimize the user’s memory load by making objects, actions, and options visible; instructions should be easily visible or retrievable whenever needed. 
  • Flexibility and efficiency of use: allow users to tailor frequent actions to cater to both experienced and inexperienced users. 
  • Aesthetic and minimalist design: dialogue shouldn’t contain irrelevant information diminishing the relative visibility of relevant units.
  • Help users recognize, diagnose, and recover from errors: error messages should be expressed in plain language and constructively suggest a solution. 
  • Help and documentation: help sources should be easy to search, focused on the user’s task, list concrete steps to carry out, and not be too large. 

Sources

• Couture B, Lilley E, Chang F, DeBord Smith A, Cleveland J, Ergai A, Katsulis Z, Benneyan J, Gershanik E, Bates DW, Collins SA. Applying User-Centered Design Methods to the Development of an mHealth Application for Use in the Hospital Setting by Patients and Care Partners. Appl Clin Inform. 2018 Apr;9(2):302-312. doi: 10.1055/s-0038-1645888. Epub 2018 May 9. PubMed PMID: 29742756; PubMed Central PMCID: PMC5943079.
• Nielsen, Jakob (April 24, 1994). 10 Usability Heuristics for User Interface Design. Nielsen Norman Group: World Leaders in Research-Based User Experience.