As 2017 wound down, Amazon’s Alexa voice assistant rose up to the top of Apple’s App Store while Amazon’s Echo Dot was its own bestseller.

But not all of them ended up in people’s homes, some of the millions of those devices sold have already found their way into healthcare in the United States.

“When we went from laptops to smartphones as our primary means of doing computing, that was a major paradigm shift,” said Dr John Halamka, CIO at Beth Israel Deaconess Medical Centre in Boston. “Ambient listening tools probably will replace mobile devices.”

Amazon is hardly alone in the market. Apple Siri, Google Home and Assistant, and Microsoft Cortana are also available. Among hospitals undertaking pilot projects, Amazon reigns – for now. At least one hospital is already girding to implement Google Home as well.

Early work at Boston’s Beth Israel Deaconess Medical Centre

Halamka’s team is conducting extensive early work with Amazon’s Alexa voice assistant and has built a variety of “skills” for the device. A skill is the computing manifestation of a task that a voice assistant performs in the real world.

Halamka said the Alexa APIs used to build skills are straightforward and easy to use and the Beth Israel Deaconess System now has a variety of skills used in the voice assistants in patient rooms.

“If you are an inpatient, what are the things typically you would like to know?” Halamka said. “When will my doctor be here? What’s for lunch? Simple types of things for which you need answers.

"Today, you pull a cord installed in 1955 that flips a relay that turns on a light that means a nurse might be there in 10 minutes. Even modern nurse call systems still are very reminiscent of things that would have been in hospital rooms in the 1950s.”

Halamka questioned why healthcare professionals still think in that particular workflow when there is data in electronic form to answer these questions.

Now, patients can say, “Alexa? Ask BIDMC to call a nurse.” And Alexa will respond, “OK, I have just sent a request for a nurse.” Or, “Alexa? Ask BIDMC what is my diet?” And Alexa replies, “You are restricted to a bland diet for the day. To order a meal, call extension 12345.”

“Why should it be any different than that?” Halamka asked. “You develop these use-cases for which questions a patient typically asks, ones that can be asked through micro-services plumbed through the Alexa API.”

Northwell’s Alexa skill: Determine ER wait times

Northwell Health in New York is currently using Alexa but soon will add Google Home into the mix. The provider organisation is experimenting with many different use-cases for voice assistants. The big one is an Alexa skill that helps users identify the wait times at emergency rooms and urgent care centres that are near a given Zip code.

“You can have Alexa ask Northwell for the shortest wait time near your Zip code or check the time at a specific location,” said Emily Kagan Trenchard, vice president of digital and innovation strategy at Northwell Health. “The skill will give you the wait time as well as the address for that location and direct users to an alternative if the shortest wait time is elsewhere.”

It’s also smart enough to handle disambiguation around places with similar names. For example, Northwell has three hospitals known as Long Island Jewish – Long Island Jewish Valley Stream, Long Island Jewish Forest Hills and Long Island Jewish Medical Center.

“Imagine a scenario where you are in the kitchen and you cut your finger while cooking,” Trenchard said. “In this instance, you know you need to get medical attention and possibly a few stitches. You would say, ‘Alexa, ask Northwell what’s the shortest wait time near 11021?’ Alexa would then query the database of our emergency and urgent care wait times, which are refreshed every 15 minutes, and look at the locations nearest to the Zip code 11021.”

Of those locations, it would then calculate the shortest wait time and report back the location’s name, the wait time and the address.

Say the result that comes back is for a Northwell GoHealth Urgent Care location, but the patient would prefer to go to the emergency department at their preferred hospital. In this case, the patient could say, “Alexa, what’s the wait time at North Shore University Hospital?” In response, Alexa would provide the wait time for that specific location.

Commonwealth: Patients drive voice tech growth

Commonwealth Care Alliance in Boston is using Alexa technology, but the organisation’s plan is to ultimately access all available voice assistants to create individualised care environments for its members and patients.

“We believe this area of patient engagement will continue to grow – driven by both patient experience and feedback,” said Dr John Loughnane, chief of innovation at Commonwealth Care Alliance, which uses voice technology systems from Orbita.

“We also see the great potential in this approach and technology to enable our members with physical disabilities to have more control over their environment.”

Commonwealth Care Alliance is leveraging Alexa to enable its members to perform a variety of tasks. Members can conduct hands-free calling to other people who have Alexa. They can conduct “drop-in” visits with members who agree to give each other access. This is important for individuals who need to check up on someone from another part of a home as Alexa acts as a built-in intercom system.

Members also can do an initial set-up of Google Calendar for personal care attendant schedules. The technology can then check on schedules to see who is going to be on the next shift. They can set reminders for routine tasks such as taking medications, setting up doctor appointments and other medical events.

A Commonwealth Care Alliance member would commonly use voice assistant technology to set up a calendar or schedule for their personal care attendant. A typical exchange might take place as follows:

Patient: Alexa, create an event: PCA #1 at 9 a.m. to 12 p.m. on Monday.
Alexa: Does this repeat?
Patient: Yes.

At this time, the personal care attendant shift is entered into the Google calendar. Once the Google calendar is complete, it is shared with the personal care attendant so they are aware of when their shifts are and who they may contact to arrange for coverage if shifts are missed.

The patient then can ask Alexa, “What are my next five events?” Alexa would inform the patient who is scheduled for their upcoming shifts. This is a powerful use of voice technology because the patients don’t have to keep asking others to check the usually written schedules.

Libertana: Voice tech helping independent living

Libertana Home Health in Los Angeles is using Alexa along with the Amazon Echo Dot. Its goal is to empower people to live independently.

“Given our client population, the ability to use their voices to activate assistance and get reminders to stay on task makes the likelihood of aging in place a reality,” said Debra Harrison, RN, assisted living waiver public subsidised housing manager at Libertana Home Health. “The inevitable physical and mental deterioration of the aging process is a difficult adjustment. Having the right assistance in the comfort of their own homes is invaluable in helping our clients acclimate.”

The pilot study has proven that Alexa becomes a “friend” to the client, Harrison added. And very important, it helps break the loneliness that most experience.

“The client with shingles whose pain makes moving unthinkable can use her voice to play music, relax and stay comfortable,” she said. “The use of cognitive games makes the clients think and laugh, the ability to set a timer, ask for directions and hear a response makes them feel less isolated.”

A patient can wake up in the morning and say, “Alexa, open Libertana.” The Libertana app is opened and there is a morning greeting individualised to that client. Alexa then continues with reminders such as taking medications at specified times and putting on their life alert pendant.

“The voice device shares information about activities happening in the facility throughout that day and provides reminders for them at various times to take specific measurements, blood pressure or blood sugar checks, for example,” Harrison said. “Reminders also are provided to exercise, drink fluids and eat snacks.”

The most widely used feature, which can help patients gain peace of mind, is the ability to request assistance from Libertana staff. With the voice assistant, a patient does not need his or her mobile phone nearby. After finally getting comfortable in a reclining chair, for instance, the patient might realise he or she left the mobile phone in another room. With Alexa, the patient need only speak to get the assistance needed – perhaps getting to the bathroom safely, asking someone to get pain medication or reporting feelings of lightheadedness.

“At the beginning of the pilot, which we limited to one of our facilities, we identified that some clients were unfamiliar with or forgot how to activate the skill,” Harrison explained. “We spent time early on to educate users on use of the device; essentially, reinforcing their need to say, ‘Alexa, open Libertana.’ Not all had this challenge, but of those that did, once they overcame it, all program participants have become very comfortable using Alexa.”

The HIPAA situation

None of the voice assistants yet have HIPAA business associate agreements to adhere to the HIPAA privacy and security rule. As the Office of Civil Rights will tell a healthcare organisation, however, as long as there are none of the 18 identifiers passing over a clear channel, the organisation is safe.

“Just telling Amazon here is a patient in 701 who wants to know when lunch arrives, there is no HIPAA information going over that kind of dialogue,” Halamka said.

Ultimately, if one can just be somewhere and ask a question, and the question is a skill on the back-end powered by micro-services that are attached to a patient’s record or to a dietary system, that is so much friendlier for all involved, Halamka said.

How-to tips for using voice assistants

Commonwealth Care Alliance, Libertana Home Health and Northwell Health are pioneering the use of ambient listening voice assistants in healthcare, and they all have so far achieved success with the technology as measured by patient satisfaction.

Northwell Health’s Trenchard offers four tips to peers at other healthcare providers looking to get started with a voice assistant project.

1. Experiment away: “First, don’t be afraid to start experimenting because both Amazon and Google now have SDKs that are open to anyone to use – you can try 50 different skill ideas before you land on one that your end users might really love,” Trenchard said.

2. Don't expect perfection: Working with voice assistants is an easy thing to start, but a harder thing to perfect. Plan on not only having developers on the project, but user experience and copy people to help generate the utterances. This will likely take up the bulk of your time once you determine the nature of the skill you want to build.

3. Utilise your APIs: A healthcare organisation does not need anything beyond its existing website to get started, Trenchard advised. Northwell has its Alexa skill running off of its content management system and taking advantage of the same APIs that the website and digital signs use. That said, there are more tools available to help visualise workflows, manage utterances and even cross-compile code for Google, Alexa and other voice assistants. Once a healthcare organisation gets up and running, these tools could be a great help, she said.

4. Step-by-step: “And fourth, think about small, simple tasks and information requests,” Trenchard said. “If it’s going to take more than one or two steps to get the information a user is looking for, it might be better accomplished in a visual interface than in voice.”

The results so far

It remains very early days for the use of voice assistants in healthcare.

Consequently, provider organisations experimenting with the burgeoning technology do not have many hard results like return on investment to report. But these organisations are showing what the technology can do, and have patients to back them up.

“We’ve proven the technology is very robust,” BIDMC’s Halamka said. “In all of our testing and piloting, the technology works. We’ve proven the development time to create these things is very short because Amazon has really democratised the technology. You do not write programs, you write scripts and configuration files. It’s the kind of thing a person can do in a weekend.”

The use of the skill has been growing steadily, Trenchard added.

“So far we’ve seen over 1,000 utterances captured in hundreds of sessions,” she reported. “I think it’s still a very novel idea that consumers can find this kind of information from Alexa. From patients we’ve spoken to, they feel it gives them a sense of comfort to know how long they will have to wait when they walk in at an urgent care centre or emergency room before they leave the house.”

Voice assistant technology also has been a great way for Northwell’s internal development and user experience teams to experiment with voice interfaces and learn from the feedback from real-world users.

“There’s no substitute for having live user experience data when exploring how a new technology could serve our patients,” Trenchard said.

Libertana Home Health does not yet have documented results but the pilot study’s purpose was threefold.

“To see if clients were able to use it consistently, benefitted from it, and felt empowered by using it,” said Harrison of Libertana Home Health. “We did see results in all areas. One client with a cultural based fear of using Alexa, due to having Big Brother watching, would not use it in the beginning. I actually thought he was a lost cause. A few months later, however, with constant support from our staff, he saw its benefits and now uses it on a daily basis.”

Commonwealth Care Alliance currently is collecting data and expects it to be available in the first quarter of 2018. In terms of anecdotal results, members have reported they find the use of voice assistants makes it much easier for them to call and create schedules, giving them more autonomy in their lives, said Loughnane.

While provider organisations experimenting with ambient listening voice assistants may not have a lot of hard data results to report quite yet, they have learned lessons.

“We’ve learned that voice assistants need lots of training,” stressed Trenchard. “We have over 1,500 utterances logged to date and there are still things that Alexa sometimes does not know how to process. We’ve also learned that Alexa is not very good at dealing with accents. We have high hopes for voice assistants in the future.”

Today, commercially available voice assistants are intended for group settings, and that makes security a very tricky thing to tackle, especially when dealing with sensitive health information. While newer releases have shown that voice agents can get better at recognising which individual in a household might be speaking, the technology is not yet sophisticated enough that Northwell feels comfortable sharing care information with patients through the devices. For now, Northwell is focusing on use-cases that don’t involve sensitive data.

At Beth Israel Deaconess System, the early feedback from patients, board members, administrators and caregivers is that the technology is “fabulous,” Halamka reported.

“The idea that instead of using a pull-cord or a phone or an email, that you can just in the room have ambient listening and actions happen reduces the burden on everyone,” he said. “How much burden do people in healthcare feel right now in the era of post-meaningful use HIPAA Omnibus rules? Burden seems like the keyword of 2017. The idea that this is a real time-saver and burden-reducer is a real lesson learned.”

The impact on IT is very low – a healthcare organisation does not need to find someone new or spend a lot of money, and that is an important lesson learned, Halamka added.

“The skills that are necessary to do this are really just scripting, not programming,” he explained. “The technology is mature, really low cost, low impact, easy to deploy. So, be aware of the privacy issues, given that it is early. Amazon will eventually get to HIPAA compliance on Alexa, they are just not there yet.”

The future: Patience with patients

Harrison cites lessons learned that revolve around the patient.

“We realise that our clients need reinforcement in learning how to open the Libertana app,” Harrison said. “Initially, this has to be done on a daily basis. This was simple for us since we have caregivers going into apartments on a daily basis. We learned the need to be patient, and were pleased with the pace of adoption once we crossed the learning curve.”

Libertana also learned that it needed to choose patients carefully. For the pilot, success was in part likely due to the time spent reviewing clients’ personalities, their willingness to participate, their general compliance in their care, and their cognitive ability.

Beth Israel Deaconess, Northwell, Commonwealth Care Alliance and Libertana Home Health have proven that voice assistants can play a very helpful role in the delivery of healthcare. They also have shown there is some work yet to be done for these voice tools to proliferate within healthcare organisations.

Ultimately, voice assistants hold great promise for healthcare — promise that may be realised sooner than many now think.

Originally published in the US edition of Healthcare IT News.

 

WEBINARS AND EVENTS

WHITE PAPERS AND WEBINARS

White papers