A voice user interface (VUI) research case study for Google Map integration.
14 weeks, Sept - Dec 2023
Primary drivers of personal vehicles
Accommodating different levels of tech literacy
Has prior exposure to VUIs in the world
School Team Project (4 people)
Voice User Interface Research
Featured at World IA Day Los Angeles 2024:
During the 14-week studio I led user-test trial planning, team project management, and voice prototyping. I was in charge of creating all the dialogue trees using information architecture, and translating them into actual voice feedback on proto.io as a functional prototype for user testing. As a team we collaborated on the micro-interactions and visual feedback.
Voice User Interfaces (VUIs) are invisible systems. Sometimes a VUI system may be accompanied by simple visual signifiers for interactions and feedback loops. However the primary interacted interface is with via our auditory senses using our voice, regardless of sight abilities. There are growing numbers of VUIs entering the tech space, with several well-known examples in the consumer market, including Siri, Amazon Alexa, Hey Google, Bixby, and more. This research project investigates what actually goes into engineering successful VUIs, and how to prototype simple interactions to test different variables for this invisible interface.
This research project studies different aspects of voice user interfaces (VUIs) through prototyping a new voice interface on an existing system (Google Maps). Through user testing trials each testing specific variables, we investigated what goes into creating a successful VUI by studying friction points, conversation patterns, mental models, and different feedback signifiers.
proto.io VUI prototypeThe goal is to obtain authentic responses from testing through the use of live prototypes.
Guiding Questions for the user test trials:
How Might We...
...prototype personalized food recommendations with VUI based on individual user preferences, previous orders, and dietary restrictions?
...minimize distractions and ensure safe VUI operation experiences?
...design VUI to answer multiple requests effectively in one command rather than separate conversation interactions?
...structure a conversation that inspires trust in a VUI to execute commands perfectly?
...integrate food ordering with in-car navigation and driving through VUI?
...influence user trust, especially in terms of order accuracy, timely delivery, and payment security?
User Testing Recruitment Criteria - An even mix of user testers were recruited for each test. (4 recruits per test)
Generations:
Millennials
Gen Z
Technological Skill:
Experienced users of in-car smartphone mirroring technology such as Apple CarPlay and Android Auto
Occasional to experienced users of Google Maps for navigation
Average to savvy technological literacy
Average to no experience with using Voice User Interfaces
This research project was conducted in 4 segments:
1. Testing a sound/voice-based alarm/notification
2. Testing different screen-based feedback for voice inputs
3. Testing different mental models
4. Testing overall dialogue flow and visual feedback
These 4 separate tests were devised to methodically research different aspects that voice user interface design needs to consider. Each test was performed using a working live prototype on Protopie (proto.io) that reacted directly to voice input.
Tests 1, 2 and 3 utilized A-B testing and numeric scoring to evaluate effective design choices. Each test used a specific task flow to test variables. The final 4th test on overall dialogue flow utilized qualitative feedback and numeric scoring to evaluate effectiveness. 3 different task flows were tested for potential friction points, comprehension, and overall ease of use.
Test 1
Test A: “ping” sound notification
Average Situation Awareness Level:
8.75/15
Test B: “ping” sound notification and voice over
Average Situation Awareness Level:
13.5/15
Testing notifications and feedback using Situation Awareness Evaluation on 3 levels:
Perception (How well the user's attention is attained)
Comprehension (How quickly the user understands the prompt/notification)
Projection (How well the user can predict what should happen next)
Results:
A “ping” sound combined with voice over is useful for creating Situation Awareness, especially when it is the driver’s first time ever hearing this notification. The “ping” is the necessary attention grabber to create successful Perception.
The voice-over explains what is going on to create successful Comprehension, as well as asks the driver if they want to switch to a new route. The voice-over’s question that follows the description of the situation informs the driver of what they can expect to be the next procedure to create successful Projection.
Test 2
Test A: VUI screen feedback with color only
Average Situation Awareness Level:
13/15
Test B: VUI screen feedback with text only
Average Situation Awareness Level:
11/15
Testing notifications and feedback using Situation Awareness Evaluation on 3 levels:
Perception (How well the user's attention is attained)
Comprehension (How quickly the user understands the prompt/notification)
Projection (How well the user can predict what should happen next)
Results:
Color UI was more effective than using text for an in-car voice user interface based on Situation Awareness. When asked about a potential combination of color and text, testers were hesitant because they felt it would be informational overload. Testers did mention that a verbal repeat of their total order would be helpful to double check that their order was correctly heard.
Test 3
Test A: “Switch to a different route? Yes or No”
Ease of Use Score:
5/5
Test B: “Switching to a different route in 5 secs. Cancel.”
Ease of Use Score:
2.5/5
Testing user's mental models and evaluated user projection under an emergency navigation change:
Switch to a different route? Yes or No (Ignore is stay on current path)
Automatically switching to a different route in 5 secs. Cancel. (Ignore is to choose new path)
Results:
Providing a “yes or no” choice is a much more preferred option to cancelling an impending change during driving.
The choice provides a much clearer indication to the driver of what process is happening and when they should provide input to the VUI.
Test 4
Qualitative feedback was collected for the flows in this test for overall impressions.
Test Dialogue Flows:
Placing an order for pick up or delivery home from Mendocino Farms.
Making a restaurant reservation, routing to multiple locations, and sharing your ETA to multiple parties.
Picking up a curbside order from Ralph’s Grocery Store, routing to multiple locations, and sharing ETA to multiple parties.
Results:
The flows were easy to follow overall and having redundant information both spoken and written on screen was helpful in case the user missed something audibly.
Users overall would like to have a unique sound to indicate when it’s user’s turn to speak in addition to the yellow glow.
Successful design was determined upon higher numeric scoring from our user test trials based on our evaluation methods.
Outcome 1:
Clear verbal affirmation and reiteration is needed for lengthy orders, commands or reservations.
Outcome 2:
Visual feedback can be kept simple and minimal, as long as the auditory feedback is sufficient in notifying the user if the VUI is listening, what the VUI has heard, and whether a command has successfully been executed.
Outcome 3:
The visual feedback should serve to support the voice as a reassurance, and not distract the user’s eyes away from the road.
Outcome 4:
The VUI should never assume the user’s intentions. Whether in an emergency alert situation or a food ordering situation, the VUI should provide the user options rather than an automated decision so that the user still feels in control and can build trust.
VUI Persona
This is Giga, the persona for our Voice User Interface. We wanted to create a VUI bot that was reliable and calm, especially for the use case within a car as a driver or passenger.
Giga’s personality is more on the warm side, however he maintains a more formal tone when speaking.
Emotionally we have designed him to be more neutral as well so that he remains calm in a stressful environment like driving.
Visually we designed Giga with several different faces. The largest you see is the default face, with four other variations below dependent on the situation.
Importance of Visual Signifiers in VUI
Visual signifiers are still very important to the success of a VUI.
Color Scheme:
Blue - Voice UI Speaking
Yellow - Prompt for User to Speak
Visual Confirmation Feedback:
Order confirmations
Destination confirmation
Notification Icons:
Representing different requests performed through the VUI as well as updated system status based on prior dialogues.
I was able to learn a lot about Voice User Interfaces through rigorous user-testing in this project.
This research project primarily helped myself realize through practical experience the importance of building strong Information Architecture in planning out user experiences. Whether it be pages on a visual UI project, or dialogue flows in a VUI, Information Architecture helps identify where friction points are located, where design iterations should occur, and where strategy can come into play.
Particularly with an invisible interface such as VUI, information architecture is a straightforward way to visualize what goes on in a dialogue flow as well as the decision trees involved. It can help the designer realize if a dialogue is insufficient, or is over-complicated. It reveals the potential length of a conversation and informs the designer if it could be potentially too long for practical use inside a moving vehicle. It shows cases where verbal feedback should have visual signifiers in the screen for more immediate comprehension. Overall through this project I have found a great appreciation and use for information architecture in UX Design.
