Class Project
We have created a hypothetical scenario, where Spotify users would be recruited to test the “Hey Spotify” feature (voice assistive feature by Spotify) for listening to music while driving.
Project Overview
UserScout is a chatbot that helps researchers and recruiters in screening out the right candidates for a usability testing study. It is a bot where companies can customize screener questions for their study.
6 Weeks | Spring 2022
Class Project
Conversational UX Design
Jichen Zhu | Sammika Rastogi
Analyzed the following metrics: number of sessions, page views, sequence of pages visited
Time Frame: 1 January 2022 - 5 October 2022
Analyzed real-time recordings, website page clicks, mouse movements, and scrolling patterns.
Time Frame: Last 30 days
We used Google Optimize to create our A/B Testing variations. It allows running some experiments that are aimed to help increase visitor conversion rates and overall visitor satisfaction.
The time frame is short to provide quantity and quality data for the website.
Hotjar could not track the Map and Zip Code Calendar, making it difficult to understand how users use and interact with them.
We used Google Form to conduct a survey asking about user testing recruitment.
We used Excel to keep and organize the Training Data and Prompts neatly.
We used Lucidchart to create our flow diagram to make sure there are no lose ends.
We used Voiceflow to build our prototype and used it to test it on with our participants.
Our initial concept was to create a voice bot for screening the right participants for an usability study. With the help of research we formed few assumptions about using the voice bot for our screener.
Preferred Method For Screening:
Usually via email messages with links to a screening
questionnaire, or through a user testing site like usertesting.com or optimal.
Preferred Method For Scheduling:
Via email, a calendar or tools like Calendly that is set up to accept certain time slot selections from the participants.
Challenges in the Participant Recruitment Process:
Finding the right participants for the research
“Yes, this is always a challenge. There are fewer and fewer places where you can share a screener to get interested and relevant participants.” - UX Researcher 1
People don’t show up for the interviews
“Sometimes people just don’t show up. It can be difficult to get people to come in person. Zoom has actually made some user research a bit easier because people will usually show up on zoom. But if you definitely need them to be in person, then it can be challenging.” - UX Researcher 2
Efficient
The interaction should be precise which would convey only relevant information. Also provides users flexibility to the scheduling/canceling appointments.
Reliable
The interaction is a screening questionnaire, so the participants should feel the sense of security and trust.
Instructive
The interaction of helping the users navigate through the screening questions and scheduling process with ease is important as it might be new to them.
Task-oriented, infrequent interaction
Simple impersonal task which requires no relationship or bond between the system and the user. Collecting users’ data, anonymously, isn’t necessarily a personal task.
Higher level of personification = more human-like behavior
A voice call doesn’t have a backstory as that may be a potential distraction in a task-oriented scenario.
In this scenario, the system builds trust through three key factors: Efficiency, transparency and consistency.
Screener Questions Flow
GREETING
Hello, we at Spotify would like to learn more about people who are using the our newly launched feature,"Hey Spotify”, while driving.
We want to know what our users think about it. Would you be interested in participating in our study?
READY_COMFIRM
I’ll start by asking you a few questions to see if you are the right fit for the study.
Let me know when you’re ready.
ENDING_RESPONSE
Thank you for your time. Your response has been recorded and you would not be contacted for participating in this study.
USAGE_PERIOD
Awesome. To start off with, how long have you been using Spotify?
LISTENING_HOUR
How many hours per day do you listen to music?
LISTEN & DRIVE
Do you listen to Spotify while driving?
NEW_MUSIC
Is it difficult to find new music while driving?
FEATURE_INFO
Have you used "Hey Spotify" yet?
Hence, we decided to Switch to a Chatbot prototype instead of voice, as that process may seem more natural to the users and also could prove to be Less Time Consuming.
Why a Chatbot?
Chatbots are excellent for working through complex chat flows.
Training a chatbot is much easier than training a voice bot.
Chatbots can communicate using not just text but also audio-visual media.
A chatbot can be incorporated into apps, without navigating the users to a different flow.
.webp)
Learnings
Our work on this project helped us understand many fundamental requirements for conversational UX products. We also learned about various challenges associated with designing an excellent conversational UX product. An example would be if one had to design a conversation flow when designing a complex and tedious conversation flow to cover the content but with an appropriate amount of speech.
Biggest Takeaway
Screener is still a very immature product but has a lot of potentials. There are so many good features waiting to be designed and developed to help UX researchers or companies to manage their user testing screener. However, due to the scope of this class project and the time limitation, those features have to be proposed for future steps. Overall, We felt satisfied with designing our first chatbot product.
Next Step
For future consideration, the companies can gather information regarding user’s usage patterns and provide that data to the bot for a better and smooth screening process. Also, to enhance the bot’s accessibility, we would like to add a voice call feature for less tech savvy users who are not too comfortable with google forms, surveys or chatbots.
