Identifying the usability issues of a validated online questionnaire.
QEESI is a validated questionnaire used for assessing Chemical Intolerance. It is also a self-assessment tool that patients can bring to their doctors to explain their chemical exposures and symptoms.
This is a group project collaborating with UT Health San Antonio. Our goal of this project is to identify the usability issues of QEESI and providing our client with feasible recommendations to improve the design of the questionnaire.
Competitive Analysis | Usability Testing | Analyzing Test Results
Type of Project
Group project | Usability course project collaborating with UT Health San Antonio
Though the QEESI is a validated questionnaire, users can fail to accurately respond to the questions and share it with their doctors due to the current UI design. We’d like to help our client make better design decisions to solve these problems in the future, so we wanted to find out:
1. How users currently take the QEESI and what issues they will have while taking QEESI?
2. What are the greatest obstacles prevent users from sharing the results with their users?
3. What are the feasible solutions QEESI can adopt?
We planned to use heuristic evaluation, competitive analysis, and usability testing to answer these questions.
We conducted the heuristic evaluation to identify the problems of QEESI interfaces based on the 10 heuristics proposed by Jakob Nielsen. Compared to usability testing, it is a much easier way for us to recognize the usability issues from expert perspectives. From the evaluation, we had a holistic understanding of QEESI and recognized its potential problems, which helped us identify what obstacles users might encounter during the usability testing.
We also gave each issue a severity score, which could be 0-4, to find out which issues were the most serious that should be addressed first. We selected the following issues that we think should be given high priority:
Wordy instruction and lack of visual hierarchy
Easy to lose sight of scale while scrolling
Two different color lines in Symptom Star do not match users’ expectations
No salient way to remind users to save their results
Poor error reporting
Since we’ve identified the potential problems of QEESI, we wanted to evaluate the competitors to see how others solve the same problems and what can make QEESI stand out, and then determine what design feature could be the best practices for QEESI. We divided our competitors into three categories: Direct Competitors, Health-related Indirect Competitors, and non-health-related Indirect Competitors.
*Direct competitors - the most direct rivals in the market
*Indirect competitors - different industry/products, can provide insight into how others deliver similar concepts
The Take Away
Making good use of font size, color, and spacing to create a visual hierarchy can help users read effortlessly. Also, using icons/images to communicate can facilitate users to digest the information.
Using labels, color, or size on scale points can help users easily identify them and are also more engaging.
Providing suggestions with the results - steps to take going forward, who to contact, additional materials to read, etc.
Spreading questions out to a few pages and providing progress indicators can help users to have more sense of control to know where they are.
Speaking the language users use can help them understand results and communicate with doctors.
Allowing users to track changes over time by creating a profile is a way for QEESI to stand out.
To recognize the real problems users might have, we conducted nine 45-60 minutes of usability testing. We used BREESI, a screening tool with three questions to determine whether an individual should take the QEESI, as a screener to recruit participants. Our goals are to learn how users currently take this questionnaire, as well as what obstacles prevent users from responding to questions accurately, interpreting the results and sharing results with doctors.
To engage the participants with tasks scenario, we made our task realistic by incorporating the information on the TiLT website and given by our clients, including when people will normally be exposed to the chemical substance and experience the symptoms.
Goal: Take the questionnaire and finish the Symptom Star
You recently moved to a new apartment and experienced some symptoms, like dizzy and skin rashes. Your doctor has recommended you to take the QEESI before your appointment and wants to talk through your results at the appointment.
Goal: Save the whole questionnaire to the computer
Now, you've finished the questionnaire and want to share this questionnaire with your doctor to have further discussion in your appointment, how will you do?
5/9 participants had trouble with the instructions, which were too wordy and long that they tended to ignore them.
Would be better if they were broken up and not doctor, patient, researcher all together.
Lots of text at the beginning, would skip it, mentioning the doctor doesn’t seem relevant.
I wouldn’t read all the stuff before the chart.
6/9 participants thought the 10 point scale was too much.
4/9 participants needed to scroll back up to see the labels.
The choices are so subjective; what’s an 8 compared to a 6?
There are too many choices. Five would be better.
The numbers don’t follow you, so I don’t know which bubble is which, I would end up guessing rather than scrolling back to check.
5/9 participants commented that the assessment was very long, and they had to keep scrolling on the QEESI.
It’s quite long. I wish they had explained how long it would take at the beginning.
Oh, Lord. This is so long.
This page is so long, my hands got tired.
Participants struggled with interpreting the symptom star and 4/9 of them thought that the example of the symptom star was their own results.
Does that mean high tolerance or is it bad for me? Is it good or bad to have high score?
I'm not really sure how to read this.
“Oh! I thought this is my result,” said after the test.
5/9 participants expected to be able to share results electronically with the doctor (e.g., submitting it automatically, and email).
I would expect to be emailed a copy, but they didn’t ask for my email address.
I think that when I clicked ”submit,” it was automatically sent to the doctor. I would be annoyed if I had to print it out and share results that way.
I hope it has summary page and I’ll print it from there after I click submit.
Overall, we found the critical problem of the QEESI was its clutter instruction and description which make it less legible that users tend to ignore or do not see the instruction. Thus, even it indicated that users need to save the questionnaire to their computer, print it and then bring to the doctor, many of our participants still thought that the questionnaire was sent to the doctors automatically.
Based on our research, we provided our client with recommendations on how to make QEESI more user-friendly for the patients.
Simplify the wording of the instruction and make them hierarchical by using different font size, font-weight, and colors. Also, only present relevant information to users.
16 Personalities's instruction before test
Truity split their assessment into several pages and provide progress bar.
Split the questionnaire into several parts and provide a progress bar to let users have more sense of control.
Make the labels follow or close to the scale points can help users to identify the scale points.
Mindoula not only put the label along with scale points, but also user different shades to help users distinguish the scale points.
Psychology Today used simple bar chart and further details to help users understand their assessment results.
Generate the results automatically and possibly change Symptom Star to other visual graphs, such as bar chart that people can easily understand.
Remind the users after they finish the questionnaire to let them know what they need to do for the next step. Also, provide users with an easy way to share the results with doctors, for example, email or send the QEESI to doctors automatically.
16 Personalities provide their users with a message to let them know they can email their results after the assessment easily.
What I learned
Since there were a lot of research works needed to be done in 3 months for this project, we were divided into smaller groups to do the user research. Meanwhile, we also reviewed each other's work to help everyone empathize with users. I learned a lot about trusting the team members and providing different perspectives to prevent us from narrowing our thoughts on a specific phenomenon or solution.
The importance of team collaboration
For me, it was an exciting project that I could have many opportunities to learn how to plan a usability test, conduct usability testing both in-person and through UserZoom, and organized and analyze the results. Thought the whole process was more challenging than I thought, I really appreciated being able to implement what I learned from the class and recognize what aspects I still need to keep improving.