Products live and die by their adoption. Companies developing products and services are placing User eXperience (UX) at the top of their concerns since it creates more positive interactions and makes people enjoy and use their products over those of their competitors.
Enter the UX Scorecard, a tool to help developers make a UX evaluation of their products based on key metrics and heuristics.
In this article, we’ll dive into a UX scorecard and how to make a scorecard for your products. Let’s begin!
What is a UX Scorecard?
A UX scorecard is a report that lets developers of a product quickly visualize metrics of the experience end users have with it. These can take the form of comparisons with previous findings to inform about changes over time.
It is also a useful tool to compare your products with similar ones from your direct competitors, so you can check for improvement points to convert users to your product instead.
How Do I Make a UX Scorecard? The 6 Steps To A Successful UX Evaluation
Making a UX scorecard is relatively easy, but you’ll need a lot of prep work beforehand. Let’s go over the 6 steps needed to create a scorecard.
1. Define the Goals and Parameters of the Evaluation
Before anything else, the biggest question you should ask is: “What am I trying to accomplish?”. There are many avenues to pursue when talking about UX improvement:
- Testing the product’s efficiency and user satisfaction, either as it currently stands or tracking and comparing it over time.
- Identify issues and other problems with the product.
- Compare the product with a competitor or an industry benchmark.
If the test requires comparison with competitors, determine which competitors will be subject to evaluation. Knowing this will give you information about the kinds of tests you can perform to make effective comparisons.
Also, you need to determine the scope of the study. You can do it internally (among the dev team), externally with a control group, or a mix of both. Doing the study internally is a good idea for starting projects or small-scale tests, but if you want your test to resonate with the product’s audience, you’ll want to involve an external control group at some point.
2. Establish the Metrics
UX scorecard metrics are pivotal to evaluating your product successfully. But what are UX metrics? These can be divided into two types:
2.1 Product Metrics
These metrics apply to the product as a whole:
- System Usability Scale: Measures the usability and satisfaction of a product overall.
- Standardized User Experience Percentile Rank Questionnaire: Measures users’ perceptions of the product in terms of appearance, usability, trust and credibility, and loyalty.
2.2 Task Metrics
These metrics are related to specific tasks the users perform around the product:
- Task success: Whether the user completes a task successfully. This can range from failure to struggle to completion.
- Time spent on task: The amount of time a person devotes to a task until it is complete or given up on. Depending on the type of task, the ranges here can be in seconds, minutes, or even hours.
- Ease Questionnaire: The user reports their perceived difficulty when performing the given task.
- User confidence: Asks users how confident they are that they completed the task successfully.
- Usability Metric: Combines Task success, time spent on task, satisfaction, and error counts. These need to be tracked separately and then analyzed using a custom formula.
3. Select Tasks
Now that we’ve established the metrics that can be tested, the time has come to write a test specific to the product and the type of things you want to test.
Usually, you’ll want to test critical functionality from your product so you can be assured it is working correctly and according to users’ needs. Determine what are the types of things your users will do the most often and focus on those tasks. Or conversely, you can also test niche functionality to determine if the development team should devote time to maintain or even upgrade it further.
After deciding the types of functionality you want to test and listing the tasks users must perform, you’ll need to evaluate how long this test will run and the number of tasks present. You’ll want to be thorough but not so much as to tire out your testers. Set a max time frame for the test (1 hour is usually a good stopping point) and a small number of tasks to perform (10 max is usually a good number).
Of course, these values can be higher depending on the product but remember you want quality over quantity here. A lot of small tasks will probably not tell you as much as a complicated tasks where the users must interact with several components of the product.
4. Define Your Metric Grading
Now that both the metrics and tasks have been defined, it’s time to determine how to grade your metrics.
These are usually graded from A to F where:
- A: Excellent performance
- B: Good performance
- C: Okay performance
- D: Poor performance
- F: Very Poor performance
Depending on the type of tasks you test, the grading ranges will change. A very complex task can have an A if it is completed in 5 minutes, while a simple task in those 5 minutes will be considered a D or an F, for example.
The metric grading can also be done with the aid of mathematical formulas.
Success Rate
Jakob Nielsen (2001) gives us a formula for the success rate of a task as:
SR=((Successes+(Passes∗0.5))/NumberOfTests)*100So a test with 50 participants where 45 successfully completed the task and 3 passed would give us a success rate of 93%.
SR=((45+(3∗0.5))/50)*100
SR=0.93*100
SR=93
It’s up to you to define the ranges of values and how they map to the grades established previously. For example, you can have a system where:
| SR | Grade |
| 90 % – 100% | A |
| 80 % – 89% | B |
| 70 % – 79% | C |
| 60 % – 69% | D |
| < 60% | F |
A more strict or relaxed system can change these ranges.
UMUX-lite Score
Another way to do this is by using the UMUX-lite score. Given a task, you’ll need to ask 2 questions and have your testers grade based on the level of agreement.
- This system’s capabilities meet my requirements.
- This system is easy to use.
The calculation of the UMUX-lite score will depend on the scale used.
UMUX−L=((Question1/Scale)+(Question2/Scale))/2*100If you’re using a 5-point scale (from 0 to 4, where 0 is complete disagreement and 4 is complete agreement), the formula becomes:
UMUX−L=((Question1/5)+(Question2/5))/2*100What is a good UMUX-lite score? The closer the value is to 100%, the better. You’ll want to average out your UMUX-lite score results and then apply a grading system much like the one used for success rate:
| UMUX-lite | Grade |
| 90 % – 100% | A |
| 80 % – 89% | B |
| 70 % – 79% | C |
| 60 % – 69% | D |
| < 60% | F |
5. Conduct Your Study and Evaluate the Responses
After gathering all the necessary information about conducting your study, the time has come to put it into practice. Gather the necessary people and have them perform the test you’ve designed. Depending on the type of analysis you want to perform, you’ll want a combination of people responsible for monitoring the person in real time during the test or a questionnaire for the tester at the end.
The answers and data gleaned from these tests will then need to be compiled and processed to create the UX scorecard. Depending on your needs, the values will mostly have to be averaged out or compiled using other mathematical formulas.
6. Create the Scorecard!
Now that you have your results, now is the time to create the scorecard! Choose appropriate ways to convey your results (such as graphs and pie charts), and present them to the people who can use the information gleaned from the analysis.
This can be done physically on paper, digitally using images, or even in an interactive medium, such as a small slideshow or application. That being the case, ensure your scorecard UI is simple enough for anyone to use and navigate comfortably.
Know Your UX Inside And Out With DistantJob´s Experts
Now you know all about UX scorecards and how to create them successfully. Scorecards such as these are invaluable tools to find the faults or issues with a product and communicate with the development team about where the pain points lie so they can better focus their resources in the future.
If you’re looking for expert UX developers to help you put your findings into practice, DistantJob is here to help. We have a vast array of remote developers and other experts under our wing and are sure to find the perfect fit for your needs and company culture at a fraction of the price of what you can find locally. Don’t hesitate to contact us to find out more!
