top of page

Summary

Goal:

replace the current five star rating system

Problem

the five star rating system is flawed due to high cognitive thought on the user, which leads to grade inflation

Solution

a revamp of the five-star rating system. Each star represents a different category, the star is given via thumbs up or thumbs down. The user will also have an optional option to add comments on the experience. 

Research Findings:

  • choice architecture can increase rating participation

  • grade inflation is when there is an overwhelming number of good reviews

  • five stars are the most visually appealing 

  • thumbs up/down takes the least cognitive effort

  • users post positive much more than negative reviews

  • users want low cognitive effort

Methods:

  • Interview Study

  • Research Article Analysis

  • Journey Map
     

User Group:

lyft passengers who are indecisive, but would like to convey quick accurate feedback to the company.

197765e0af037c960864d4efa05ef73c.png

UX learning studio team project at Purdue University with no affiliation to Lyft

​

February 2022 - March 2022

Final Design

My project (13).png

Reflection

Reflection

This was one of my first UX projects, and also my most fun one yet. This team had three members, yet I was the only UX design major in my group. This led me to lead this project and find the right solution. 

​

Leading a project so early on in my UX knowledge went smoothly, and I am happy with the result. 

Full Case Study

Lyft-Logo.png

Redesigning the Five Star Rating System

Background.

Lyft is one of the fastest-growing companies in the “sharing” or gig economy, reaching a milestone of 1 billion rides in September 2018. Consider that one of the world’s largest transportation companies owns no fleet of vehicles, and employs less than 5000 people, despite the fact they facilitated over 50 million rides a month in hundreds of US cities prior to the pandemic, covering over 95% of the US population. However, health risks associated with COVID-19 caused Lyft’s revenue to drop by almost two-thirds in 2020, impacting both Lyft corporate and their contracted drivers. One of the most important mechanisms that enables the gig economy is qualified and accurate ratings from users, which allow operators to divert resources appropriately and reward contractors that provide high levels of customer service. The current rating system tacitly encourages users to only rank their driver using very good or very bad rankings, due to the known limitations of a five-star scale—a system that is even more problematic when both riders and drivers are in short supply. Your goal is to replace the current five-star system. You must consider the ways various rating and review systems constrain the input of users, the ease of use (e.g., speed of input in a busy urban environment, evolving pandemic policies), and the ways in which this information could be used maliciously against drivers, while also building on the evolution of rating approaches in the gig economy at large.

Problem

The
Problem
.

The five-star rating system is a flawed way to rate drivers for Lyft. Due to the high cognitive thought that it takes for a user to think about and score their driver/passenger the user is likely either a) going to skip the review or b) going to score the user either really good or really bad, usually users tend to score really good. Grade inflation is a major flaw of five-star ratings. Grade inflation can be seen in Lyft's driver rating requirements. Lyft drivers must stay above a 4.8-star review rating or they will be at risk of becoming deactivated. This makes a 4-star review appear bad, even though â…˜ stars equates to an 80% score. This shows that most users are rating drivers a 4.8 or above, which is at the top 96% tile of the five-star system. This shows heavy inflation.

 

​

Bad

Good

Group 5.png

4.8

Our user group are Lyft passengers who are indecisive but would like to convey quick accurate feedback to the company.

Research

Understanding Lyft's Current Review System

Lyft Currently uses a five-star system. The user chooses the number of stars they think the reviewee deserves. They then justify with pre-written prompts. They do this for a driver and passenger rating.

oldlyft1.png

1.  User Adds Tip

2. User gives Star Rating

​

oldlyft2.png

3.  User Justifies Rating with Prompts

4. User Submits Rating

​

Things
to Note:
  • A Lyft driver must maintain at least a 4.8-star review average or else they will become at risk of being deactivated. This implies that anything lower than a full five-star rating will hurt the driver.

  • Unrated rides automatically become a 5/5 rating.

  • Unfair ratings can be contested.

  • All feedback given to a driver is anonymous. (lyft.com)

How can we encourage users to review?

The environment for a user review is important. This is called choice architecture. The wording of a review prompt is important for the user. If a reviewer is simply asked to leave a review they are less likely to review. If the user is asked something along the lines of “Help your driver improve with your feedback” they are more likely to leave a review (Donaker et al. 2019).

Understanding Grade Inflation.

Grade inflation is the leading cause of the downfall of the five-star system. Grade inflation is when there is an overwhelming number of good reviews, more than the subject deserves. Users tend to either leave a really good review: five stars, or a really bad review: one star (Fowler 2017).

The average shopping sites review is 

4.3

Shopping Sites.png

46%

of Yelp's reviews are 5 stars

5 star.png

The minimum rating until a Lyft driver can be deactivated is 

4.8

Lyft AVG.png
"unless the experience is really bad, i leave a five star review." - an interviewee

Five Star
or 
Thumbs Up?

Rating Table.png

There are three main types of rating systems: binary visual, binary textual, and the five-star system.

(Chen 2017)

(Fowler 2017)

A comparison of the three rating systems. 

fivestars.png
Five Star
  • Most Visually Appealing

My project (4).png
Binary Visual
  • Lowest Cognitive Effort 

  • Visually Appealing

What do Lyft Users Want to Be Rated On?

I asked drivers and passengers what they thought was most important via a write-in survey. 

Here's how I broke down the data:

  • This showed what drivers/passengers feel positively or negatively about their driver/passenger.

  • Informed us of visual hierarchy based on user rating topics of importance. 

  • Following this hierarchy can increase choice architecture 

Ideation

Pulling it all Together.

Project #2 Affinity Diagram (5).jpg
Affinity Diagraming

Journey Mapping.

Lyft Journey Map.png
Forming an idea

Putting all of our research together, a solution was formed. Seeing that the five-star system is the most visually appealing to a user we wanted to not get rid of the five-star system, but fix it. The biggest problem the five-star system had was grade inflation. Based on our findings grade inflation was influenced by the cognitive effort of users. Due to users only wanting to leave a really good or really bad review, normally a good review, we wanted to help the user think about their star ranking decision. To do this we gave each star a meaning. We gave the users just two decisions to help the user decide. 

We found that the binary visual system took the least amount of cognitive thought, yet was still visually appealing. We decided that we wanted to rate each star by a thumbs up or thumbs down process. We also needed to look at the wording of our review. To help our choice of architecture we chose the statement  “Help  your driver.” It used the info received from our polls to select and order topics the user wanted to review. The order also supported choice architecture as the user is likely seeing something on their mind at the top. Due to users only liking to leave a very good or very bad review, two options are given.

first1.png
Chosen Design Solution Sketch
Five Stars
Thumb Up or Down Rating Topics
unnamed.png
unnamed (1).png
Other Design Solution Sketches

Testing for Usability

We conducted a usability test with random participants. We simply showed the user our initial sketches and informed them to rate their driver. This opened us up to key issues with our design.

Findings

  • Every user wanted to interact with the stars upon first contact by tapping them. 

  • Users quickly learned the concept.

  • A user pointed out the lack of interface to view rating results 

  • Another user wanted to leave more info

Solutions for Usability

  • Hide stars from the user until a topic has been rated

  • Add option to leave a comment

  • Add an interface to see user feedback

Solution

Final
Design. 

A revamp of the five-star rating system. Each star represents a different category, the star is given via thumbs up or thumbs down. The user will also have an optional option to add comments on the experience. 

 

Example: 

Star #1 - Cleanliness - Thumbs up or Down

Star #2 - Politeness - Thumbs up or Down

Star #3 - Driving - Thumbs up or Down

Star #4 - Respectfulness - Thumbs up or Down

Star #5 - Above and Beyond - Thumbs up or Down

Frame 15.png
Final Rating Design Sketch
Frame 16.png
Final Ratings Overview Sketch
Post Ride Interface
13 Pro - 1.png
Passenger Rates Ride
13 Pro - 2.png
Ratings Overview
13 Pro - 3.png
Comment Expanded
13 Pro - 4.png
Bottom

Why it Works

This solution takes the five-star system and does half the thinking for the user. It takes the user's low cognitive effort to rate their driver. The user is appointed with two options for each category, and each star has a category attached. The rating system is displayed on one single screen. This creates an ease of use for the user. The driver/passenger can be rated in several different categories. This system lets the user only give a five star unless they really deserve it, easing grade inflation. The system is visually satisfying as based on our research the five-star system is the most visually appealing rating system. 


Users do not lack constraint on what they can say, as each category is the most popular issue with drivers/passengers. The user also has the optional ability to leave additional comments regarding their experience. These comments will be reviewed by Lyft and accessible to the driver/passenger.

bottom of page