For this project, our client was a food delivery company with strong customer acquisition but low long-term retention. The goal was to identify barriers to customer loyalty and develop strategies to increase repeat usage.
I designed a customer survey to analyze user behavior, preferences, and reasons for switching between delivery apps. Survey results showed that users were highly price-sensitive and frequently switched platforms to take advantage of promotions and discounts.
Based on these insights, we proposed a loyalty rewards program and I designed an A/B test to evaluate its effectiveness. Users were segmented into single-app users and comparison shoppers, and randomly assigned to either a control group or a treatment group with access to the loyalty program.
The experiment demonstrated that users exposed to the rewards program showed increased app usage, higher order volume, and greater overall spending. The effect was particularly strong among comparison shoppers.
These findings informed a recommendation to implement a loyalty program and to target incentive-based strategies toward price-sensitive users to improve retention.
Select Survey or A/B Test to see the final results. Additionally, you can select Overview to see how these two methods worked together to inform decision making
Survey
Overview: The survey will be emailed to current users of our app. Users will receive a reminder email two to three times unless they complete or decline the survey. Once the user has completed the survey, they will stop receiving emails to take the survey and will be barred from completing the survey multiple times. The survey consists of eight questions and will take users five to ten minutes to complete. The survey will only be available for a month. Those that complete the survey will receive a 15% discount on their next order. The discount is both an incentive to increase engagement and to repay them for their time and insight.
The survey questions:
How frequently do you use our app?
- Never
- Once a month
- 2–3 times a month
- Once a week
- Multiple times a week
How frequently do you use other food delivery apps?
- Never
- Once a month
- 2–3 times a month
- Once a week
- Multiple times a week
How many food delivery apps do you use on a weekly basis?
- None
- 1
- 2-3
- 4-5
- 6+
Do you typically use our app for takeout, delivery, or both?
- Takeout
- Delivery
- Both
How likely are you to use our app as your primary delivery service?
- Very likely
- Somewhat likely
- Neutral
- Somewhat unlikely
- Very unlikely
How satisfied are you with the speed of our deliveries?
- Very satisfied
- Somewhat satisfied
- Neutral
- Somewhat unsatisfied
- Very unsatisfied
How satisfied are you with the quality of your order?
- Very satisfied
- Somewhat satisfied
- Neutral
- Somewhat unsatisfied
- Very unsatisfied
How satisfied are you with the price of our service?
- Very satisfied
- Somewhat satisfied
- Neutral
- Somewhat unsatisfied
- Very unsatisfied
How would a rewards program affect your likelihood of using our app?
- Much more likely
- Somewhat more likely
- No change
- Somewhat less likely
- Much less likely
What are the two most important aspects of picking a delivery app?
- Lower prices or better deals
- Faster delivery times
- Better restaurant selection
- Promotions or discounts
- Previous negative experience
- Loyalty/rewards programs on other apps
- Recommendations or reviews
- Other: ______
What are the main reasons you choose another delivery app instead of ours? (Select all that apply)
- Lower prices or better deals
- Faster delivery times
- Better restaurant selection
- Promotions or discounts
- Previous negative experience
- Loyalty/rewards programs on other apps
- Recommendations or reviews
- Other: ______
A/B Test
Objective: Based on survey findings, users frequently prioritize price and promotions when choosing between delivery apps. To address this barrier to loyalty, we will test whether introducing a loyalty points program improves customer retention and purchasing behavior. Survey responses also suggest that users fall into two usage patterns: those who primarily rely on a single delivery app and those who frequently compare multiple apps before purchasing. The experiment will evaluate the effectiveness of the loyalty program across these two customer segments.
Hypothesis: A points program will encourage customer loyalty and increased purchases.
Group conditions: The A/B Test will use a segmented approach. Participants will be classified and placed into a group based on their shopping habits. Group 1 will consist of customers who state they prefer to use a single app for purchases. Group 2 will consist of people who selfidentify as comparison shoppers.
The Test: Half of the individuals in both Group 1 and Group 2 will be randomly assigned into either Test A, or Test B. In Test A, participants will be shown the app, its features, the new loyalty points program, how it works, and the different daily and weekly specials. Those in Test B will be shown the app, its features, and different daily and weekly specials. Over the course of one month, the app will track usage frequency, number of deliveries, and spending per user.
Metrics: The effectiveness of the loyalty program will be evaluated using three metrics:
frequency of app usage, number of deliveries per user, and total spending per user. These metrics will be compared between Test A and Test B across both shopper groups.
If users in Test A demonstrate higher usage frequency, delivery volume, and spending than users in Test B, the loyalty program will be considered successful. Differences between shopper segments may also indicate whether the program should be rolled out broadly or targeted to specific customer groups.
Overview
To understand our customer base and to make decisions based on how to improve customer loyalty and retention for the app we recommend administering a customer survey and conducting an A/B test.
The survey will be administered to current app users, and users who participate will receive a reward of 15% off their next order to increase participation. The survey will run for one month, and data provided by our users will be analyzed to find areas where we can improve the app, and increase our customer retention.
The survey data will be analyzed using descriptive statistics, including frequency distributions and percentages to summarize user behaviors and preferences. For satisfaction-related questions, responses will be converted to numerical values and average satisfaction scores will be computed. We will also use cross-tabulation to compare responses across user segments, such as single-app users and comparison shoppers, to identify differences in loyalty drivers. The goal of this step is to determine which factors most strongly influence loyalty and retention. Based on early survey findings, a rewards program emerged as a potential strategy to encourage repeat purchases.
To evaluate the effectiveness of the proposed loyalty program, an A/B test was designed.
The test will consist of two customer segments: single-app users and comparison shoppers. Within each segment, participants were randomly assigned to either Test A, the app with loyalty program; or Test B, the standard app without the loyalty program.
Over the course of one month, the app will track three key metrics:
- Frequency of app usage
- Number of deliveries per user
- Total sales generated per use
For the A/B test, we will compute the mean (average) usage frequency, number of deliveries, and total spending per user for both Test A and Test B. We will then compare the differences in these averages to evaluate the impact of the loyalty program. These comparisons will also be conducted within each customer segment to determine whether the program is more effective for specific types of users.
The results of this analysis will inform strategic decisions to improve customer retention. If the loyalty program significantly increases usage and spending, the company can implement the program more broadly across the platform.
If the program proves particularly effective for comparison shoppers, the company may choose to target this group through personalized promotions or marketing campaigns. Alternatively, if the program does not significantly improve engagement, the company may explore other retention strategies such as improved delivery speed or expanded promotional offers.
Olivia Drexler 