Grocery HelpAR

A mobile application aiming to help international students do grocery shopping


Course Project


Sep - Dec 2016


Geunbae Lee
Dong Whi Yoo
Miroslav Malesevic


UX Design
Mobile App


This is a semester project I did in Georgia Tech. We were aiming to create better shopping experience for international students who are new to the US. Based on user research, we identified their problems in item names translation, navigation and selecting from unfamiliar brands. We converged 40+ ideas into the final one, Grocery HelpAR, which provides automatic translation, in-store navigation, brand recommendation and item recognition. It won the 2nd place in the Convergence Innovation Competition held by Georgia Tech Research Network Operations Center and the Institute for People and Technology.

To get a quick understanding of how our design ideas solved their problems, please watch the video ↓↓↓

My role

We did the user research and brainstormed ideas all together. My main contribution to the final ideas was about how our design solution seamlessly fit into customer's journey. Then I created the wireframes, drew storyboard and took charge of the UI design. I decided most of the interface design details and modified it based on my teammates' feedback. I was also the leading role in the evaluation part and made recommendations of what needs to be changed.



Paper&pen, Balsamiq, Sketch, Photoshop, inVision

Problem Space

Grocery shopping is an usual activity most people will do weekly. But with dozens of shelves and thousands of items in one store, it is not easy to locate one item very quickly. For international students who are new to the U.S, there could be more problems because of the language and cultural bias. Right now when people have troubles in grocery shopping, the only way they can do is to ask shop assistants for help, but usually there are not many can be found.

Alleviating and solving the grocery shopping problems for international students could significantly save their time and help them get accustomed more quickly to the life in the U.S. Moreover, it could provide insights and be adapted to solve certain grocery shopping problems for native shoppers.

User Group

Our target users are the international students who are new to the U.S. Their characteristics are:

- Not familiar with types or names of items.

- Usually make their own meals.

- Need to manage time wisely because they are busy with school.

Task Analysis

We did hierarchical task analysis to break down grocery shopping into steps, which helped us prepare for user research.

User Research

For user research, we used contextual inquiry and semi-structured interview to dive deeper into the problems the international students have in grocery shopping as well as their shopping habits.

Semi-structured interviews

We conducted 8 semi-structured interviews in total, 6 with international students who are new to the U.S and 2 with who have been living in the U.S for more than a year.

By asking the international students about their overall experience with grocery shopping in the U.S, we aimed to discover the current problems and hardships that they face in addition to positive (or surprising) experiences.

Contextual inquiry

We conducted 3 contextual inquiries on three international students who are new to the U.S. We did this at Publix near Georgia Tech. The data was collected in forms of notes, photos, videos.

By getting users do the task in the real environment, we were able to find the problems that may not occur to participants in the interviews. We were also able to gather the their thoughts and feelings in the whole process with think-aloud protocol, which helps us to identify their behavioral patterns and get deeper understanding of challenges that they face.

Data Analysis

We collected qualitative data from our semi-structured interviews and contextual inquiries. We wrote each finding in stickers and performed affinity mapping.

We first found the behavioral patterns of international students at Georgia Tech while shopping for groceries. They often have to manage their time wisely and spend least time as possible at the store, although some of them may enjoy hanging out at the store if time allows. Most of them would make a list before going to the store, physically (on apps or paper) or mentally. They seldom ask the shop assistants for help because of language bias as well as the fact that there are not many assistants around the store. Many of the them use dictionary app or search engines to translate the item names they are looking for. Lastly, we found out that since many of them are not familiar with U.S brands which are different from their own countries, they heavily rely on friends’ recommendations to select the right items for their taste and their traditional dishes.

In short, user's goal is to do grocery shopping as quickly as possible. Their key problems are:

Don't know the authentic names of items.

Have a hard time locating items (in section as well as on shelf).

Not familiar with the brands of items.


Based on the behavioral patterns found in our research, we were able to create two personas to represent our target user, Xiaowei as the primary persona and Kuma as the secondary one.



With our personas, we did brainstorming to come up with creative solutions. We brought about 40+ ideas.

To converge the ideas, we used the creativity-feasibility map to place them on the board and sort them out. We ended up choosing the ideas that fell into most creative and most feasible category. Then we discarded the ideas that were neither creative nor feasible and voted on the rest ideas on a scale of 1 to 3. After that we took these ideas and sorted them out into 3 main categories:

- AR/VR based

- In-store Assistant

- Mobile/Web

In each category, there were some ideas overlapping or very similar, so we merged such ideas into single one and significantly reduced the number of ideas. Then we decided to develop one prototype for each of the category. We chose the ideas that had the most votes and developed them by combining with the other good ideas in the same category.

3 ideas

We developed the 3 ideas and created low-fi prototypes for each of them.

AR Shopping Aid

This idea is to make a mobile application that provides users with necessary tools for shopping (i.e. shopping list, translator, store map, suggestions on brand) and to add AR option to provide faster solution when choosing items from the shelf by overlaying information over product.

Smart Shopping Cart

The idea is to add a smart device with a relatively big screen like an iPad to the shopping cart so the hands of users can be liberated from holding the device. The smart device is able to provide diverse services including translation of item names, way finding and selecting brands.

Recipes and Shopping list

This idea is a mobile application that integrates a cookbook for traditional dishes and a shopping list based on the fact that our target users prefer to cook their own traditional dishes. This app allows users to add ingredients to their shopping list directly from the recipes.


Then we had "sprint" session to get feedback from our classmates. Since about three quarters of our classmates are international students, most of them are our target users, therefore their feedback also reflect our users' attitudes towards the design solutions.

Most of the classmates like the idea of AR shopping aid, although some doubted that if it was feasible to get all the information of items. One brought about that international students may be shy to scan every item labels on the shelf.

For the smart shopping cart, the biggest concern was the safety problem of the smart devices on shopping carts, which can be difficult to solve in the real situation. Besides that, classmates liked this idea.

sprint results

Recipes and shopping lists was quite acceptable to all the classmates. A lot suggested that it can be incorporated into the former two ideas. Some of them also mentioned that some users might decide what they cook based on the ingredients that they already have. If the list include the ingredients the users have in their refrigerator, the solution will be able to provide more options to start working on.

Final Idea

Based on the feedback from the sprint session, we extracted the good features of the 3 design ideas, like the AR scan, recommendation and recipes, then incorporated them into our final design solution, Grocery HelpAR. It's a mobile application. The main features are:

- Translation: Automatically translates user's shopping list into English when user makes a list.

- Recipe: Search for recipes and import ingredients to the list.

- In-store navigation: Search for recipes and import ingredients to the list.

- Recommendation: Recommend products based on user's ethnic group.

- Accurate location: Show the exact location of one product on the shelf.

- AR features: Translate the information and recommendation when user scan a product.



Lo-fi Prototype

Usage Scenario

Interactive Prototype

Demo of AR Scan


We combined A/B testing with usability test to evaluate our prototype. From the A/B testing results we can conclude that, overall the design was a success. As we recorded time for individual tasks it proves that using our app, time spend on grocery shopping is cut almost by half.

What worked

Even though recipe was not one of the main features from user research but rather suggestion for time management, it over-performed all the other features, even with the international student who have been in the US for more than a year.

Indoor navigation received great feedback from all users regardless of how much they were familiar with the store layout. Recommendation (needs some improvement with “filter by”) and AR function also performed very well.

What didn't work

To our biggest surprise, translation was not highly rated, even though it came out of initial research as one of the biggest problems.

List creation did not either performed as expected.

Due to some design issues with UI, almost all participants have missed the function Location on shelf, which was one of the important features for time saving. However, once users were introduced to it, they gave very positive feedback.


Based on the evaluation results, we were able to identify some of the usability issues and iterate the design.

Link "Shopping list" and "Navigation" tab

Right now the "list" and "nav" tabs seem detached. It is not clear that you can apply the shopping list you have made to the store map. After iteration, there will be a message on the shopping list page when the app detects that the user is at a grocery store. Users can directly apply the list to the store map.

Give feedback after ingredients are added from recipes

After users import ingredients to the shopping list from the recipes, there is no feedback confirming with users that items are successfully added and they need to go to shopping list tab to check. After iteration, there will be a message telling users that their action is successful and a shortcut that lets users directly go to shopping list. If they haven't viewed their updated shopping list, there will be a green dot on the icon in the tab bar indicating the number of newly added items.

Add text to the icons in the tab bar

Icons standing only do not speak well to users. It is necessary that add text below the icon to describe the main function of the tabs.

Show image of the item after its name is translated

We also want to show the image of the item after its name is translated so that users can confirm that it's the right item.

Show number of reviews

The rating number is not reliable to users without the number of reviews. So we need to show it right next to the ratings.


Be careful with assumptions before research.

The most important lesson I learned from this project is that we should be careful with assumptions we made before user research. I think we overstated the importance of the problem that users don’t know the English names of items because we assumed this problem before user research. Sometimes assumptions can help us find problems more quickly. But they may also bias us during the research process. We may just try to validate our assumptions on users instead of trying to understand our users and listen to what they say about their problems.

Where can the data come from?

The most concerns about our final solution are about where the data comes from, like the store map, reviews of product, etc. As the warehouses are becoming smarter and smarter, we feel this is the same trend with grocery stores. In the near future, grocery stores may just use computer and robots to restock. At that time, information like store maps will be easy to access. We also think we may start from one kind of grocery store. A deep cooperation also makes it possible that users can just check out on their mobile phones and leave reviews of products. Just like Amazon Go.

Workato Redesign

A new experience for Workato users to create a "recipe"


A mutual aid community that helps people endure earthquake