Grocery HelpAR

A mobile application aiming to help international students do grocery shopping


Course Project


Sep - Dec 2016


Geunbae Lee
Dong Whi Yoo
Miroslav Malesevic


UX Design
Mobile App


This is a semester project I did in Georgia Tech. We were aiming to create better shopping experience for international students who are new to the US. Based on user research, we identified their problems in item names translation, navigation and selecting from unfamiliar brands. We converged 40+ ideas into the final one, Grocery HelpAR, which provides automatic translation, in-store navigation, brand recommendation and item recognition. It won the 2nd place in the Convergence Innovation Competition held by Georgia Tech Research Network Operations Center and the Institute for People and Technology.

To get a quick understanding of how our design ideas solved their problems, please watch the video ↓↓↓

My role

We did the user research and brainstormed ideas all together. My main contribution to the final ideas was about how our design solution seamlessly fit into customer's journey. Then I created the wireframes, drew storyboard and took charge of the UI design. I decided most of the interface design details and modified it based on my teammates' feedback. I was also the leading role in the evaluation part and made recommendations of what needs to be changed.



Paper&pen, Balsamiq, Sketch, Photoshop, inVision

Problem Space

Grocery shopping is an usual activity most people will do weekly. But with dozens of shelves and thousands of items in one store, it is not easy to locate one item very quickly. For international students who are new to the U.S, there could be more problems because of the language and cultural bias. Right now when people have troubles in grocery shopping, the only way they can do is to ask shop assistants for help, but usually there are not many can be found.

Alleviating and solving the grocery shopping problems for international students could significantly save their time and help them get accustomed more quickly to the life in the U.S. Moreover, it could provide insights and be adapted to solve certain grocery shopping problems for native shoppers.

User Group

Our target users are the international students who are new to the U.S. Their characteristics are:

- Not familiar with types or names of items.

- Usually make their own meals.

- Need to manage time wisely because they are busy with school.

Task Analysis

We did hierarchical task analysis to break down grocery shopping into steps, which helped us prepare for user research.

User Research

For user research, we used contextual inquiry and semi-structured interview to dive deeper into the problems the international students have in grocery shopping as well as their shopping habits.

Semi-structured interviews

We conducted 8 semi-structured interviews in total, 6 with international students who are new to the U.S and 2 with who have been living in the U.S for more than a year.

By asking the international students about their overall experience with grocery shopping in the U.S, we aimed to discover the current problems and hardships that they face in addition to positive (or surprising) experiences.

Contextual inquiry

We conducted 3 contextual inquiries on three international students who are new to the U.S. We did this at Publix near Georgia Tech. The data was collected in forms of notes, photos, videos.

By getting users do the task in the real environment, we were able to find the problems that may not occur to participants in the interviews. We were also able to gather the their thoughts and feelings in the whole process with think-aloud protocol, which helps us to identify their behavioral patterns and get deeper understanding of challenges that they face.

Data Analysis

We collected qualitative data from our semi-structured interviews and contextual inquiries. We wrote each finding in stickers and performed affinity mapping.

We first found the behavioral patterns of international students at Georgia Tech while shopping for groceries. They often have to manage their time wisely and spend least time as possible at the store, although some of them may enjoy hanging out at the store if time allows. Most of them would make a list before going to the store, physically (on apps or paper) or mentally. They seldom ask the shop assistants for help because of language bias as well as the fact that there are not many assistants around the store. Many of the them use dictionary app or search engines to translate the item names they are looking for. Lastly, we found out that since many of them are not familiar with U.S brands which are different from their own countries, they heavily rely on friends’ recommendations to select the right items for their taste and their traditional dishes.

In short, user's goal is to do grocery shopping as quickly as possible. Their key problems are:

Don't know the authentic names of items.

Have a hard time locating items (in section as well as on shelf).

Not familiar with the brands of items.


Based on the behavioral patterns found in our research, we were able to create two personas to represent our target user, Xiaowei as the primary persona and Kuma as the secondary one.



With our personas, we did brainstorming to come up with creative solutions. We brought about 40+ ideas.

To converge the ideas, we used the creativity-feasibility map to place them on the board and sort them out. We ended up choosing the ideas that fell into most creative and most feasible category. Then we discarded the ideas that were neither creative nor feasible and voted on the rest ideas on a scale of 1 to 3. After that we took these ideas and sorted them out into 3 main categories:

- AR/VR based

- In-store Assistant

- Mobile/Web

In each category, there were some ideas overlapping or very similar, so we merged such ideas into single one and significantly reduced the number of ideas. Then we decided to develop one prototype for each of the category. We chose the ideas that had the most votes and developed them by combining with the other good ideas in the same category.

3 ideas

We developed the 3 ideas and created low-fi prototypes for each of them.

AR Shopping Aid

This idea is to make a mobile application that provides users with necessary tools for shopping (i.e. shopping list, translator, store map, suggestions on brand) and to add AR option to provide faster solution when choosing items from the shelf by overlaying information over product.

Smart Shopping Cart

The idea is to add a smart device with a relatively big screen like an iPad to the shopping cart so the hands of users can be liberated from holding the device. The smart device is able to provide diverse services including translation of item names, way finding and selecting brands.

Recipes and Shopping list

This idea is a mobile application that integrates a cookbook for traditional dishes and a shopping list based on the fact that our target users prefer to cook their own traditional dishes. This app allows users to add ingredients to their shopping list directly from the recipes.


Then we had "sprint" session to get feedback from our classmates. Since about three quarters of our classmates are international students, most of them are our target users, therefore their feedback also reflect our users' attitudes towards the design solutions.

Most of the classmates like the idea of AR shopping aid, although some doubted that if it was feasible to get all the information of items. One brought about that international students may be shy to scan every item labels on the shelf.

For the smart shopping cart, the biggest concern was the safety problem of the smart devices on shopping carts, which can be difficult to solve in the real situation. Besides that, classmates liked this idea.

sprint results

Recipes and shopping lists was quite acceptable to all the classmates. A lot suggested that it can be incorporated into the former two ideas. Some of them also mentioned that some users might decide what they cook based on the ingredients that they already have. If the list include the ingredients the users have in their refrigerator, the solution will be able to provide more options to start working on.

Final Idea

Based on the feedback from the sprint session, we extracted the good features of the 3 design ideas, like the AR scan, recommendation and recipes, then incorporated them into our final design solution, Grocery HelpAR. It's a mobile application. The main features are:

- Translation: Automatically translates user's shopping list into English when user makes a list.

- Recipe: Search for recipes and import ingredients to the list.

- In-store navigation: Search for recipes and import ingredients to the list.

- Recommendation: Recommend products based on user's ethnic group.

- Accurate location: Show the exact location of one product on the shelf.

- AR features: Translate the information and recommendation when user scan a product.



Lo-fi Prototype

Usage Scenario

Interactive Prototype

Demo of AR Scan


We combined A/B testing with usability test to evaluate our prototype. From the A/B testing results we can conclude that, overall the design was a success. As we recorded time for individual tasks it proves that using our app, time spend on grocery shopping is cut almost by half.

What worked

Even though recipe was not one of the main features from user research but rather suggestion for time management, it over-performed all the other features, even with the international student who have been in the US for more than a year.

Indoor navigation received great feedback from all users regardless of how much they were familiar with the store layout. Recommendation (needs some improvement with “filter by”) and AR function also performed very well.

What didn't work

To our biggest surprise, translation was not highly rated, even though it came out of initial research as one of the biggest problems.

List creation did not either performed as expected.

Due to some design issues with UI, almost all participants have missed the function Location on shelf, which was one of the important features for time saving. However, once users were introduced to it, they gave very positive feedback.

Design Suggestions

Based on the evaluation results, we were able to identify the improvements that should be made to the current design and prototype. Priorities were determined by how frequent the problem occurred, how many participants talked about it and how much it affected the use of the app.

High Priority ★★★★★

Clearer flow

Right now the "list" and "map" tabs seem detached. It is not clear that you can apply the shopping list you made to the store map. We need to add button in the ‘shopping list’ tab that directly leads to the ‘map’. Also in the ‘map’ tab, we need to enable users edit the shopping list.

Pop up "Location" button

Some participants didn’t notice this button, others said they didn't expect this ‘location’ to be the location on the shelf. So we need to make the function of this button clearer.

Medium Priority ★★★

Feedbacks after adding items

We need to give feedback that the items that user selects have been successfully added to the list after they made that action.

Make the cards seem more clickable

Many participants didn’t know the cards of product are clickable on the recommendation page. So we should give more indicators to show that, like a grey right arrow on the right side of the card.

Optionally show the fastest route

There is lack of flexibility if we force the user to follow the shortest route. So we’d like to change to only show the locations of the items on the map, and add a button of ‘shortest route’. Users can choose if to show the route.

Automatic rotation of map

Some participants rotated the phone to read the map. To make this map easier to use, it should be rotated automatically to make the current path straight forward (like Google map).

Add text to the tab icons

Icons standing only do not speak well to users although we have changed it after pilot testing. So it is necessary that we add text below the icon to describe the main functions of the tabs.

Low Priority ★

Number of reviews

Right now the app doesn’t show how many people rated the product so the rating numbers are not very reliable to some users. We would add numbers of reviews right next to the ratings.

Labels for products

The descriptions in the detailed information page of a product like ingredients are hard to understand for some users. It would be helpful if there are labels that summarize the information.

Move the info of next stop

Currently the information of next item blocks the aisle numbers on the map. We would move this information out of the map (on the head or below).


Show image after translation

To increase users’ sense of security that they have added the right thing, we would also show the image of the typed item along with translation.

Change button text

Wording on this button is unclear. “Got it” gave users impression that they were confirming buying an item instead of checking it off the list.