This is a semester project I did in Georgia Tech. We were aiming to create better shopping experience for international
students who are new to the US. Based on user research, we identified their problems in item names translation,
navigation and selecting from unfamiliar brands. We converged 40+ ideas into the final one, Grocery HelpAR, which provides
automatic translation, in-store navigation, brand recommendation and item recognition. It won the 2nd place in the
Convergence Innovation Competition held by
Georgia Tech Research Network Operations Center and the Institute for People and Technology.
To get a quick understanding of how our design ideas solved their problems, please watch the video ↓↓↓
We did the user research and brainstormed ideas all together. My main contribution to the final ideas was about how our design solution seamlessly
fit into customer's journey. Then I created the wireframes, drew storyboard and took charge of the UI design.
I decided most of the interface design details and modified it based on my teammates' feedback. I was also the leading role in
the evaluation part and made recommendations of what needs to be changed.
Paper&pen, Balsamiq, Sketch, Photoshop, inVision
Grocery shopping is an usual activity most people will do weekly. But with dozens of shelves and thousands of items
in one store, it is not easy to locate one item very quickly. For international students who are new to the U.S, there could
be more problems because of the language and cultural bias. Right now when people have troubles in grocery shopping,
the only way they can do is to ask shop assistants for help, but usually there are not many can be found.
Alleviating and solving the grocery shopping problems for international students could significantly save their time
and help them get accustomed more quickly to the life in the U.S. Moreover, it could provide insights and be adapted to
solve certain grocery shopping problems for native shoppers.
Our target users are the international students who are new to the U.S.
Their characteristics are:
- Not familiar with types or names of items.
- Usually make their own meals.
- Need to manage time wisely because they are busy with school.
We did hierarchical task analysis to break down grocery shopping into steps, which helped us prepare for user research.
For user research, we used contextual inquiry and semi-structured interview to dive deeper into the problems
the international students have in grocery shopping as well as their shopping habits.
We conducted 8 semi-structured interviews in total, 6 with international students who are new to the U.S and
2 with who have been living in the U.S for more than a year.
By asking the international students about their overall experience with grocery shopping in the U.S, we
aimed to discover the current problems and hardships that they face in addition to positive (or surprising) experiences.
We conducted 3 contextual inquiries on three international students who are new to the U.S. We did this at Publix
near Georgia Tech. The data was collected in forms of notes, photos, videos.
By getting users do the task in the real environment, we were able to find the problems that may not occur to
participants in the interviews. We were also able to gather the their thoughts and feelings in the whole process
with think-aloud protocol, which helps us to identify their behavioral patterns and get deeper understanding of
challenges that they face.
We collected qualitative data from our semi-structured interviews and contextual inquiries. We wrote each finding in
stickers and performed affinity mapping.
We first found the behavioral patterns of international students
at Georgia Tech while shopping for groceries. They often have to manage their time wisely and spend
least time as possible at the store, although some of them may enjoy hanging out at the store if time allows. Most of them
would make a list before going to the store, physically (on apps or paper) or mentally. They seldom ask the shop assistants
for help because of language bias as well as the fact that there are not many assistants around the store. Many of the them
use dictionary app or search engines to translate the item names they are looking for. Lastly, we found out that since
many of them are not familiar with U.S brands which are different from their own countries, they heavily rely on friends’ recommendations
to select the right items for their taste and their traditional dishes.
In short, user's goal is to do grocery shopping as quickly as possible. Their key problems are:
Don't know the authentic names of items.
Have a hard time locating items (in section as well as on shelf).
Not familiar with the brands of items.
Based on the behavioral patterns found in our research, we were able to create two personas to represent our target user,
Xiaowei as the primary persona and Kuma as the secondary one.
With our personas, we did brainstorming to come up with creative solutions. We brought about 40+ ideas.
To converge the ideas, we used the creativity-feasibility map to place them on the board and sort them out.
We ended up choosing the ideas that fell into most creative and most feasible category. Then we discarded
the ideas that were neither creative nor feasible and voted on the rest ideas on a scale of 1 to 3. After that
we took these ideas and sorted them out into 3 main categories:
- AR/VR based
- In-store Assistant
In each category, there were some ideas overlapping or very similar, so we merged such ideas into single one
and significantly reduced the number of ideas. Then we decided to develop one prototype for each of the category.
We chose the ideas that had the most votes and developed them by combining with the other good ideas in the same category.
We developed the 3 ideas and created low-fi prototypes for each of them.
AR Shopping Aid
This idea is to make a mobile application that provides users with necessary tools for shopping
(i.e. shopping list, translator, store map, suggestions on brand) and to add AR option to provide
faster solution when choosing items from the shelf by overlaying information over product.
Smart Shopping Cart
The idea is to add a smart device with a relatively big screen like an iPad to the shopping cart so
the hands of users can be liberated from holding the device. The smart device is able to provide diverse
services including translation of item names, way finding and selecting brands.
Recipes and Shopping list
This idea is a mobile application that integrates a cookbook for traditional dishes and a shopping list
based on the fact that our target users prefer to cook their own traditional dishes. This app allows users
to add ingredients to their shopping list directly from the recipes.
Then we had "sprint" session to get feedback from our classmates. Since about three quarters of our classmates
are international students, most of them are our target users, therefore their feedback also reflect our users' attitudes
towards the design solutions.
Most of the classmates like the idea of AR shopping aid, although some doubted that if it was feasible to get all
the information of items. One brought about that international students may be shy to scan every item labels on the shelf.
For the smart shopping cart, the biggest concern was the safety problem of the smart devices on shopping carts,
which can be difficult to solve in the real situation. Besides that, classmates liked this idea.
Recipes and shopping lists was quite acceptable to all the classmates. A lot suggested that it can be incorporated
into the former two ideas. Some of them also mentioned that some users might decide what they cook based on the ingredients
that they already have. If the list include the ingredients the users have in their refrigerator, the solution will be able
to provide more options to start working on.
Based on the feedback from the sprint session, we extracted the good features of the 3 design ideas, like the AR scan,
recommendation and recipes, then incorporated them into our final design solution, Grocery HelpAR. It's a mobile application.
The main features are:
- Translation: Automatically translates user's shopping list into English when user makes a list.
- Recipe: Search for recipes and import ingredients to the list.
- In-store navigation: Search for recipes and import ingredients to the list.
- Recommendation: Recommend products based on user's ethnic group.
- Accurate location: Show the exact location of one product on the shelf.
- AR features: Translate the information and recommendation when user scan a product.
Demo of AR Scan
We combined A/B testing with usability test to evaluate our prototype. From the A/B testing results we can conclude that, overall the design
was a success. As we recorded time for individual tasks it proves that using our app, time spend on grocery shopping is cut almost by half.
Even though recipe was not one of the main features from user research but rather suggestion for time management,
it over-performed all the other features, even with the international student who have been in the US for more than a year.
Indoor navigation received great feedback from all users regardless of how much they were familiar with the store layout.
Recommendation (needs some improvement with “filter by”) and AR function also performed very well.
What didn't work
To our biggest surprise, translation was not highly rated, even though it came out of initial research
as one of the biggest problems.
List creation did not either performed as expected.
Due to some design issues with UI, almost all participants have missed the function Location on shelf, which was one of the
important features for time saving. However, once users were introduced to it, they gave very positive feedback.
Based on the evaluation results, we were able to identify the improvements that should be made to the current design and prototype.
Priorities were determined by how frequent the problem occurred, how many participants talked about it and how much it affected
the use of the app.
High Priority ★★★★★
Right now the "list" and "map" tabs seem detached. It is not clear that you can apply the shopping list you made to
the store map. We need to add button in the ‘shopping list’ tab that directly leads to the ‘map’. Also in the ‘map’ tab,
we need to enable users edit the shopping list.
Pop up "Location" button
Some participants didn’t notice this button, others said they didn't expect this ‘location’ to be
the location on the shelf. So we need to make the function of this button clearer.
Medium Priority ★★★
Feedbacks after adding items
We need to give feedback that the items that user selects have been successfully added to the
list after they made that action.
Make the cards seem more clickable
Many participants didn’t know the cards of product are clickable on the recommendation page.
So we should give more indicators to show that, like a grey right arrow on the right side of the card.
Optionally show the fastest route
There is lack of flexibility if we force the user to follow the shortest route. So we’d like to
change to only show the locations of the items on the map, and add a button of ‘shortest route’. Users can choose if to show the route.
Automatic rotation of map
Some participants rotated the phone to read the map. To make this map easier to use, it should be
rotated automatically to make the current path straight forward (like Google map).
Add text to the tab icons
Icons standing only do not speak well to users although we have changed it after pilot testing.
So it is necessary that we add text below the icon to describe the main functions of the tabs.
Low Priority ★
Number of reviews
Right now the app doesn’t show how many people rated the product so the rating numbers are not
very reliable to some users. We would add numbers of reviews right next to the ratings.
Labels for products
The descriptions in the detailed information page of a product like ingredients are hard to understand
for some users. It would be helpful if there are labels that summarize the information.
Move the info of next stop
Currently the information of next item blocks the aisle numbers on the map. We would move this information
out of the map (on the head or below).
Show image after translation
To increase users’ sense of security that they have added the right thing, we would also show the image of
the typed item along with translation.
Change button text
Wording on this button is unclear. “Got it” gave users impression
that they were confirming buying an item instead of checking it off the list.