This is a semester project I did in Georgia Tech. We were aiming to create better shopping experience for international
students who are new to the US. Based on user research, we identified their problems in item names translation,
navigation and selecting from unfamiliar brands. We converged 40+ ideas into the final one, Grocery HelpAR, which provides
automatic translation, in-store navigation, brand recommendation and item recognition. It won the 2nd place in the
Convergence Innovation Competition held by
Georgia Tech Research Network Operations Center and the Institute for People and Technology.
To get a quick understanding of how our design ideas solved their problems, please watch the video ↓↓↓
We did the user research and brainstormed ideas all together. My main contribution to the final ideas was about how our design solution seamlessly
fit into customer's journey. Then I created the wireframes, drew storyboard and took charge of the UI design.
I decided most of the interface design details and modified it based on my teammates' feedback. I was also the leading role in
the evaluation part and made recommendations of what needs to be changed.
Paper&pen, Balsamiq, Sketch, Photoshop, inVision
Grocery shopping is an usual activity most people will do weekly. But with dozens of shelves and thousands of items
in one store, it is not easy to locate one item very quickly. For international students who are new to the U.S, there could
be more problems because of the language and cultural bias. Right now when people have troubles in grocery shopping,
the only way they can do is to ask shop assistants for help, but usually there are not many can be found.
Alleviating and solving the grocery shopping problems for international students could significantly save their time
and help them get accustomed more quickly to the life in the U.S. Moreover, it could provide insights and be adapted to
solve certain grocery shopping problems for native shoppers.
Our target users are the international students who are new to the U.S.
Their characteristics are:
- Not familiar with types or names of items.
- Usually make their own meals.
- Need to manage time wisely because they are busy with school.
We did hierarchical task analysis to break down grocery shopping into steps, which helped us prepare for user research.
For user research, we used contextual inquiry and semi-structured interview to dive deeper into the problems
the international students have in grocery shopping as well as their shopping habits.
We conducted 8 semi-structured interviews in total, 6 with international students who are new to the U.S and
2 with who have been living in the U.S for more than a year.
By asking the international students about their overall experience with grocery shopping in the U.S, we
aimed to discover the current problems and hardships that they face in addition to positive (or surprising) experiences.
We conducted 3 contextual inquiries on three international students who are new to the U.S. We did this at Publix
near Georgia Tech. The data was collected in forms of notes, photos, videos.
By getting users do the task in the real environment, we were able to find the problems that may not occur to
participants in the interviews. We were also able to gather the their thoughts and feelings in the whole process
with think-aloud protocol, which helps us to identify their behavioral patterns and get deeper understanding of
challenges that they face.
We collected qualitative data from our semi-structured interviews and contextual inquiries. We wrote each finding in
stickers and performed affinity mapping.
We first found the behavioral patterns of international students
at Georgia Tech while shopping for groceries. They often have to manage their time wisely and spend
least time as possible at the store, although some of them may enjoy hanging out at the store if time allows. Most of them
would make a list before going to the store, physically (on apps or paper) or mentally. They seldom ask the shop assistants
for help because of language bias as well as the fact that there are not many assistants around the store. Many of the them
use dictionary app or search engines to translate the item names they are looking for. Lastly, we found out that since
many of them are not familiar with U.S brands which are different from their own countries, they heavily rely on friends’ recommendations
to select the right items for their taste and their traditional dishes.
In short, user's goal is to do grocery shopping as quickly as possible. Their key problems are:
Don't know the authentic names of items.
Have a hard time locating items (in section as well as on shelf).
Not familiar with the brands of items.
Based on the behavioral patterns found in our research, we were able to create two personas to represent our target user,
Xiaowei as the primary persona and Kuma as the secondary one.
With our personas, we did brainstorming to come up with creative solutions. We brought about 40+ ideas.
To converge the ideas, we used the creativity-feasibility map to place them on the board and sort them out.
We ended up choosing the ideas that fell into most creative and most feasible category. Then we discarded
the ideas that were neither creative nor feasible and voted on the rest ideas on a scale of 1 to 3. After that
we took these ideas and sorted them out into 3 main categories:
- AR/VR based
- In-store Assistant
In each category, there were some ideas overlapping or very similar, so we merged such ideas into single one
and significantly reduced the number of ideas. Then we decided to develop one prototype for each of the category.
We chose the ideas that had the most votes and developed them by combining with the other good ideas in the same category.
We developed the 3 ideas and created low-fi prototypes for each of them.
AR Shopping Aid
This idea is to make a mobile application that provides users with necessary tools for shopping
(i.e. shopping list, translator, store map, suggestions on brand) and to add AR option to provide
faster solution when choosing items from the shelf by overlaying information over product.
Smart Shopping Cart
The idea is to add a smart device with a relatively big screen like an iPad to the shopping cart so
the hands of users can be liberated from holding the device. The smart device is able to provide diverse
services including translation of item names, way finding and selecting brands.
Recipes and Shopping list
This idea is a mobile application that integrates a cookbook for traditional dishes and a shopping list
based on the fact that our target users prefer to cook their own traditional dishes. This app allows users
to add ingredients to their shopping list directly from the recipes.
Then we had "sprint" session to get feedback from our classmates. Since about three quarters of our classmates
are international students, most of them are our target users, therefore their feedback also reflect our users' attitudes
towards the design solutions.
Most of the classmates like the idea of AR shopping aid, although some doubted that if it was feasible to get all
the information of items. One brought about that international students may be shy to scan every item labels on the shelf.
For the smart shopping cart, the biggest concern was the safety problem of the smart devices on shopping carts,
which can be difficult to solve in the real situation. Besides that, classmates liked this idea.
Recipes and shopping lists was quite acceptable to all the classmates. A lot suggested that it can be incorporated
into the former two ideas. Some of them also mentioned that some users might decide what they cook based on the ingredients
that they already have. If the list include the ingredients the users have in their refrigerator, the solution will be able
to provide more options to start working on.
Based on the feedback from the sprint session, we extracted the good features of the 3 design ideas, like the AR scan,
recommendation and recipes, then incorporated them into our final design solution, Grocery HelpAR. It's a mobile application.
The main features are:
- Translation: Automatically translates user's shopping list into English when user makes a list.
- Recipe: Search for recipes and import ingredients to the list.
- In-store navigation: Search for recipes and import ingredients to the list.
- Recommendation: Recommend products based on user's ethnic group.
- Accurate location: Show the exact location of one product on the shelf.
- AR features: Translate the information and recommendation when user scan a product.
Demo of AR Scan
We combined A/B testing with usability test to evaluate our prototype. From the A/B testing results we can conclude that, overall the design
was a success. As we recorded time for individual tasks it proves that using our app, time spend on grocery shopping is cut almost by half.
Even though recipe was not one of the main features from user research but rather suggestion for time management,
it over-performed all the other features, even with the international student who have been in the US for more than a year.
Indoor navigation received great feedback from all users regardless of how much they were familiar with the store layout.
Recommendation (needs some improvement with “filter by”) and AR function also performed very well.
What didn't work
To our biggest surprise, translation was not highly rated, even though it came out of initial research
as one of the biggest problems.
List creation did not either performed as expected.
Due to some design issues with UI, almost all participants have missed the function Location on shelf, which was one of the
important features for time saving. However, once users were introduced to it, they gave very positive feedback.
Based on the evaluation results, we were able to identify some of the usability issues and iterate the design.
Link "Shopping list" and "Navigation" tab
Right now the "list" and "nav" tabs seem detached. It is not clear that you can apply the shopping list you have made to
the store map. After iteration, there will be a message on the shopping list page
when the app detects that the user is at a grocery store. Users can directly apply the list to the store map.
Give feedback after ingredients are added from recipes
After users import ingredients to the shopping list from the recipes, there is no feedback confirming with users
that items are successfully added and they need to go to shopping list tab to check. After iteration, there will
be a message telling users that their action is successful and a shortcut that lets users directly go to shopping
list. If they haven't viewed their updated shopping list, there will be a green dot on the icon in the tab bar
indicating the number of newly added items.
Add text to the icons in the tab bar
Icons standing only do not speak well to users.
It is necessary that add text below the icon to describe the main function of the tabs.
Show image of the item after its name is translated
We also want to show the image of the item after its name is translated so that users can confirm that
it's the right item.
Show number of reviews
The rating number is not reliable to users without the number of reviews. So we need to show it right next to the ratings.
Be careful with assumptions before research.
The most important lesson I learned from this project is that we should be careful with assumptions we
made before user research. I think we overstated the importance of the problem that users don’t know
the English names of items because we assumed this problem before user research. Sometimes assumptions
can help us find problems more quickly. But they may also bias us during the research process.
We may just try to validate our assumptions on users instead of trying to understand our users and
listen to what they say about their problems.
Where can the data come from?
The most concerns about our final solution are about where the data comes from, like the store map,
reviews of product, etc. As the warehouses are becoming smarter and smarter, we feel this is the
same trend with grocery stores. In the near future, grocery stores may just use computer and robots to
restock. At that time, information like store maps will be easy to access. We also think we may start
from one kind of grocery store. A deep cooperation also makes it possible that users can just check out
on their mobile phones and leave reviews of products. Just like Amazon Go.