On the 9th of June the Help The Penguin toy was part of the one day UQ Interaction Design Exhibit, hosted at The Edge, in Brisbane, Australia. During this exhibiton more than 50 users played with our toy, and we were quick to collect responses from them on how they liked using it.
Skip to:
To measure the success of our product, we collected survey answers from people who had used the penguin at the exhibit. We looked at how well we met the success critera we had set for our project earlier during the semester - directly derived from the core values we defined for our product during our concept design phase - that it should be easy to use, mobile and fun. In addition, we included an optional open ended question, where people could give us any feedback they wanted.
Before I show you the results, there is one reservation regarding the results I have to make. We designed the penguin as a toy for children, as ideally children is the user group we should have evaluated it on. However, our evaluation was opportunistic, in that we encouraged anyone at the exhibition that had played with the penguin to take this survey. We had the choice between getting data with possibly questionable validity - or no data at all.
Age of of survey respondants - survey result. Survey powered by Surveymonkey.
'Easy to use' survey result. Survey powered by Surveymonkey.
86% of our users agreed or strongly agreed that it was easy to help the penguin (find his lost things), and only one user, and only one person disagreed.
'Mobility' survey result. Survey powered by Surveymonkey.
'Fun to use' survey result. Survey powered by Surveymonkey.
93.3% of our users agreed of strongly agreed that they found the penguin fun to use, and no one disagreed.
'Other comments' Five representative answers to survey question. Survey powered by Surveymonkey.
A majority of the feedback we recieved from the other comments we got were positive, stating that the users liked using the penguin, thought it was cool, or wanted to play with it again. Some users provided us with ideas on how to improve the toy, most often asking for a different voice, or more functionality. 5 of the 25 people (25%) who answered this question gave us negative feedback. 4 of these users critised the voice recognition, stating that it did not pick up (enough) of what they said. 1 person said the dialouge with the penguin was slow, and 1 person said they disliked the voice being used.
Looking at the results of our survey, my general impression is that we achieved our success criteria. The majority of our users did find the penguin easy to use, mobile, and fun to use. Before I reflect further on the results, I want to state once again, for the record, that these results are however not reliable for our actual user group, as they were provided by adults, and not children.
It is interesting to see that more people complained about accuracy of the voice recognition than did respond that the pengion was hard to use. I would say that the people having problems with the voice recognition had so only because of the really large amount of people standing around our booth during the later parts of the exhibition, and the noise associated with this crowd.
Unfortunately the camera one of the members of the project group had set up for recording a time lapse of the event shut down before this crowd arrived at the scene, but some of the users of the penguin toy can still be seen in the video below.
'Time lapse exibition'. Users using the Help the Penguin toy. Recorded by Khoa Tran
After the exhibition ended, we had a last late user testing the device, and the accuracy of the voice recognition during that test was 100%. That being said, we designed the toy to work in reasonably noisy environments, such as a kindergarten. In conclusion, I would say our test point towards that our device works reasonable well for such environments, as 84% of our users gave no negative feedback regarding the voice recognition, and, as stated above, and a similar percentage reported the penguin being easy to use.
My gut feeling about the higher number (10%) of responses agreeing with that the penguin did limit their ability to move around, the opposite of the result we wanted is similar to for what I said regarding how easy the penguin was to use. Towards the end of the evening the space around the user testing the penguing got completely crowded, and it was impossible for our tester to move around very much without bumping into people. On another note, however, we never prompted our users to move around that much. As can be seen in the video, the users only move around in our test booth, which was no larger than a very small room. Our technology however performs perfectly reliable up to a much larger distance, well over 75 meters (~250 feet). A test that would have made participants realise this might have produced more positive responses.
As a final note regarding the mobility question, this question was our only reverse-phrased question. It is possible that this might have confused some participants and caused them to provide the wrong answer, however. A better design would not have placed the reverese-phrased question last, as we now can't expect this possible effect to equally effect any non-reversed question.
Summarising our evaluation, all we can say is that it points towards that people enjoy using our penguing, find it being a mobile product, and find it easy to use. However these results can not reliably be generalised to children.
