Now, the smart mirror got extended by a voice assistant to become a speaking mirror.
The case study
The case study was set to be a teamwork of two or three students so I grouped up with a fellow student (Benjamin) to work on this project.
So the base question was:
How to implement a voice assistant into a smart mirror?
This also leads to some follow up questions:
What voice assistants are available?
How to choose one of them?
How to actually implement it?
Initial situation
First, we had a working smart mirror which showed weather information, time and date as well as the latest news headlines.
It consisted of an old monitor, a wooden frame, a spyglass and a Raspberry Pi. To learn more about that part, check my original post about building a smart mirror.
For our case study, we were describing this parts and some other basics like the internet of things and so on of course.
Different voice assistants
The next big question was, which voice assistants are available and which one to choose for the project.
Voice assistants are one of the main topics in tech at the moment. Even Google showed up at the CES in Las Vegas this year to present different consumer products that include voice assistants. And even before that, at the Google I/O 2016, Sundararajan Pichai, the CEO of Google, was talking about the importance of voice search and assistants for the future of technology in his keynote: "In the US, on our mobile app in Android, one in five queries — 20% of our queries — are voice queries and that share is growing."
Which voice assistants did we look at?
For our case study, we took a look at the tree voice assistants that were the most popular and obvious to us:
- Google Voice Assistant
- Amazon Alexa
- Apple Siri
How to compare the voice assistants?
To compare these assistants we build a list of 46 questions of different kinds which we asked each assistant. The questions where consisting different areas. Following you will find examples of the different categories.
Questions about calculations:
"What is the 3rd root of 625?"
"What is 5 US-Dollar in Euro?"
Questions about common knowledge:
"What is the difference between bit and byte?"
"What is the Brownian motion?"
Follow up questions where the subject of the first question had to be considered for the second question:
"Who is Benedict Cumberbatch?"
"In which movies did he play?"
Questions about the sport and other local events:
"How did Bayern play?"
"What's going on in the cinema today?"
Location-based questions:
"Where is the next cinema?"
"What's the time in Bali?"
"Are there any events on my calendar?"
"How long is my commute?"
Fun questions:
"Who is Cortana?"
"Sing Happy Birthday!"
"Tell me a joke!"
Each question was asked to each of the assistants up to three times to make sure no other noises, like a car driving by, would have a bad influence on the result. If the question was recognized correctly, no further tries where applied.
The questions were also asked by the same person to all assistants to have comparable conditions for each of them.
The devices we used to compare the voice assistants:
- Google voice assistant
at an Android smartphone (OnePlus 3T with Android 7.1.1) - Amazon Alexa
Amazon Echo Dot - Apple Siri
iPad with iOS 9.3.5
Comparing the results
After all the questions have been asked and the results were gathered, we created some graphs to show them in comparison in three different categories.
I hope you don't bother with me because the graphs are labeled in German as our case study was written in German. :)
I hope you don't bother with me because the graphs are labeled in German as our case study was written in German. :)
Question recognized
The first category was, how well the spoken question was recognized by the voice assistants. This could be determined by checking the voice log where the text was shown, that the voice assistant recognized.
 |
| Question recognized |
Question answered
The second category was whether the question was answered. This had to be split up not only into yes and no but also into a third area "partwise". This was introduced as we recognized how some of the questions were answered but not by voice feedback, rather by showing the result at the display.
 |
| Question answered |
This graph shows, how well the questions were answered. Again, Google did a very good job in this regard. Alexa did not show any results visually, because of the nature of the device as it had no display. Siri and Google on the other side where tested on devices with a display. The number of answers given by Siri by displaying the information rather than giving voice feedback was surprisingly high.
Answer helpful
The last category was whether the answer given to the question was fitting and helpful. This was a bit subjective, but we had no disagreement on the judgment here at all.
 |
| Answer helpful |
The quality of the answers given was best by Google, followed by Siri. The last useful answers were given by Alexa.
Conclusion
All in all, we summed up the positive answers of all categories to get a summary of the graphs in one place.
 |
| Sum of all positive results |
As it's clear to see, Google did best in all categories.
A bit more to consider was that Apple had restricted the use of their assistant to use in their devices only which made it impossible to use it for our project.
Also a positive aspect we recognized in addition to our questions was, that Google did tell the source of information before answering the questions. This also allows judging if the given answer is trustworthy or not. The Google assistant was the only one to do this.
Considering all of this, we decided to use the Google assistant to implement in the smart mirror.
