As it is not suprising that Siri, being in the “beta” status still, is using a very new technology for voice recognition for mobile devices, which from a study made by Piper Jaffray analyst Gene Munster has found that the abilities of Siri are still thrown off in the text-entry Google searching which have been around for years.
This new study also includes the presentation of Siri with 800 queries within each of two environments, which includes a busy urban street and a quiet room, and along with the assessment of the ability of Siri to take in the commands and give responses that are accurate. It specifically works on outdoor testing, as noted by Munster:
While Google comprehends 100% of searches, Siri was only able to comprehend 83% of the prompted questions on a busy Minneapolis street. In terms of answer accuracy, Siri correctly answered 62% of queries on the street compared to Google’s estimated 86% answer accuracy. Based on these results, we give Google an A+ for comprehension and a B+ for accuracy while we give Siri a B for comprehension and a D for accuracy.
Though as Siri has lots of room for even more improvements, it has been confirmed that Munster that this new technology is within it’s “infancy” and he is remaining “optimistic for it’s own future” with iOS 6 potentially bringing much bigger improvements.
The testing that Piper Jaffray had done was using the built-in microphone of the iPhone in a very quiet setting and also a Skullcandy microphone headphones for testing outdoors, and since Siri’s abilities were used to correctly comprehend the queries falling from 89% to around 83% when going outdoors to a very noisy urban setting. On the side of accuracy, Google’s 86% rating is derived from comScore’s data that shows that Google creates one 1.14 search result pages for every search, which goes to show that around 86% of the time Google shows the consumer the data that their looking for within the first results on demand.
Something else that was analyzed by Piper Jaffray, is the sources that Siri uses. He found that Google had provided 60% of all results, with Yelp and aother sources that filled in the remainder.
Of queries excluding commands (i.e. call Jay, send text) Google would provide 60% of the answers, Yelp 20%, WolframAlpha 14%, Yahoo 4%, and Wikipedia 2%. Breaking down Siri’s reliance further, Google provides 100% of navigation results, 61% of information results, 48% of commerce results and 42% of local results. Among other result aggregators, Yelp provided the most local results (51%) and commerce results (51%), while WolframAlpha provided 34% of information results.
It is estimated by Munster that Google’s share of results of Siri will be dropping to 48% within the release of iOS 6 as the navigation, sports, and last but not least movie queries will be moving over to Apple and their newest partners. Can Apple live without Google’s search results for Siri?, we’ll have to wait and see, stay tuned.