ChaCha Voice Search on iPhone outperforms voice-enabled search from Google and Yahoo!
By Dusan Belic on Wednesday, July 22nd, 2009 at 12:51 AM PST In Research, Services, iPhone
Don’t just love David vs Goliath stories, when a little guy beats the big bully? On the similar note, we caught a press release from ChaCha. Apparently, MSearchGroove found that ChaCha’s voice-enabled mobile search “proved superior” to two other options for the iPhone: the Google (NSDQ: GOOG) Mobile App and Yahoo Mobile App.
This, of course, shouldn’t come as a surprise as in most cases it’s humans who interpreted the search query for ChaCha, as opposed to voice-to-text technology in case of Yahoo and Google.
From the press release:
According to the study, ChaCha interpreted natural language search queries, that is, queries asked as questions, accurately in 94.4% of the tests and delivered an accurate search result in 88.9% of cases. The Google voice recognition technology interpreted queries accurately in 16.7% of tests and delivered accurate search results in 22.2% of tests. The Vlingo for iPhone voice recognition technology correctly interpreted queries in 72.2 % of cases and delivered accurate results (via Yahoo!) in 27.8% of tests.
For the evaluation purposes, the researchers asked a series of 18 queries representative of six typical mobile search categories: Navigational, Directions, Information Local, Information General, Social, and Long-Tail. For each query the researchers evaluated nine performance characteristics including response time, results accuracy, voice recognition accuracy, number of results received, keytaps required, relevancy of the result, location awareness, use of advertising, and presence of other value-added features. Moreover, researchers also conducted two rounds of tests, using both natural language (speaking the questions in sentences) and keyword (reducing questions to a few specific terms) queries, and ChaCha outperformed the two other contenders here as well…
Those who want to know more, the full report waits you as a free download from MSearchGroove’s website.
This is an interesting study. ChaCha does offer great accuracy, however, if you’re looking for speed, Vlingo and Google provide search results much faster in my experience. For users that need some form of directory assistance answers, I’ve always found GOOG-411 as one of the fastests and accurate ways to search, and the great thing is it’s available from any phone because you only need to dial 1-800-GOOG-411. This is in large part because the system itself guides you down a path to narrow the searchable voice grammar file. In contrast, Vlingo and Google’s voice search application are trying to recognize spoken words and match it to a database of millions of possible voice grammar files.
All said, for the right user application, GOOG-411 and Vlingo can actually provide a better experience than ChaCha.