NIST Evaluates Smart Phone-based Language Translators
The National Institute of Standards and Technology (NIST) is evaluating three smart phone-based Pashto and Dari language translation devices the U.S. Defense Department is developing for use in Afghanistan. The institute recently assessed Pashto language translation capabilities, and in late August, it will evaluate translation technology for Dari, another Afghani language. The devices are being developed under the Defense Advanced Research Projects Agency (DARPA) Spoken Language Communication and Translation System for Tactical Use (TRANSTAC) program, which is intended to develop and field the technology rapidly. NIST has previously assessed Iraqi Arabic devices.
The tactical situation in
“There are things that are intangible—reading body language, understanding different dialects, things of that sort. A phone will never be able to read body language, and just the ability to build rapport will always be easier with a human translator,” says Craig Schlenoff, NIST project lead for the DARPA TRANSTAC evaluation team.
Previous generation translation software was installed on laptop computers with microphones. Although the underlying technology may differ, the three smart phone devices all work essentially the same way. An English speaker talks into the phone, and automatic speech recognition software generates a text file that it then translates into the desired language. Text-to-speech technology converts the text file into an audible response. The process is simply reversed for the foreign language speaker.
To evaluate the devices, NIST called in U.S. Marines who have served in
The Iraqi Arabic translation capabilities have been developed over the past four years and are more mature than the devices currently being evaluated. “For these Pashto evaluations, the systems certainly are not as mature yet. In general, they were able to give concepts back and forth, and they were able to carry on dialogues, but there were a lot more problems with interaction. The translations were definitely not perfect. Key words came across, but how the words were put together and how understandable they were could be questionable at times,” Schlenoff relates.
Still, he added that the smart phone devices compare favorably to previous generation laptops. “So far, I’ve been pleasantly surprised with the performance of the phone-based system as opposed to the laptop-based system because the processing is not as powerful in the phone, but the performance is really not that bad, considering,” he notes.
The one-week evaluation involved a variety of nonscripted scenarios, including checkpoints, medical assessments and training sessions with Afghani soldiers. The scenarios were chosen through feedback military personnel previously provided about how they would most likely use the devices. Each scenario took about 30 minutes, during which the Marines and Pashto speakers used the translators to communicate. On-site judges observed each scenario and interviewed participating Marines and Pashto speakers afterward. Weeks later, another panel of judges fluent in both English and Pashto viewed videos of the simulations and evaluated the performance of the three devices. NIST is providing a written report to DARPA to aid future decisions on funding and program direction.
Although language translation software is already available for smart phones, Schlenoff says they do not meet the needs of troops in
Three contractors—SRI International, Raytheon BBN Technologies, and IBM Corporation—are developing technologies under the program.