Go to Moscow to Learn Russian (Sort of)
New virtual reality technology provides instant language immersion without leaving home.
Language study is a national imperative, and the technology shaping it goes as far back as 1877, when Thomas Edison’s phonograph promised to break down geographical barriers to let Chicago learners practice German as it is spoken in Berlin.
Fast-forward 140 years to an era when virtual reality (VR) is transforming language instruction as we know it. Exciting breakthroughs capitalize on the rapidly progressing technology to help deliver critical language and sociocultural content and experiences faster than ever before with fewer resources than full immersion experiences.
One revolutionary approach combines VR with empirically based cognitive-learning principles, marrying technology and pedagogy. Called REVEAL 360 Degrees: Research-Enhanced Virtual Environments for Acquiring Language, the prototype provides a virtual 360-degree cinematic experience for immersive foreign language learning. Innovators at the University of Maryland Institute for Advanced Computer Studies (UMIACS) created the platform, and language-learning researchers from the university’s Center for Advanced Study of Language (CASL) provided the content that runs on it. The tool immerses learners in a setting that requires them to focus on a foreign language conversation amid background noises and distractions. This type of realistic practice is critical for developing high-level listening skills. REVEAL could boost engagement in lessons and contribute to an increased rate of learning and better learning retention.
The benefits also extend beyond the classroom to the working world. Consider how an immersive experience could aid a language analyst who wants to work in foreign policy. Such learners require authentic language input, not canned textbook conversations that fail to be truly realistic. REVEAL could expose these types of learners to the right linguistic and cultural content at the right time. Educational experiences that otherwise would be too expensive—such as study abroad—or complex or dangerous become within reach for real-life practice thanks to this VR technology.
The current prototype transports a learner to an embassy cocktail party in Moscow to listen to multiple Russian conversations in a noisy setting. The conversations range from casual how-do-you-dos and friendly introductions to intensive talks about international policies on immigration and refugees. Using a hand-held clicker, the learner seamlessly can “teleport” from one conversation to another and actively listen to realistic dialogue between native and non-native Russian speakers.
VR technology not only makes this doable but also affords learners multiple options, offering them choices between digitally created worlds and recorded videos with actors. The 360-degree video—an audiovisual recording of an environment with views from all directions—was chosen because it conveys realism, surpassing even the best available computer graphic environment. The ambisonic recording, a full-sphere surround-sound technique, preserves the language context and captures detailed cues essential for high-level language learning. REVEAL can render facial expressions and microexpressions, gaze directions, gestures and other subtle body language signals—important communication mechanisms that a completely digitally generated environment cannot replicate without significant production and computing power.
But this offers another advantage. Initial research suggests that visual memory is stronger when learning in a 360-degree immersive environment compared with looking at the same scene on a 2-D computer screen. To evaluate the REVEAL prototype’s efficacy, the CASL-UMIACS team in April began a study comparing the learning outcomes of participants who viewed a 360-degree video of a conversation with the outcomes of those who viewed a 2-D video of the same conversation. The key will be determining whether participants feel a strong sense of being present in the immersive VR environment and if that feeling translates to greater language comprehension, knowledge or recall. There might be noncognitive benefits of this immersion tool as well, such as greater confidence or enthusiasm for using the language, because learners prefer being part of the scene rather than sitting in a classroom.
REVEAL already has been screened by Russian language instructors in focus groups. All commented that they were impressed by the quality, authenticity and richness of the conversations and the level of engagement in the virtual space. Several said they felt as if they were flies on the wall, while others said it was “overwhelmingly real” and that they “would watch a million times, it is so rich.”
These first impressions of the prototype suggest there is a place for language-learning virtual training environments for the entire educational pipeline, from elementary all the way to university-level students and even for work force development.
Amitabh Varshney, Eric Lee, Sida Li and Duane Shaw contributed to this article.
Medha Tare, Ewa Golonka and Martyn Clark are research scientists in second-language acquisition at the University of Maryland (UMD), College Park, Maryland. Amitabh Varshney is a computer science professor and the interim vice president for research at UMD. Duane Shaw is a research systems engineer at CASL, and Eric Lee and Sida Li are research programmers at the university’s Augmentarium, a virtual and augmented reality facility for the study of visual augmentation of human intelligence and amplify situational awareness. The views expressed are their own.