Texas Computer Scientist Outflanks Next-Generation Computer Viruses

October 3, 2011
By Max Cacas, SIGNAL Online Exclusive

A researcher at the University of Texas at Dallas has discovered a new way to anticipate the actions of computer viruses, possibly heralding a new generation of tools and strategies to combat malware that attacks networks, servers and individual personal computers.

An added bonus: those tools take advantage of computing capabilities and instructions already “baked into” a wide range of computer chips now in use.

Dr. Kevin Hamlen is a researcher with UT/Dallas’ Cyber Security Research Center. He has received a five-year, $500,000 Faculty Early Career Development Award from the National Science Foundation to continue his work in what he calls the “virus-antivirus arms race.”

Hamlen says his current project is an outgrowth of research originally conducted with fellow researcher Latifur Khan for the U.S. Air Force Office of Scientific Research (AFOSR) in the area of “reactively adaptive malware defense.”

The goal, he says, was to try to determine a way to anticipate “next-generation malware attacks.”

“Right now, the way most viruses work,” he explained, “is that they randomly propagate throughout the network, and they randomly mutate themselves to avoid being exact copies, so they become harder to detect. What our research was looking at was could these viruses get worse by, instead of randomly mutating, mutating in a direct fashion, so they infect a machine, actively detect what sorts of defenses are on that machine, learn about them using advanced machine learning techniques, and then actively work to defeat those defenses in a network fashion.”

Hamlen says so far, cybersecurity experts have not seen widespread evidence of malware targeting existing anti-virus applications, but he believes that’s only a matter of time.

He says he and his colleagues realized the best defense is not to wait till the malware reacts “but proactively try to figure out what it’s likely to do in the future.”

Hamlen’s research involves advanced algorithms based on “language-based security,” involved in applying programming-language research to software security.

“In programming-language research, there are advanced algorithms that try to predict what programs will do once you execute them,” he explains, adding that computer programmers commonly use such algorithms to debug their software.

Hamlen says those same formulas can be used to predict the actions of malware programs in the microseconds before those programs begin to execute and mutate.

“We discovered that there’s a way to automatically interrupt viruses at precisely the moment they de-crypted the malicious payload, but before it starts executing.”

An added benefit, says Hamlen, is that his research takes advantage of computing capabilities and feature sets already programmed into the vast majority of CPU chips now used in everything from laptops to large-scale cloud-computing network servers, and used for other purposes within devices.

“Nobody has to go out and buy new chips or new software, it’s a combination of features that already exist, and if you put them together, and configure them in software just right, you can interrupt (the malware) at this magic point, at the exact point where you can apply language-based analysis, and do all sorts of prediction of what these viruses are likely to do.” The capabilities that Hamlen’s research takes advantage of are operating-system agnostic.

Over the next five years, Hamlen says he hopes to continue his research into which algorithms will prove to be the most effective in creating new anti-virus programs. In some cases, he says, as his research continues incremental findings on effective anti-virus formulas could be promptly integrated into existing anti-virus programs. He believes the product of his NSF-backed research can be successfully used to enhance existing software. In addition, once compatibility issues are addressed, software manufacturers could also develop new anti-virus applications.

In addition to the NSF and AFOSR, the Cyber Security Research Center has conducted research backed by the National Geospatial-Intelligence Agency, NASA, the Office of Naval Research, the Intelligence Advanced Research Projects Activity, and the Multidisciplinary University Research Initiative Program.