Air Force Squadron Shares Cyber Training Lessons Learned
Passing the National Security Agency’s comprehensive cybersecurity training is no easy feat, but the Air Force’s 39th Information Operations Squadron has achieved unusually high success rates in getting students through the programs.
The National Security Agency (NSA) once offered a course known as Remote Interactive Operator Training (RIOT). It now provides a course called Future Operator Readiness, Growth and Enrichment (FORGE). The course includes three phases: the basic Computer Network Operator Qualification course; the Operations course, which teaches students to use cryptologic organization tools to successfully carry out missions; and the Evaluations course, which assesses a student’s ability to make effective decisions, including risk management and mitigation decisions.
Excluding the higher Air Force numbers for successful students, the Defense Department generally achieves about a 60 percent success rate for the first phase and 50 percent for the final course. The Air Force, however, has an 85 percent passing rate so far this year. Last year, they passed 44 students out of 50, an 88 percent success rate, through the first phase and nine out of 10 students for the Evaluations phase, according to Maj. Solomon Sonya, USAF, 39th Information Operations Squadron, Detachment 1 commander, who leads the Offensive Cyber Operations Initial Qualifications Training program.
Numbers across the department dipped in 2020 due largely to training turmoil caused by the COVID-19 pandemic, Maj. Sonya, says. That year 77 percent of students from the 39th passed the first phase, but only three out of nine students succeeded through the final course. Prior to that, the squadron achieved a 63 percent pass rate in 2018 and 89 percent in 2019.
“We have more people coming in a few months, and I’m confident we’ll stay at a good, high 80 percent this year,” he adds.
An NSA spokesperson was unable to verify the numbers, indicating that doing so is complex. But the agency public affairs office did offer the following statement to describe its cyber training efforts: “In collaboration with military services, NSA develops training standards for use by the military schoolhouses responsible for training service members. Success in NSA’s computer operator network training programs depends on a combination of deep technical understanding and the ability to apply learning at a level commensurate with the operational threat environment. Successful candidates either have prior academic exposure or previous experience in at least 50 percent of the course material, which is roughly equivalent to an undergraduate degree in a related STEM (science, technology, engineering and mathematics) field.
This training program fills the dual purpose of providing specialized training and evaluating job readiness. The program provides in-depth and practical applications necessary to proficiently identify, understand and navigate the digital environment; utilize a given set of tools to develop situational awareness within the digital environment; understand network operations methodologies; and demonstrate strategic practical application.”
When the major took over the Air Force’s offensive cyber training unit in 2018, the service’s average annual pass rate was between 36 percent and 44 percent, so Maj. Sonya, who formerly taught at the Air Force Academy, set about making some changes.
The 39th instituted several new programs designed to support students every step along the way, including a more deliberate selection process to ensure the Air Force was accepting qualified students for the cyber training positions; skills enhancement tailored to each candidate as they await the next phase of training; trend analysis to determine where and why students fail most often; a ready review process to determine if students are prepared for the next test; a resiliency program to support mental and emotional health; mentorship with actual operators and a formal feedback program aimed at making continual improvements.
Maj. Sonya says he relied on his academic background to assess why the program originally had a poor passing rate. “I hypothesized that one of the reasons people were failing so much is because we’re bringing in the wrong people to start with for this training. There should be a selection program and an interview process to now better bring in the people who have the right skill sets and the right cyber proclivities to get started into this pipeline,” the major recalls. “If stronger people who have a background get started, then hopefully more people will graduate through the program. Unfortunately, we didn’t have a selection process at that time, so I created it.”
Under the skills enhancement effort, dubbed the Cyber Assessment of Recommended Training, the team at the 39th identifies a student’s weaknesses and then looks for ways to improve their skills. “Looking at many of the open-source and freeware tools, programs and courses that are available, we came up with an index of courses, and then we use our assessment as well as another assessment by the National Cryptologic School. I take that assessment, I see how people perform on it, and then I pigeonhole different courses based on what I can see the weaknesses are for the students, so that again, they can be stronger while they’re waiting for their training,” Maj. Sonya explains.
The 39th instituted the trend analysis program with help from the NSA. The major reports that the NSA initially didn’t have a trend analysis process in place, but officials there were generous in sharing an entire year’s worth of data so that he could develop his own. “It turned out that Unix was the highest failure rates. If people were going to fail, it’s usually because they did not know Unix. So now, I’m going to make sure that the first thing everybody learns is how to utilize the Unix operating system,” Maj. Sonya states.
The data analysis also showed that the first phase, which is a 90-day, self-paced course, actually takes most students up to 5.5 months to complete. “It is hard to pace yourself in a four-to-five-month, self-based program. It’s very hard to pace yourself,” the commander notes. “But looking at the data, I was able to understand we can bifurcate the students based on those who are passing and those who are failing. On each day, I can see where the majority of students had passed each of the modules they need to study, as well as how are they performing.”
Armed with new knowledge, the squadron established a baseline mark. Those not meeting the mark are assigned additional training and will also be held accountable so that they stay on schedule. “I keep them on pace so no one fails our pipeline because of training days or training day delays, which was a reason people were failing before,” Maj. Sonya asserts.
He compares the ready review program with the Air Force practice of periodically evaluating pilots to ensure they are still “airworthy.” It gives the instructors “perfect insight” into whether a student is prepared to pass a test in the NSA program. “Something I intentionally do not know is what questions the NSA asks. I just look at the objectives that they’re expected to know for the exam that they’re about to take, and then I had us create different virtual machines along with different scenarios, and then I test the student. If you’re ready to take this test by the NSA, you should pass the assessment I’ve created here,” the major asserts.
He stresses the importance of the resiliency program as well. “The resiliency piece is really important because not only do we work on the students’ technical acumen—that’s not the only key to succeeding in this really long, rigorous program—there’s also a mental aspect. There’s a perseverance, a mental strength piece. We have to make sure that the students are as strong, as rested and as ready as possible to continue in the program.”
The mentorship program also aims to boost morale. “We still need to let students know there is light at the end of the tunnel. We bring in actual operators; we have them tell their war stories. We have them say what’s going on in operations right now. Without that, some people lose their motivation.”
Finally, with the formal feedback program, the Air Force team is able to see where training needs improvement. Maj. Sonya’s team recently awarded a contract worth hundreds of thousands of dollars to an “external vendor” for offensive cyber operations curriculum development. That curriculum could be in place as early as July, possibly September.
Maj. Sonya says he has shared information about the Offensive Cyber Operations Initial Qualifications Training with the Army, Navy, NSA and U.S. Cyber Command. Other organizations, in turn, have helped his team as well. The Air Force’s Basic Operations Course, for example, builds on a similar course the Army created and shared. “All of the material we’ve created and we utilize to touch all of our students, I share with everyone. Anyone who needs it—the Army, U.S. CyberCom [Cyber Command]—they have our materials. I want the entire enterprise to succeed, not just our Air Force people.”