Blog: Disaster Recovery Exercise Covers the Bases
A major disaster recovery exercise is concluding today in Washington D.C. The week long event was held by AT&T to test, evaluate, refine and improve how the company restores communications in the wake of a natural or manmade disaster. The network disaster recovery (NDR) exercise filled the capitol's Robert F. Kennedy Memorial Stadium with equipment trailers and personnel providing satellite and broadband fiber optic communications links.
Held several times a year, this is the 54th and largest field NDR exercise conducted by AT&T, says Mark Francis, vice president of the company's Global Network Operations Center in Bedminster, New Jersey. The company gathered more than 45 disaster recovery trailers and vehicles, including 28 tractor trailers, two emergency communications vehicles, four hazmat trailers, one satellite cell on light truck (COLT), and five smaller utility and support trailers.
During a disaster recovery, Francis explains that the company's strategy is to set up equipment in a large open area such as a stadium. All of the equipment forms the core of an AT&T regional office. The trailers can replace the fiber optic backbone equipment in an office and replace the central network. He notes that one of the goals of this exercise was to test high speed optical transport equipment operating at 40 gigabytes per second. The event also sets up mobile cellular communications systems for first responders.
It took 27 hours to set up the site and get the network running, says Francis. He adds that this was satisfactory because it fell within the company's operational goal of reestablishing communications within 24 to 36 hours after the equipment arrives. The tests involve responding to contingencies such as the loss of generator power and shunting power to backup systems.
AT&T established its NDR capability in 1992, investing more than $500 million to date. The company maintains a fleet of more than 300 self-contained equipment trailers and support vehicles located at five strategic warehouses across the United States. Equipment for the Washington event came from three of the company's national warehouses, Francis shares.
Over the years, the company's NDR teams have responded to a variety of disasters, including restoring service after the Oklahoma tornadoes in 1999; the September 11th attacks on the World Trade Center in 2001; hurricanes Katrina and Rita in 2005; and hurricanes Gustav, Dolly and Ike in 2008. When hurricane Ike struck the Texas coast in 2008, the company's Galveston office was inundated under seven feet of seawater. NDR teams deployed within 72 hours and were able to restore service to local university offices, and provided communications to the local hospital and high school, which served as an emergency shelter.