Building a Federal Healthcare Data Hub

Facing a pandemic and an aging legacy medical record system with limited data storage capacity at an on-premise data center in Charleston, South Carolina, the Enterprise Intelligence & Data Solutions (EIDS) team, sprang into action to complete a game-changing cloud migration project. The effort, called the Accelerated Migration Project, or AMP, moved petabytes of secondary healthcare data and related applications to the cloud. The project digitally transforms access to U.S. Defense Department medical records and offers better data analytics and more reliable information discovery, driving improved outcomes in patient care and business operations, experts say.

The platform serves a variety of users, including clinicians, medical treatment facilities, civilian facilities, the Defense Department, the Department of Veterans Affairs (VA), data scientists and analysts. The health data delivery system provides those users a better understanding of the data and increases confidence in its validity, says Chris Nichols, program manager for EIDS, Program Executive Office, Defense Healthcare Management Systems (PEO DHMS).

“The week we commenced the project, we could already see the data’s significance for the Military Health System’s (MHS) COVID-19 response,” Nichols states. “The data we needed to move was and is critical to our health care teams’ ability to track and support patients affected by the virus. Our ability to succeed—and to do so both quickly and without disruption to health care—was imperative to the MHS COVID-19 response.”

The PEO DHMS reports directly to the Office of the Under Secretary of Defense for Acquisition and Sustainment, and administratively, is attached to the Defense Health Agency (DHA). The project benefited from top leadership support with Lt. Gen. Ronald Place, USA, director, DHA, making it a focus for the young agency, which was stood up seven years ago as a joint, integrated combat support agency to manage all of the service’s garrison healthcare for MHS beneficiaries. Before then, each service managed its own electronic health care records.

Besides the enormous scope and scale of the project, another remarkable aspect of AMP was the team’s speed in completing it—93 days.

“Gen. Place made this a priority in mid-February, and we met in the first week of March in Charleston,” Nichols says. “We turned on everything in AWS [Amazon Web Services] on May 31, and we turned off and disconnected the legacy environment on the 12th of June.”

Justin Fanelli, chief architect, EIDS Program Management Office and technical director, Navy Manpower, Personnel, Training and Education, at the service’s Navy Information Warfare Center, adds, “We learned we can do these types of projects more efficiently, more effectively than expected. We learned most of all how to take down the roadblocks and build an execution plan to deliver capabilities at the speed of relevance.”

As a Defense Department and intelligence community veteran, Fanelli, an engineer and technologist, has championed innovative software, architecture and system development over the past two decades, working to create intelligence, command and control, and enterprise systems, including architecture stand-up of the Space Development Agency. Fanelli helped steer AMP’s engineering and architecture for the DHA team, a group that also included engineering and technical staff, a cyber team and data innovation driven by clinicians, including pharmacists, physicians and data scientists.

The team secured multiple authority to operate decisions in rapid fashion, an uncommon feat. For that accomplishment, the project leads relied on a team of teams, including assistance from the DHA J-6, Joint Staff; Defense Information Systems Agency (DISA); several Navy commands for cyber, technical and operations; and interface partners, such as the Coast Guard and others, Fanelli says.

To create the healthcare data hub, the team relied on commercial cloud hosting services through AWS, and Sharepoint (Carepoint), Qlik, Tableau and others for its data visualization and user interface. These companies told the DHA team that AMP was a record-breaking project in terms of capacity.

Overall, the effort involved working with 20 outside vendors, restructuring to 14 cloud native services, managing 60 separate applications and consolidating several hundred virtual machines. The data that had to be migrated was actively used by tens of thousands of healthcare teams across the MHS. “Essentially, the team learned to crawl, walk and run at the same time for this activity because we recognized early that perfection would be the enemy of putting good capability in users’ hands when they need it,” Fanelli notes.

From Amazon, the team purchased 16 data storage devices, called Snowballs, to house the critical data. “We had so much data that we couldn’t send it over the wire,” he says. The team bought two devices at a time, and quickly added themselves to the waitlist for Amazon’s next version, the Snowball Edge, to be manufactured. Each Snowball Edge has a capacity of 100 terabytes. Given the urgency of the pandemic and possible looming system failures on the legacy equipment, Amazon representatives even drove the Snowball devices down to Charleston from Washington, D.C., where the PEO DHMS is located, to move the data off the legacy systems.

“The pandemic hit right at the beginning of the project, just as the need grew for tracking of people with the virus, and PEO DHMS had to build related applications, such as for symptom checking, right on top of our platform,” Fanelli shares.

The AMP was originally planned to take a year and a half, but given the circumstances the experts moved quickly. “We were looking down the barrel of hardware refreshes of over 300 servers,” Nichols states. “We couldn’t just lift and shift.”

And aside from the 16-hour days and numerous energy drinks, the results are incredible, he notes. “We are now harnessing the power of the cloud,” Nichols says. “We restructured all the migrated services and data into cloud-native services. Most people have a fuzzy sense of what the cloud is. They think of it primarily as data storage. But it’s truly a powerful data service too, and those services that are available to us through the cloud free my team to do what we do best. We no longer have to develop and manage our own machine learning, artificial intelligence or deep insight tools. The cloud does that for us.”

Nichols served in the U.S. Army as a combat medic, licensed practical nurse and officer, and after serving, moved into healthcare information technology, with a stint as a civilian in the Army Surgeon General’s office, running clinical systems integration and managing all of the electronic health record systems from an Army perspective. While battling cancer shortly after, Nichols received treatment from various Defense Department and VA facilities across the country. He saw firsthand how disconnected his medical records were as he received care in Florida, Washington, Maryland and Oregon, resulting in six volumes of paper records.

“My data was kind of placed everywhere,” he exclaims. “As a clinical person myself, it was hard to imagine that I couldn’t even get my hands around where to find the data that I needed to help inform my care.”

After successfully beating cancer, Nichols wanted help with the department’s pursuit of improved data management for patients. “What matters with the data, in my opinion, is how we use it to impact clinical care, and how we use it to impact the business of healthcare, decrease our costs and provide the best outcomes for patients,” he states.

Nichols emphasizes that the Defense Department is in a unique position, compared to traditional healthcare providers, in that it holds medical records during people’s whole professional lives in the military. From enlistment, all the way to military retirement, it is a diverse and lengthy set of data.

“DOD maintains one of the most robust longitudinal health data records in existence,” he says. “It is a treasure trove of population health data and benefits researchers in a unique way because it’s now consolidated into a single, authoritative source.”

Now that the team has gathered the data together in a central hub, they are working to add capability to provide “deep insights” to users. “First, we are working to create transactional continuity between organizations,” Nichols observes. “In other words, as a patient, I can go from the VA to DOD with ease and confidence. Second, we’re positioning to help the MHS, researchers and our partners capitalize on data insights and the promise of artificial intelligence and machine learning via cloud services.”

As part of the second phase of the AMP project, the team will work on developing a related data dictionary, and then they will add new data sources to the hub. The idea is to provide so-called augmented intelligence to support providers’ or researchers’ decision-making processes.

“We plan to add major new data sources to the system, add and optimize tools, transform databases, spin up new capabilities faster and improve our program interfaces to allow better access to get the right people the right data at the right time,” Nichols states. “Medical device data and social determinant data sources are growing, and we want to find the balance for how we manage care for patients but also provide the right tools for clinical decision support from core electronic health records. We did this with COVID, and we will leverage those lessons to continue change in the future.”

Also, Nichols suggests that Phase 2 of AMP will be less about technology and more about integrating and partnering with clinicians in the field. “How do we partner with researchers and administrators to provide impactful change?” he ponders. “How do we clean up what we have and make it more useful for users? How do we get ahead of a road map we planned over the next 12 months around machine learning and artificial intelligence?”

Moreover, the data hub and its advanced compute capabilities will allow researchers to harness prescriptive analytics and to delve into big data medical research questions, such as genomics or analysis of health-related Internet of Things devices.

“This health data delivery system will support the future of federal health data management and data sharing,” Nichols concludes. “It is the first major step toward creation of a federal health data hub. As a clinician, I think that the core enabler really is putting the right data in the right hands at the right time to make it actionable. It is going to enable us to really start to pull in and leverage data as a strategic asset for this organization, more than we were ever doing before. And the progress we made ensures that the system will maintain the highest levels of sustainability and reliability.”