Accurate Metrics Support Efficient Spectrum Use
Database on electromagnetic-dependent equipment reveals battlespace electronic environment.
The U.S. Joint Spectrum Center is developing and will maintain an extensive set of databases to directly support both the spectrum management and electromagnetic environmental effect communities. To accomplish this task, the center has established the Data Quality Metrics Program, which will monitor and enhance the quality of its databases.
The demand for spectrum management and electromagnetic environmental effect data is rapidly increasing. As more of the electromagnetic spectrum is used by both commercial and military systems, the potential for interference multiplies. As a result, it is becoming more difficult to ensure maximum performance of warfighter systems. Consequently, the demand for Joint Spectrum Center data, which is used to analyze and predict the operational performance of warfighter systems, also is increasing.
Joint Vision 2010 and the new Joint Vision 2020 (JV2010/20) have significantly increased the requirements to support warfighter spectrum management and interference resolution, and the quantity and complexity of newer, more advanced telecommunications and radar systems have progressively grown. Today, the warfighter must ensure electromagnetic compatibility in diverse areas of the world for both joint and coalition force deployments. These new requirements are significantly expanding the need for better quality information.
The Joint Spectrum Center (JSC), Annapolis, Maryland, supports information/decision superiority and full spectrum dominance by serving the Office of the Secretary of Defense, the Joint Staff, unified commands, military departments and defense agencies to ensure the U.S. Defense Department’s effective use of the electromagnetic spectrum. Its work will result in fielding warfighter equipment and systems that can function compatibly in the intended operational electromagnetic environments.
To support the warfighters, the center maintains five primary databases that contain technical and administrative data for a wide range of applications that warfighters need.
The first two databases feature information about equipment usage. The Frequency Resources Record System (FRRS) includes administrative and technical data that describe the authorized use of electromagnetic-dependent Defense Department systems in a specific environment. The background environment information (BEI) database includes administrative and technical information of the authorized use of all known electromagnetic-dependent non-Defense Department systems worldwide.
The last three databases the JSC maintains contain records on equipment. The Spectrum Certification System (SCS) includes administrative and proposed technical information that is used for approving equipment acquisitions. The technical characteristics of communications systems, radar and space-based platforms as well as the orbital characteristics of satellites reside in the equipment characteristics/space (EC/S) database. Finally, the center’s tactical database (TACDB) is the repository for information about military force structure, equipment and platform inventories, and platform configurations. The inventory data concern communications-electronics equipment and the platforms that are authorized and assigned to military units. Platform data includes a list of the communications-electronics equipment found on weapon systems and mobile platforms.
To meet future challenges, the center must maintain more accurate, complete and current records. Toward this end, the center currently is implementing the Data Quality Metrics Program to focus on meeting two of the primary JV2010/20 objectives of information/decision superiority and full spectrum dominance. The goal is to monitor and improve the overall quality of the JSC databases by using a standard set of data quality metrics. Based on industry best practices in data quality management, the center measured three primary attributes of data quality: accuracy, completeness and currency.
Data is considered accurate if it is technically correct and is a true representation of the electromagnetic environment. Accuracy is measured by the percentage of records that accurately reflect source data, the percentage of valid critical data elements across all records in the database and the percentage of the database records that include verified information. Data that represents the total electromagnetic environment is considered complete. The JSC measures this by determining the percentage of user-specified data requirements that are included in the database, the percentage of fields filled for critical data elements and the percentage of total expected records captured in the database. Information that is up to date relative to the actual electromagnetic environment is considered current. This attribute is measured by determining the average age of the source data. Standard units of measurement are used to quantify the three quality attributes relative to each of the primary databases.
The JSC data quality improvement process is continuous and starts with defining the data quality metrics and establishing database-specific quality goals. This step is followed by measuring the quality of the data in each database, assessing the quality of the data relative to the quality goals, and defining and implementing database quality improvement initiatives. Once assessment and data improvement steps are complete, the data quality goals are reassessed and the procedure continues.
As of the end of fiscal year 2000, the JSC has implemented automated capabilities to measure approximately 70 percent of these data quality attributes. Additional automated capabilities are being developed and will be operational in fiscal year 2001 to ascertain the quality of nearly all of its data.
The center has performed an initial assessment of the quality of the information in its databases. Because the capability to measure data quality objectively currently does not exist for some databases, the JSC is developing automated capabilities so that objective measurements can be made in the near future.
To date, the center has made substantial progress in evaluating the quality of the information in the five primary JSC databases. For all five databases, the center has verified that between 80 and 100 percent of the records accurately reflect the source data. However, it has not yet determined the percentage of valid critical data elements across all records in the database or the percentage of database records that contain verified information.
In the area of completeness, the JSC has measured all five primary databases and determined that the percentage of user-specified data source requirements included range from a low of 75 percent for the BEI to a high of 100 percent for the FRRS and SCS. The EC/S and TACDB have a 98 percent and 86 percent rating in this category, respectively. The data fields filled for critical data elements range from 62 percent to 98 percent across the five databases. While 100 percent of the total records that are expected to be in the FRRS database have been captured, the TACDB currently stands at 98 percent, the BEI at 50 percent and the EC/S at 94 percent; the percentage for SCS still needs to be determined.
Data currency figures show that either moderate or additional quality improvement is needed. Of the five databases, the BEI is closest to the goal of one-year-old data, as it currently stores information that is 1.4 years old. The goal for the TACDB also is one year; however, at this point the information is 2.1 years old. The FRRS data is currently 3.5 years old with a goal to contain data that is no more than two years old. The EC/S database presently contains information that is eight years old; however, the goal is to reduce this to five years. Finally, the JSC has determined that the SCS database has the least current information of the five primary databases. Information in that repository is 19.4 years old. The goal is for the database to contain data that is no more than five years old.
The JSC needs to improve the quality of its data in several areas to fully support user requirements. To mitigate these deficiencies in the quality of its data, the center has identified specific database improvement initiatives.
The first action is to add electronic warfare integrated reprogramming (EWIR)/electronic order of battle (EOB) data in the EC/S and BEI databases. The JSC database user community has requested additional source data on foreign equipment. The Defense Intelligence Agency maintains the EOB database, which contains location information on foreign fixed and mobile equipment. Under this task, the collateral version of the EOB, which is classified Secret, will be integrated into the center’s BEI database. Because the EOB records identify only the location of equipment, the technical characteristics of that equipment must be captured from a separate source by integrating the EWIR database, developed by the National Air Intelligence Center, into the center’s EC/S database. A link between the BEI and EC/S databases will be established to augment the EOB location records with equipment characteristics information.
The center’s second task is to capture and maintain additional system data in EC/S databases. The JSC currently maintains records on Defense Department communications and radar systems. Very limited technical data is maintained on the commercial systems that the warfighter uses, electronic warfare systems and military digital networks. Because the performance of these systems directly affects the warfighter’s mission effectiveness, the center must be able to capture and maintain technical data needed to perform electromagnetic environmental effect analyses on these systems.
Third, the JSC is developing a software capability to augment FRRS and BEI data. The center maintains frequency assignment data in the FRRS and BEI databases that outside sources provide. Many of these records are missing technical information. The task is to develop a software capability to identify missing and incorrect technical data and, where possible, to add estimated technically correct data values. When incorrect data is found, both the source-provided values and expected technically correct values will be stored.
Finally, the center will review SCS data. The database contains applications for frequency allocation (J/F-12) data provided by each military service. J/F-12 (joint format 12) is a format that is used mainly by spectrum managers to allocate spectrum/frequency to systems. Reviewing will entail a periodic technical review of existing J/F-12 data in the SCS database. An engineering review of 600 J/F-12s will be performed each year. When data is found to be inaccurate, the responsible services will be contacted to update their records. The results of each year’s data reviews also will be used to estimate the accuracy of the entire SCS database provided by the services.
The JSC has developed a comprehensive approach to monitoring the quality of its databases to identify the best areas for improvements. The process being implemented is based on industry best practices that have proven to be effective in quantifying and evaluating the quality of the data maintained in the center’s databases. The JSC continues to develop and refine its metrics program. By the end of fiscal year 2001, the center plans to have a full suite of automated capabilities to measure data quality across all of its databases. Specific initiatives are already in the planning stages.
Stuart Kennison is the data administrator and the division manager for database operations at the Illinois Institute of Technology’s Research Institute. He oversees the long-range planning of the data resources at the Joint Spectrum Center. Jamileh Soudah is the team leader and project manager for the database and modeling and simulation programs at the Joint Spectrum Center.
