DIF RFC TWG Final Report

ESDS-RFC-012 Technical Working Group Final Report

The ESDS-RFC-012 Technical Working Group (TWG) has conducted a review of ESDS-RFC-012 Directory Interchange Format (DIF) Standard with the following conclusion:

That the Standards Process Group should endorse ESDS-RFC-012 as a Recommended Standard.

Recommendation

The TWG bases its recommendation on an analysis of the following factors in a NASA context.

Strengths: The DIF is a mature specification. It was first developed in 1987, was chosen by the CEOS IDN (Committee on Earth Observation Satellites International Directory Network), and has been used by the NASA GCMD (Global Change Master Directory) as the means of metadata exchange. The DIF continues to be actively used and maintained by a large variety of organizations, both as a means of interaction with the GCMD as well as for other purposes, including internal data management. The DIF metadata model is compatible with the ISO 19115 metadata model.

Weaknesses: While the DIF is compatible with ISO 19115, there is a feeling among some metadata providers that the DIF should be superseded by ISO 19115.

Applicability: The DIF is meant to contain collection level metadata for Earth science data sets. It is not designed for use as granule, or inventory level metadata. The DIF is primarily a metadata interchange format. Data holders often manage metadata in different internal formats, choosing to export and import metadata in DIF format when interchanging metadata with external organizations, primarily the GCMD.

Limitations: Since DIF is not suited directly as an inventory level metadata format, some data holders perceive this as a limitation. They would prefer to be able to use the same format for inventory level data.

Overall, the DIF is a widely implemented and widely deployed specification that provides a standard way of storing and exchanging Earth science metadata. The ESDS-RFC-012 TWG thus recommends its endorsement by the SPG as an Earth Science Data Systems Standard.

History ESDS-RFC-012 was submitted by Lola Olsen and Chrissy Chiddo of the GCMD.

Overall, the TWG conducted a single review. The review was tailored specifically to reflect the fact that DIF is a mature (some would say de facto) standard with extensive operational use over a nearly 20-year time span. This review was meant to elicit opinions from the community of users of NASA Earth science data.

The review of the specification was completed over a period of approximately 2 months. Based on the responses received from the survey questions and from additional research, the TWG concludes that the DIF specification demonstrates sufficient operational readiness to be endorsed by the SPG.

RFC Overview ESDS-RFC-012 proposes the Directory Interchange Format (DIF) as an ESDS Standard. Its functionality is described in the excerpt below:

The GCMD database holds more than 20,000 descriptions of Earth science data sets and services covering all aspects of Earth and environmental sciences. One can use the search box or select from the available keywords to search for data and services.

The GCMD offers authoring tools, which conform to international standards. The web-based docBuilder tool allows metadata authors to add or modify data set descriptions (DIFs). Tools are available to write, directly submit, and directly update metadata records. In addition, Subscription services are available to notify users of new entries.

The GCMD offers descriptions of Earth science data sets using a specified set of information - known as the Directory Interchange Format (DIF). The GCMD's philosophy for using the DIF is that the specific set of attributes, the content metadata, is the critical set needed for a user to determine if the data sets returned from a database query are those that define viable data sets for the user's needs. Using a specific set of fields also helps to "normalize" the search for data sets through the use of several alternative search engines.

Material from ESDS-RFC-012. Refer to pages 1-5 of ESDS-RFC-012 for a complete description and history of the DIF. Pages 21-25 provide an extensive list of users of the DIF.

The RFC can be downloaded here: http://www.esdswg.net/spg/rfc/esds-rfc-012

Further information about the DIF is available here: http://gcmd.nasa.gov/User/difguide/difman.html