Cluster Active Archive: Overview

Abstract

The four-satellite Cluster mission investigates the small-scale structures and physical processes related to interaction between the solar wind and the magnetospheric plasma. The Cluster Active Archive (CAA) (URL: http://caa.estec.esa.int) will contain the entire set of Cluster high-resolution data and other allied products in a standard format and with a complete set of metadata in machine readable format. The total amount of the data files in compressed format is expected to exceed 50 TB. The data archive is publicly accessible and suitable for science use and publication by the world-wide scientific community. The CAA aims to provide user-friendly services for searching and accessing these data and ancillary products. The CAA became operational in February 2006 and as of Summer 2008 has data from most of the Cluster instruments for at least the first 5 years of operations (2001–2005). The coverage and range of products are being continually improved with more than 200 datasets available from each spacecraft, including high-resolution magnetic and electric DC fields and wave spectra; full three-dimensional electron and ion distribution functions from a few eV to hundreds of keV; and various ancillary and browse products to help with spacecraft and event location. The CAA is continuing to extend and improve the online capabilities of the system and the quality of the existing data. It will add new data files for years 2006–2009 and is preparing for the long-term archive with complete coverage after the completion of the Cluster mission.

1 Appendix: CEF Metadata

The users are strongly advised to become familiar with the CEF metadata dictionary given by Harvey et al. [3] in order to get a full benefit of the metadata provided with the numerical data. This appendix points out some of the metadata details that users should be aware of. We have included an example of metadata from the FGM SPIN resolution data file on 1 January 2003, 0–1 UT, which is given in the end of this appendix.

The user should be aware that the delivered files to the users and the actual source files in the CAA database are not identical. The source files can be of any length and most of the metadata are usually given in detached header files. When the data are requested by the user, CAA software tools are used to create a file of the desired interval from the source files and the metadata are included in the file. The FILE_NAME starts with the dataset name and the requested interval followed by the version number that tells the latest ingestion date of all input data files used for the requested file, which is 2006 August 15 in the case of the example file.

The top-level metadata provides some general level information about the mission and the experiment:

• MISSION: all MISSION metadata are the same for all datasets and are provided by a detached MISSION header file.

• OBSERVATORY: There are four OBSERVATORY header files, one for each Cluster spacecraft. These metadata are the same for all datasets from the same spacecraft.

• EXPERIMENT and INSTRUMENT: these metadata provide some details of the experiment and are identical to all the datasets of one experiment on one spacecraft.

• INSTRUMENT_CAVEATS: the instrument level general caveats are given here, the relevant dataset is given here. This may include references to external files.

The important part of metadata are given at the dataset level and are global to the all files of the given dataset. Some of the useful keywords are:

• DATASET_ID is a unique identifier of the dataset, which is also used in the beginning of the filename. This ID is needed when requesting data via the command line interface.

• DATASET_TITLE is a concise description of the dataset.

• DATASET_DESCRIPTION provides a detailed description of the dataset.

• TIME_RESOLUTION is a time interval between two data points, expressed in seconds. If this is not constant, the two further keywords, MIN_TIME_RESOLUTION and MAX_TIME_RESOLUTION, must be used.

• PROCESSING_LEVEL tells the level to which the data have been processed. Possible values are: Raw, Uncalibrated, Calibrated, Derived, and Auxiliary.

• DATASET_CAVEATS provides miscellaneous information about the dataset. This may include references to external files.

• LOGICAL_FILE_ID is the same filename.

• VERSION_NUMBER is the date of the latest ingested file used to create the requested file.

• DATASET_VERSION lists all the individual files used to produce the requested file.

• FILE_CAVEATS provides specific caveats of individual files. This may include references to external files.

• FILE_TYPE is normally CEF but for plots and some other files it can be something else. In principle any type of file including binary files can be included in the CAA database as long as a proper detached header is provided.

• FILE_TIME_SPAN tells the requested time interval.

Each parameter is described with a number of useful keywords where the most important ones are:

• CATDESC provides a concise description of the parameter.

• COORDINATE_SYSTEM provides an acronym of a coordinate system (for vectors, tensors or their components). Allowed acronyms are provided by Harvey et al. [3].

• DELTA PLUS (DELTA MINUS) provides the number of units (of the parameter) to be added to (or be subtracted from) the nominal value to obtain the upper (or lower) limit of the parameter interval within which the data was acquired. This is used in particular with time tags to provide the time interval where the data value is valid.

• DEPEND_0 ties explicitly the parameters to the independent variable parameter(s) on which it depends.

• FIELDNAM describes the parameter and which can be used, for example, to give a title for a plot (use LABLAXIS for the axis label). This could be the same as CATDESC. Note, however, that while FIELDNAM is optional, CATDESC is not.

• FILLVAL is used to replace bad or missing data.

• FRAME VELOCITY is important to the physical parameters whose values depend upon the motion of the coordinate system in which they are measured. Three possible values are accepted: Observatory (default); Inertial (i.e. GEI); Earth-Corotating.

• LABEL_i is used for multi-dimensional parameters to label individual components (as LABLAXIS is not sufficient to describe the variables).

• LABLAXIS can be used to label the y-axis of a plot or to provide a column heading for a data listing. For more than one-dimensional parameters, the labels can be given in LABEL_i.

• QUALITY provides a quality of the parameter, with the following values: 0 – Not applicable; 1 – Known problems, use at your own risk; 2 – Survey data, not for publication; 3 – Good for publication, subject to PI approval; 4 – Excellent data which has received special treatment.

• REPRESENTATION is used to provide essential information describing vectors and tensors; for details, see Harvey et al. [3].

• SI_CONVERSION factor is required to take the archived value to the corresponding value in a standard SI unit. For instance, “1. 0E − 9 > T” is used to transform the magnetic field data stored in nanotesla (nT) into tesla (T).

• SIGNIFICANT_DIGITS provides the number of decimal digits required to preserve the precision of the parameter and can have a value of any positive integer.

• SIZES provides the dimensions of the array that is required for any parameter represented by more than one component (vectors, tensors, spectral arrays, etc.). For example, SIZES = 3 is for a vector of 3 components.

• TENSOR_ORDER: 0 for scalars (default value), 1 for vectors (e.g., magnetic field or velocity), 2 for tensors (e.g., plasma pressure).

• UNITS: this provides the unit of the parameter that can be indicated on the axes of a plot. If parameter has no unit, this must be specified by UNITS = “unitless”.

• VALUE_TYPE provides identification of the value type (essential for ASCII conversion) with possible values: CHAR; DOUBLE; FLOAT; INT; ISO_TIME; ISO_TIME_RANGE.

The following example shows the metadata for the FGM SPIN resolution data file on 1 January 2003, 0–1 UT, requested on 17 February 2009:

Table 2