GEWEX RFA Data File Format and File Naming Convention

The data and documentation supplied for participation in the GEWEX RFA must adhere to the following format and file naming requirements.

Table of Contents:

General Information File Format Satellite data products Requested Data Types Data Format Specification Sample Files Ground-based data products Requested Data Types Data Format Specification Sample Files Explanation File Naming Convention General Example file names Satellite data products Ground-based data products

Data Descriptor Tables Table 1. Data Categories Table 2. Data Subcategories Table 3. Temporal Sampling or Averaging Periods Table 4. Spatial Sampling or Averaging Areas Table 5. Parameters Table 6. Zones Table 7. Measurement sites Land-ocean mask

General Information

1. For purposes of the Radiative Flux Assessment, a data product is defined to consist of all of the files sharing the same <ProductName> and <ProductVersion> tags in their file names. (See naming convention below.) A single data product may consist of numerous parameters, but all of these should be related, being produced by the same algorithm, institution, etc. Selection of the data product name is left to the discretion of the data provider, however the name should be clearly related to the name of the original archived data set. If the data is not a subset of a currently available public/archived data set (e.g., a new calibration has been applied, etc.), please append -MOD at the end of the product name. Examples of typical data products and possible RFA-style names for them are shown below.

   Data Product	<ProductName>_<ProductVersion>
   ISCCP FD TOA and surface data	ISCCP-FD_Ed000
   NASA/GEWEX SRB version 2.7, SW fluxes from Pinker algorithm	GEWEX-SRBGSW_Ed027
   CERES Monthly TOA/Surface Averages from Terra, incorporating data from geostationary satellites, Edition 2d including the CERES Rev. 1 correction	CERES-SRBAVG-Terra-GEO-MOD_Ed02d
   Official BSRN surface measurement data	BSRN_Ed001

2. A single data description file, containing data quality information and references to complete documentation, will be stored for each data product. Each time data is added to, removed from, or modified in the file set for a product, an updated documentation file must be submitted. The name of this file must be <ProductName>_<ProductVersion>.txt, following the convention given below.

3. Sample images are to be in GIF, JPG, PNG, or TIFF format with the appropriate extension.

4. Data are to be written in ASCII according to the formats specified below. Data files must have the extension .asc.

5. Each data file will contain a single parameter as described in Table 5 below.

6. Unless otherwise specified, all sampling/averaging times are to be givenin UTC.

7. The date is defined to change at 0:00 hours whether time is specified in LST or UTC. Note that, for LST, the date is set relative to the 180° meridian, not the International Date Line.

8. "Monthly" data should correspond to actual calendar months, including 29 days in leap-year Februaries.

9. Time series files containing 15-minute data should start at the top of an hour to avoid ambiguity in the file name time tag. Averaging intervals for these data are 0-14, 15-29, 30-44, and 45-59 minutes in each hour.

File Format

Satellite data products (surface or top of atmosphere)

Requested Data Types

While data providers are free to submit data of any or all types defined in this document, it is recommended that at least the following be supplied for each satellite data product:

• 2.5° global maps
• Global mean time series
• At least some zonal (or regional) mean time series

Data Format Specification

• All satellite data files are to be written in ASCII format, one F10.3 value per line. In place of missing values, use -9999.000.

• Map data must must have global coverage and be provided on a 2.5° latitude/longitude equal-angle grid. The grid begins at the South Pole and the International Date Line (-90°, -180°) and increments eastward and then northward for a total of 10368 global grid cells. The first grid cell (cell 1) is bounded by latitude -90° to -87.5° and longitude -180° to -177.5°. The second grid cell (cell 2) is bounded by latitude -90° to -87.5° and longitude -177.5° to -175°. The last grid cell (cell 10368) is bounded by latitude 87.5° to 90° and longitude 177.5° to 180°. The center points of the grid cells are listed for reference.

• Satellite values at scales greater than one day should represent averages over the entire interval selected (from Table 3). For scales less than one day (including monthly-diurnal), satellite values may be averaged or instantaneous, but this should be indicated in the documentation file.

Sample Files:

• Satellite data description file: SAMPLE_EdXXX.txt
• Satellite map data: SAMPLE_Ed021_SFC-MAP-MON-GLOB-ALWUP_1983079999_RFA01.asc
• Satellite time series data: SAMPLE_EdXXX_SFC-TS-MON-E13-ASWDN_2001019999-2002129999_RFA03.asc

Ground-based measurement data

Requested Data Types

• Long-term low temporal resolution: These files should cover a minimum of 5 continuous years.
  • Monthly averaged in UTC or LST
  • Daily averaged in UTC or LST
  • Monthly averaged 15-minute (diurnal) in UTC or LST
• Short-term high temporal resolution: These files should span the year 2004.
  • 15-minute averaged in UTC

Data Format Specification

• All ground-based measurement data files must be in ASCII and use the following format for each line:

  F10.3, 1x, F13.4, 1x, I6, 1x, I6, 1x, I6, 1x, F10.3

  where the data formats correspond to the following variables:

  • mean flux
  • time stamp of the form YYYYMMDD.hhmm
  • number of measured values used to compute the mean (and standard deviation)
  • number of fill values used to compute the mean (and standard deviation)
  • total possible number of values for these computations
  • standard deviation of the averaged values

Sample Files:

• Surface measurement data description file: EXAMPLE_Ed003.txt
• Surface measurement data: EXAMPLE_Ed003_MEA-TS-MIN15-FPE-ASWDHEM_2004010100-2004123123_RFA00.asc

Explanation

• Time conventions

  Values given in ground site files should be averaged over the period selected from Table 3, not instantaneous samples.

  The chief goal of the LST averages is to have both sunrise and sunset on the same day. For this reason, data in local solar time will also be accepted in this category. The time system used should be noted in the data documentation file.

  In the time stamp, hh and mm are not decimal times but just the standard hour and minute separated from the date by a period. Any unused digits should be filled with 9s, i.e., 20021099.9999 is a possible time stamp for monthly data, 20021030.9999 for daily data, 20021099.1415 for monthly mean diurnal data, and 20041030.1415 for 15-minute data.

  For both types of 15-minute data, the time stamp used should correspond to the BEGINNING of the averaging period. Thus the choices for the minute portion of the time stamp are 00, 15, 30, and 45. This standard is NOT meant to imply that data collection for the first quarter hour must cover the period hh:00:00-hh:14:59. Deviations of up to a minute on either end of the averaging period will be allowed.

• Other items in format

  We realize that flux measurements are not made to mW m^-2 precision. Please add trailing zeros to fill in unused decimal places.

  Time stamps are required. Despite this, no gaps are to occur in files, i.e., every possible sample time should have a value. If no credible value is available for a given time, the fill value of -9999.000 should be substituted.

  No QC flags will be provided. It is up to the data provider to determine whether the values they have are credible or not; if not, do not include this data.

  Inclusion of the standard deviation is optional. If standard deviations are not included, please substitute the standard fill value of -9999.000 in this position.

  Whether standard deviations are provided or not, a breakdown of the number and type of sample points used to determine the mean as well as the total possible points for the interval should be provided. If these numbers are not available, please substitute the fill value -9999 in these positions.

File Naming Convention

Data files must adhere to the following file naming conventions. Use the tables below for specific fields within the file name:

<ProductName>_<ProductVersion>_<DataDescriptors>_<TimeTag>_<RFAVersion>.<Extension>

where

• <ProductName>: Unique product name selected by data provider. This name should be clearly related to the name of the original archived data set. If the data is not a subset of the currently available public/archived data set (e.g., a new calibration has been applied, etc.), please append -MOD at the end of the product name.
• <ProductVersion>: Version of data product, in format Edccc, where ccc is an alphanumeric string (letters and numbers only), or EdXXX if not applicable.
• <DataDescriptors>: Concatenation of the following fields separated by hyphens (-)
  • DataCategory (Table 1)
  • DataSubCategory (Table 2)
  • TimeInterval (Table 3)
  • SpatialArea (Table 4)
  • Parameter (Table 5)
• <TimeTag>:
  For maps, time of validity in the format YYYYMMDDHH. (For data averaged over a precession cycle, use the averaging start time.)
  For time series, start and end times of data in the format YYYYMMDDHH-YYYYMMDDHH, where
  • YYYY - 4 digit year
  • MM - 2 digit month, 01 through 12, 99 if not applicable
  • DD - 2 digit day, 01 through 31, 99 if not applicable
  • HH - 2 digit hour, 00 through 23, 99 if not applicable

  (For data averaged over a precession cycle, use the start time of the first averaging period and the end time of the last period. Time series files containing 15-minute data should start at the top of an hour to avoid ambiguity in this tag.)

• <RFAVersion>: Submittal version, in format RFAnn, where nn is a zero-filled integer. Increment nn by 1 each time a given set of data is resubmitted (due to formatting or submission errors).
• <Extension>: Allowed values are txt, asc, png, gif, jpg, tiff, tif.

Example file names:

Satellite and model product file names:

1. TOA monthly gridded all-sky SW up data for January 2002 from CERES TISA:
   CERESTISA_Ed01a_TOA-MAP-MON-GLOB-ASWUP_2002019999_RFA01.asc
2. Sample image for same TOA monthly gridded all-sky SW up data:
   CERESTISA_Ed01a_TOA-MAP-MON-GLOB-ASWUP_2002019999_RFA01.png
3. ISCCP gridded daily average net surface flux for 1 May 1999 using experimental calibration values (unreleased data):
   ISCCP-MOD_Ed031_SFC-MAP-DAY-GLOB-ATOTNET_1999050101_RFA02.asc
4. Surface clear-sky LW up hourly time series for the tropical zone, 1999-2003:
   SRBQSW_Ed025_SFC-TS-HRY-20S20N-CLWUP_1999010101-2003123124_RFA01.asc
5. Time series of daily-averaged clear-sky surface albedo at Barrow, Alaska, for January 1994:
   CERESPM2_Ed02b_SFC-TS-DAY-BAR-CALB_1994010199-1994013199_RFA02.asc
6. Time series of TOA monthly-averaged 3-hourly diurnal clear-sky LW down for 2000:
   ERA40_Ed026_TOA-TS-MOD3-GLOB-CLWDN_2000019999-2000129999_RFA01.asc

Ground-based surface flux measurement file names:

1. Monthly time series of all-sky SW down for 2003 at Barrow, Alaska, from BSRN:
   BSRN_EdXXX_MEA-TS-MON-BAR-ASWDN_2003019999-2003129999_RFA01.asc
2. Hourly time series clear-sky LW up for August 2003 at Boulder SURFRAD site:
   SURFRAD_Ed031_MEA-TS-HRY-BOS-CLWUP_2003080101-2003083124_RFA04.asc

Data Descriptor Tables