CALIPSO Quality Statements:
Lidar Level 2 Cloud and Aerosol Layer Products
Version Release: 2.01

In the very best case, lidar surface elevations are as reliable as the DEM. GTOPO30 tends to be very reliable over oceans, and considerably less so in rugged terrain such as in the Andes mountains of Peru. However, due to aberrations in the signal caused by a non-ideal transient response in the 532 nm detectors, the geometric thickness associated with the lidar surface elevation (i.e., surface top - surface base) can be extremely misleading. IMPORTANT: At present, users should treat ALL signal beneath the reported lidar surface elevation top as being pure instrument artifact introduced by the non-ideal transient response of the detectors. No geophysical significance should be attributed to the (apparent!) subsurface portion of the lidar return.

Bits 1, 2, and 3 indicate the horizontal resolution at which the surface was detected:

• 0 = not detected
• 1 = detected at 1/3-km averaging
• 2 = detected at 1-km averaging
• 3 = detected at 5-km averaging
• 4 = detected at 20-km averaging
• 5 = detected at 80-km averaging

Bits 4 and 5 are not used and are set to zero. Taken together, bits 6, 7, and 8 report the 5-km detection frequency:

• 0 = detection frequency = 0%
• 1 = detection frequency = 20%
• 2 = detection frequency = 40%
• 3 = detection frequency = 60%
• 4 = detection frequency = 80%
• 5 = detection frequency = 100%

The uncertainties associated with detection of cloud and aerosol layers in backscatter lidar data are examined in detail in Section 5 of the CALIPSO Feature Detection ATBD (PDF). The ATBD contains quantitative assessments of feature finder performance derived using simulated data sets, for which all layer boundaries were known exactly. In the real world of layer detection, we do not have access to this underlying truth. Therefore in this document we provide the following set of "rules of thumb" that users can apply to the data products to obtain a qualitative understanding of the layer boundaries reported, and of the optical properties associated with these layers.

1. Strongly scattering features are easier to detect than weakly scattering features. The scattering intensity of each layer is reported in the 532 nm and 1064 nm attenuated backscatter statistics and by the integrated attenuated backscatter at 532 nm and 1064 nm.

2. Detection of layers during the nighttime portion of the orbits is more reliable than during the daytime portion of the orbits. Due to solar background signals, the noise levels in the daytime measurements are much larger than those at night, and this additional noise can obscure faint features, and can lead to boundary detection errors even in more strongly scattering layers.

3. Features become increasingly difficult to detect with increasing optical depth above feature top. Put another way, detection of the lower layers in a multi-layer scene is made more difficult by the signal losses that occur as the laser light passes through the upper layers. (In a sense, this is a restatement of (a), since the backscatter intensity of secondary features is reduced from what it otherwise might be by the signal attenuation caused by the overlying features.)

4. In general, our confidence in the location of the top of a layer is somewhat greater than our confidence in the location of the base of the same layer. For transmissive features, one reason for this is that the backscatter signal is attenuated by traversing the feature, thus degrading the potential contrast between feature and "non-feature" at the base. Additionally, in strongly scattering layers, multiple scattering effects and signal perturbations introduced by the non-ideal transient response of the 532 nm detectors can also make base determination less certain.

5. For opaque features that completely attenuate the backscatter signal, the base altitude reported must be considered as an "apparent" base rather than a true base.

6. Stratospheric features reported during daylight -- especially those reported between 60N and 60S -- should be treated with extreme suspicion.

Users should be aware that the opacity flag does not indicate that an individual layer is opaque. Instead, it identifies the layer in which the backscatter signal becomes completely attenuated (i.e., indistinguishable from the background signal level), so that for those features having an opacity flag of 1, the reported base altitude must be considered as an apparent base, rather than a true base. Furthermore, as noted in section 7.4 of the CALIPSO Feature Detection ATBD (PDF), the identification of any layer as "opaque" depends on the amount of averaging applied to the signal prior to initiating the layer detection algorithm. Data users intending to employ the opacity flag in their analyses are strongly advised to consult the feature detection ATBD (PDF).

Because all features reported in 1/3-km and 1-km layer products are detected at a single horizontal averaging resolution (i.e., either at 1/3-km or 1-km), the opacity flag is not reported. When using these products, opacity, in the sense described above, can be assessed as follows. If the surface was detected (i.e., the lidar surface altitude field does not contain fill values) then there are no opaque layers in the column. If the surface was not detected, then the lowest layer in the column is considered to be opaque.

For the uppermost layer in any column, the quality of the estimate for γ′₅₃₂ is determined by the accuracy of the top and base identification, the reliability of the 532 nm channel calibrations, and by the signal-to-noise ratio (SNR) of the backscatter data within the layer. For layers beneath the uppermost, the quality of our estimate for γ′₅₃₂ also depends on either obtaining an independent estimate of the two-way transmittance, T², for all overlying layers, or by estimating this quantity directly from the lidar backscatter data. Estimating T² directly from the data is something of a black art. In tractable situations (i.e., where there exists an extended region of "clear air" between successive layers, and where the uppermost layer has no more than a moderate optical depth of -- say -- 1.0 or less), the calculation can be fairly reliable. In especially awkward situations (e.g., vertically adjacent layers, such as clouds embedded in aerosols), the only way to estimate T² is to compute a full extinction retrieval for the profile being examined. Furthermore, the effects of errors caused by misestimating T² can increase sharply as the optical thickness above a layer increases. We note that the CALIOP processing scheme always attempts to correct estimates of γ′₅₃₂ for the attenuation imparted by previously identified overlying features. As a consequence, we will occasionally report unrealistically large values for γ′₅₃₂.

There are occasions (e.g., in regions of especially low SNR) where the uncertainty calculation can fail. In these cases, the value recorded in the data product will be set to negative one (-1). In all other cases, uncertainty values will be positive.

As is the case for γ′₅₃₂, in the uppermost layer within any column, the quality of the estimate for γ′₁₀₆₄ is determined by the accuracy of the top and base identification, the reliability of the 1064 nm calibration constant, and by the signal-to-noise ratio (SNR) of the backscatter data within the layer. In layers beneath the uppermost, γ′₁₀₆₄ will be underestimated by a factor equal to the total particulate two-way transmittance, T², above the layer. In contrast to the techniques applied at 532 nm, reliable estimates of T² cannot be derived from an analysis of the 1064 nm backscatter signal in the (assumed to be) clear air regions.

There are occasions (e.g., in regions of especially low SNR) where the uncertainty calculation can fail. In these cases, the value recorded in the data product will be set to negative one (-1). In all other cases, uncertainty values will be positive.

The quality of the estimate for δ_layer is determined by the accuracy of the top and base identification, the reliability of the polarization gain ratio calibration, and by the signal-to-noise ratio (SNR) of the backscatter data within the layer. In general, the CALIOP δ_layer estimates are highly reliable. Histograms of δ_layer compiled for midlatitude cirrus in the northern hemisphere compare very well with previously reported distributions (e.g., Sassen & Benson, 2001)

There are occasions (e.g., in regions of especially low SNR) where the uncertainty calculation can fail. In these cases, the value recorded in the data product will be set to negative one (-1). In all other cases, uncertainty values will be positive.

In regions with acceptable SNR, the accuracy with which the range resolved depolarization ratios can be determined will depend almost entirely on the accuracy of the polarization gain ratio calibration.

The quality of the estimate for χ′_layer is determined by the accuracy of the top and base identification, the reliability of the 532 nm calibration constant and the 1064 nm calibration constant, and by the signal-to-noise ratio (SNR) of the backscatter data within the layer.

There are occasions (e.g., in regions of especially low SNR) where the uncertainty calculation can fail. In these cases, the value recorded in the data product will be set to negative one (-1). In all other cases, uncertainty values will be positive.

Users can have high confidence in the calculation of all of the values in the attenuated total color ratio statistics fields. However, as with the 532 nm depolarization ratio statistics, the meaning of the various numbers can be somewhat misleading. Like the depolarization ratios, the attenuated total color ratios are produced by dividing one noisy number (the 1064 nm attenuated backscatter coefficient) by a second noisy number (the 532 nm attenuated backscatter coefficient). Depending on the noise in any pair of samples, the resulting values can range from large negative values to extremely large positive values. When computing layer means, standard deviations, and centroids, these outliers can dominate the calculation, and thus return entirely unrealistic estimates.

γ′_above provides a qualitative assessment of the confidence that users should assign to each layer reported. As noted earlier (see the discussion for layer base and top heights), layer detection, and the assessment of the associated layer descriptors, becomes increasingly uncertain as the overlying optical depth increases. This uncertainty cannot be easily quantified, because backscatter lidars such as CALIOP cannot measure optical depth directly, and must instead derive optical depth estimates in subsequent data processing. However, γ′_above can easily be obtained directly from the calibrated backscatter signal, and hence can provide a qualitative proxy for the optical depth above each layer detected.

Correct interpretation of the feature subtype bits depends on the status of the feature type; e.g., the interpretation is different for clouds and aerosols. For aerosols, the feature subtype is one of eight types: desert dust, biomass burning, background, polluted continental, marine, polluted dust, other, and 'not determined'. Desert dust is mostly mineral soil. Biomass burning is an aged smoke aerosol consisting primarily of soot and organic carbon (OC), clean continental (also referred to as background or rural aerosol) is a lightly loaded aerosol consisting of sulfates (SO₄^2-), nitrates (NO₃^-), OC, and Ammonium (NH₄⁺), polluted continental is background aerosol with a substantial fraction of urban pollution, marine is a hygroscopic aerosol that consists primarily of sea-salt (NaCl), and polluted dust is a mixture of desert dust and smoke or urban pollution. An aerosol will be classified as 'not determined' only in those cases where the classification scheme fails; as yet, no such cases have been identified in the version 2.01 release. The 'other' designation is a place-holder for another, yet to be determined, aerosol type. While this set does not cover all possible aerosol mixing scenarios, it accounts for a majority of mesoscale aerosol layers. In essence the algorithm trades off complex transient multi-component mixtures for relatively stable layers with large horizontal extent (10-1000 km).

The standard CAD scores reported in the CALIPSO layer products range between -100 and 100. The sign of the CAD score indicates the feature type: positive values signify clouds, whereas negative values signify aerosols. The absolute value of the CAD score provides a confidence level for the classification. The larger the magnitude of the CAD score, the higher our confidence that the classification is correct. An absolute value of 100 therefore indicates complete confidence. Absolute values less that 100 indicate some ambiguity in the classification; that is, the scattering properties of the feature are represented to some degree in both the cloud PDF and in the aerosol PDF. In this case, a definitive classification cannot be made; that is, although we can provide a "best guess" classification, this guess could be wrong, with a probability of error related to the absolute value of the CAD score. A value of 0 indicates that a feature has an equal likelihood of being a cloud and an aerosol. Users are encouraged to refer to the CAD score when the cloud and aerosol classification results are used and interpreted. Beginning with the version 2.01 release, several additional CAD score values have been added. Each of these new values represents a classification result that is based on additional information beyond that normally considered in the standard CAD algorithm.

The accuracy of the CAD score depends on how accurately the PDFs approximate the cloud and aerosol distributions found in the real world, and on how completely clouds and aerosols are separated in the selected attribute space (i.e., by the attributes of layer averaged attenuated backscatter, color ratio, and layer altitude). The PDFs incorporated into version 2.01 of the CAD algorithm were developed based on expert manual classification of all layers detected during one full day of data acquired by CALIOP during August 2006. From these results, a single set of cloud and aerosol PDFs was constructed. This set of PDFs is applied globally for all seasons and at all latitudes. Using the standard algorithm alone produced an unacceptable level of misclassifications for very dense aerosols at low latitudes and optically thin low clouds in the polar regions. To improve the classifications of features in these regions, an additional depolarization ratio threshold criteria has been incorporated into the current algorithm.

• For the vast majority of cases, CALIOP cannot provide a direct measurement of layer optical depth. In these cases, estimates of optical depth are derived using extinction-to-backscatter ratios (i.e., lidar ratios) that are specified based on an assessment of layer type and subtype. Uncertainties in the value of the lidar ratio, which can arise both from natural variability and from occasional misclassification, propagate non-linearly into subsequent estimates of layer optical depth.

• Retrievals of optical depth from space-based lidar measurements must account for contributions from multiple scattering that are generally considered negligible in ground-based and aircraft based measurements. The theoretical basis for CALIPSO's treatment of multiple scattering is provided in the extinction retrieval ATBD (PDF) and in Winker, 2003 (PDF)

• Similar to the layer detection problem, estimates of layer optical depth become increasingly fraught with error in multiple layer scenes, as errors incurred in overlying layers are propagated into the solutions derived for underlying features.

• IMPORTANT NOTICE: before proceeding, all users of the CALIOP optical depth data should read and thoroughly understand the information provided in the Profile Products Data Quality Summary. This summary contains an expanded description of the extinction retrieval process from which the layer optical depths are derived, and provides essential guidance in the apppropriate use of all CALIOP extinction-related data products.

Despite these caveats, users should not be unduly pessimistic about the quality and usability of the CALIPSO optical depth estimates. Figure 2 (below) shows a preliminary comparison of CALIPSO aerosol optical depths with the optical depths derived from MODIS for nighttime measurements acquired during January 2007. The comparison is generally good, with MODIS appearing to slightly over-estimate values at the lower end of the optical depth range.

Figure 2: Comparison of CALIPSO aerosol optical depths to those derived from MODIS (Preliminary - January 2007, nighttime only) final lidar ratio = initial lidar ratio only)

The aerosol lidar ratios used for the CALIOP analyses represent well established mean values that are characteristic of the natural variability exhibited for each aerosol species (e.g., see Omar et al., 2005 (PDF); Cattrall et. al., 2005; and Figure 3 below). The clear implication of this natural variability is that even for those cases where the aerosol type is correctly identified, the initial lidar ratio represents an imperfect estimate of the layer-effective lidar ratio of any specific aerosol layer. These same caveats apply equally to the mean values used for the initial cloud lidar ratios. For all layer types, cloud-aerosol discrimination errors can exacerbate the error associated with the specification of the initial lidar ratio. Uncertainty can also be introduced by the cloud ice-water phase classification and the aerosol subtype identification procedures. However, the CALIOP extinction algorithm incorporates some error-correcting mechanisms that in many cases will adjust the initial estimate of lidar ratio so that a more suitable value is ultimately used in the retrieval. Details of the lidar ratio adjustment scheme are provided in the extinction retrieval ATBD (PDF). Algorithm architectural information and generalized error analyses for CALIPSO's cloud-aerosol discrimination algorithms, cloud ice-water phase algorithms, and aerosol subtyping algorithms can be found in the CALIPSO Scene Classification ATBD (PDF).

Figure 3: Distributions for AERONET-derived lidar ratios, computed for aerosol types described in Omar et al., 2005 (PDF);

For weakly scattering features, the lidar ratio is most often left unchanged by the extinction solver, as a physical solution is usually obtained on the first iteration. In these cases, the uncertainties in the final lidar ratio are the same as the uncertainties in the initial lidar ratio. The exception to this statement would be if either the cloud-aerosol discrimination or the layer sub-typing procedures have misclassified the layer. However, for weak layers, the relative error in the lidar ratio is (approximately) linearly related to the resulting error in the derived optical depth estimate.

Retrievals of opaque and strongly scattering layers are very sensitive to the initial lidar ratio selection. Too large a value will cause the retrieval algorithm to, in effect, extinguish all available signal before reaching the measured base of the feature. When that point is reached the retrieval becomes numerically unstable and the calculated extinction coefficients will asymptote toward positive infinity. In these cases, a successful solution can only be obtained by reducing the lidar ratio. The CALIOP extinction routine does this automatically, and will repeat the process until a stable solution is achieved. When this happens, the final lidar ratio reported in the data products is the first one for which a physically meaningful (albeit not necessarily correct) solution was obtained for the entire measured depth of the layer.

The optical depths and extinction profiles derived in those cases where the layer lidar ratio must be reduced are generally not accurate. The current lidar ratio reduction scheme terminates after identifying (a very close estimate of) the largest lidar ratio for which a physically meaningful solution can be generated for the backscatter measured in the layer. However, the optical depths and extinction profiles reported in these situations can only be considered as upper bounds; the true values are somewhat, or perhaps even significantly, lower. Because the associated uncertainties cannot be reasonably estimated, these data should be excluded from statistical analyses of layer optical properties, and even the most sophisticated users are advised to treat theses cases with extreme caution.

When an independent estimate of layer optical depth is available from a measured layer two-way transmittance, the CALIOP extinction algorithm will retrieve the optimal estimate of the layer-effective lidar ratio, irrespective of layer type, and use this retrieved lidar ratio in the extinction retrieval. These so-called 'constrained' retrievals are more accurate than unconstrained retrievals. For constrained retrievals, the uncertainty in the final lidar ratio can be well estimated using equation 7.4 from the CALIPSO Scene Classification ATBD (PDF). An extinction QC value of 1 indicates a successful constrained retrieval.

The bit assignments are additive, so that (for example) an extinction QC value of 18 represents an unconstrained retrieval (bit 1 is NOT set) for which the lidar ratio was reduced to prevent divergence (+2; bit 2 is set), and for which the feature finder has indicated that the layer is opaque (+16; bit 5 is set). For the version 2.01 release, bits 10-15 are not used. Complete information about the conditions under which each extinction QC bit is toggled can be found in the CALIPSO Extinction Retrieval ATBD (PDF)