|
MISR Level 2 Top-of-Atmosphere/Cloud Products Quality Statement |
This statement applies to MISR Level 2 TC Stereo(F07_0011), Albedo(F03_0006) and Classifiers(F03_0004), and beyond until further improvements to MISR software are made. Quality statements covering earlier time periods may be accessed through links at the bottom of this page.
The evaluation of the product quality is on-going. Please read the summary words of caution if you have not done so already.
Many of the algorithms used in the product retrievals have been developed specifically for the MISR instrument, and as such, are relatively untested. Trade-offs with the stereo-matching algorithms have been made at times to sacrifice accuracy or coverage for speed.
In spite of all the warnings, the MISR Level 2 TC Stereo, Albedo and Classifiers software which generated these products is believed to be functioning well except where noted below. This statement highlights major known problems with the products, as well as functionalities which are currently not implemented.
The Stereo Heights, SDCM over ocean (Stereoscopically Derived Cloud Mask), and RLRA (Reflecting Level Reference Altitude) are Stage 1 Validated. This label applies retroactively to all aforementioned products (both new and reprocessed) with a version number of F05_0008 or greater. This version of the software went into production on November 12, 2002. The cloud-motion vectors (winds) are also Stage 1 Validated for all data with a version number of F07_0011 or greater, which went operational on February 4, 2004.
Several factors affect the quality of the stereo heights including the nature of the scene being matched, the co-registration of the different cameras in Level 1, and the accuracy of the wind retrievals.
EXPECTED ACCURACY
Under good matching conditions with perfect registration (see Registration
paragraph below), the winds are consistent with the theoretical limit of 3 m/s
with a corresponding height error of 400 m. Difficulties in applying the
stereo-matchers to multi-layer or low-contrast scenes limit the accuracy of
the winds and heights. The stereo heights themselves are quantized in units of
550 m. The stereo-matchers lack subpixel accuracy and a single pixel of
disparity difference translates into 550 m of height. In general, the
stereo-matchers are accurate to within one pixel.
The mean height error found in studies of clear scenes was less than the 550 m height resolution. A study of MISR cloud-top heights vs. radar shows differences of less than 500 m (for clouds less than 3 km) with a standard deviation of less than 1.1 km. The data for higher clouds is similar, with slightly larger errors. Another study showed a median height difference of 600 m for MISR-MODIS intercomparisons. The MISR results were generally closer to the lidar data than were the MODIS. Comparing multi-layer scenes is difficult because the stereo-matchers used in the MISR retrievals will lock onto a lower, brighter cloud (the level with the most contrast) and ignore a higher, thinner one.
REFERENCES
A study of the MISR-retrieved winds against GOES data is published in the vol.
28, 2001 issue of Geophysical Research Letters (GRL). Height Comparisons made
with MODIS, radar, and computationally intensive stereo matching algorithms are
documented in the July 2002 issue of IEEE-TGARS and in the August 2002 issue of
GRL. An additional study of the MISR heights and winds is available in the
proceedings of the EUMETSAT Users' Conference (Antalya, October 2001) by G.
Seiz. Overall, there is good agreement between MISR cloud heights and those
obtained from other sources.
WIND RETRIEVAL QUALITY AND HEIGHT BLOCKINESS
The accuracy of the cloud motion retrieval is a key component in the cloud-top
height calculation. The winds are very sensitive to any misregistration of the
oblique angles, and discontinuity in the wind vectors shows up as clearly
visible "blockiness" in the stereo heights (most noticeably in the
RawWinds version of the heights - see next section). Drop-outs in the wind
vectors result in a default wind field of zero being applied and this will also
appear as blockiness in the RawWind heights. In addition, failure of the
stereo-matchers will also cause poor quality winds on occasion.
The Goddard GMAO has performed a detailed analysis of 6 weeks worth of wind data (Sept 1 to Oct 15, 2003) by comparing it to the expected streamlines derived from 6 hour forecasts. Several thousand observations per day were included in this study. The results are tabulated in the wind errors table. This analysis showed the quality of the winds over ocean to be better than those over land, probably because the patterns of the land can be seen underneath the clouds, thereby confusing the stereo matchers.
Quality flags for each individual wind vector are included in the TC product. These flags are computed by looking at the signal strength returned from the stereo-matchers and do not take any possible misregistration into account. The flags are set conservatively so it is possible for "good" winds to be labelled as bad, but very few poor quality winds should pass the quality test. An individual wind-vector should only be considered of Validated quality if the Orbit_QA flag indicates "good", the mean misregistration retrievals for the Df camera (over the entire swath) is zero, and the individual wind-qa flag indicates "good" or "very good". These misregistration retrievals may be cloud-contaminated, so one should look for a modal peak located at +/- one pixel error. The Da camera is currently not used, see the Registration section for more information.
CLOUD-TOP HEIGHT CATEGORIZATION
With the addition of the WindQA flags to the product, the StereoHeights and
all related fields are now produced in three different types - BestWinds,
WithoutWinds and RawWinds.
The BestWinds parameters are only computed for those domains in which a wind retrieval (a) succeeded and (b) was of Good or VeryGood quality. They are set to NoRetrieval otherwise. These heights comprise our best guess of what the true stereo height is for each pixel. There is still some blockiness present but it is greatly diminished from previous versions of the cloud-top heights. The WithoutWinds data are calculated assuming a constant value of zero wind everywhere. Over clear or motionless areas, the WithoutWinds will equal the actual stereo height, everywhere else they instead yield a "relative height". The blockiness due to wind discontinuity is removed and the relative variation in the heights over small areas is correct.
The RawWinds product uses all available wind retrievals regardless of their quality with a default to zero wind when no cloud motion vector is available. This is the same algorithm used in previous versions of the stereo product. Therefore the heights will be blocky due to discontinuities and drop-outs in the wind vectors. This is intended as a diagnostic field to allow assessment of the cloud-top height improvement due to the inclusion of the WindQA flags.
STEREOSCOPICALLY DERIVED CLOUD MASK
The Stereoscopically Derived Cloud Mask (SDCM) is calculated from the Stereo
Heights by comparing the cloud height with that of the terrain. If the height
is more than 562 m (the resolution of the stereo-matchers themselves) above the
terrain height, it is called a cloud. It is therefore difficult to detect low
clouds close to this altitude.
The quality of the SDCM is related to the surface type. It is more difficult to pick up thin clouds over land because the stereo-matchers will see through the cloud to the surface below and match that elevation instead. The SDCM over ice-free ocean has been studied more carefully than other surface types, and has therefore been declared Stage 1 Validated. Other surface types have not been studied as intensively and remain at Provisional status for now. It is expected that the SDCM will perform slightly worse over land, and the miniumum detectable optical depth to increase. Details about the validation of the SDCM over ice-free ocean follow.
A study of the SDCM against ARM data (for the Nauru and Manus sites, 11 thin cirrus cases, 12 clear-sky) shows that the cloud mask can detect clouds with an optical depth > 0.3, and that clear scenes are not misclassified as cloudy.
The SDCM comes in four flavours - Preliminary_SDCM (BestWinds), Preliminary_SDCM (WithoutWinds), SDCM (BestWinds), and SDCM (WithoutWinds). The "BestWinds" varieties are calculated using the corresponding BestWinds stereo heights, and similarly for the WithoutWinds. The Preliminary masks are calculated using only stereoscopically derived heights, while the "final" versions use RCCM data to fill in the missing stereo heights over clear-sky ocean.
A visual verification of the SDCM using 10 random orbits was performed for each of the four different flavours of the mask. The PrelimSDCM_BestWind contained 55% NoRetrieval values, and had a classification accuracy of 71%. The PrelimSDCM_WithoutWind had 44% NoRetrieval values with an accuracy of 69%. The SDCM_BestWind had 40% NoRetrievals and an accuracy of 76%. The SDCM_WithoutWind had 28% NoRetrievals with an accuracy of 75%.
These modest classification accuracies are primarily due to the difficulties of detecting both very low clouds, and high thin ones.
REGISTRATION OF LEVEL 1 DATA AND ORBIT QUALITY FLAG
Level 1 now includes an Orbit Quality (Orbit_QA) flag that assesses the
registration quality of the orbit based on the Terra orbit attitude and
ephemeris data quality indicators. All of the TOA/Cloud products now read in
this quality flag. If the flag indicates that the registration quality of the
orbit might be poor, all the BestWind height products in the TC_STEREO file are
set to NoRetrieval since it is impossible to retrieve good quality winds if
the registration is inaccurate. This decision is made on an orbital basis and
is flagged in both the Orbit_QA and CloudMotionSource flags.
There is also an issue with the georectification accuracy of the Da camera as compared with the other eight cameras. Da camera data acquired outside of the time period August 2000 - July 2001 occasionally exhibits a mean geolocation error of up to 800 m. Please see the Georectification Page for more details. In compensation, the wind retrieval process currently does not use the Da camera. To counteract the decreased signal entering the wind retrieval process, the wind retrieval threshold has been halved, possibly resulting in slightly noisier winds. However, the bias due to the Da camera misregistration has been removed.
The registration accuracy of the Df and Da cameras in both the along and across-track directions is also reported in the product at 70.4 km resolution. These retrievals generally succeed over clear-sky land, and calculate the misregistration of the D cameras in units of 275 m pixels. There is a known problem with mistakenly calculating a spurious misregistration over cloud-contaminated areas. Thus if one is concerned about individual misregistration retrievals, the oblique views should be checked for cloudiness to ensure that the scene is indeed clear.
MULTI-LAYER SCENES
Multi-layer scenes and those without a great deal of contrast cause problems
for the stereo-matching algorithms. The variation in cloud opacity with view
angle, in particular, makes the wind retrieval (and therefore accurate height
calculation) difficult. In such cases, MISR will match the layer of greatest
contrast, rather than the highest heights. High, thin clouds over a lower-level
cloud deck are ignored.
OTHER PROBLEMS
The stereo-matchers lack a robust blunder detection algorithm and will
therefore retrieve spurious results on occasion. This results in areas of
"noise" in the stereo height field. The scene is pre-screened for
sufficient contrast and a failure in this test results in a NoRetrieval in the
stereo heights, but sometimes low contrast scenes are matched and will result
in difficulties applying the stereo matchers correctly.
Sometimes horizontal stripes of NoRetrieval values will appear in the product. See the Exceptions/Anomalies paragraph in the Level 1 Quality Statement for more details.
DATA SOURCE FLAGS
The Orbit_QA, CloudMotionSource, WindQuality and StereoHeightSource flags all
contain key information about the source of the TC_STEREO data. Their values
are listed below. The Orbit_QA flag is contained in the global file
attributes, the others are available as gridded data fields at the appropriate
resolution.
| Orbit_QA: | -9999.0 = NoRetrieval, -1.0 = Poor Registration, 0.0 = Nominal Registration |
| CloudMotionSource: | 0=Stereo Not Attempted, 1=Wind Retrieval Failed due to poor Orbit_QA flag, 2=Stereo Attempted and Failed, 3 = Stereo Succeeded for Low Cloud only, 4 = Stereo Succeeded for High Cloud only, 5 = Stereo Succeeded for Low and High Clouds |
| WindQuality: | 0 = NoRetrieval, 1 = Bad, 2 = Uncertain, 3 = Good, 4 = VeryGood |
| StereoHeightSource: | 0= NoRetrieval, 1 = Stereoscopically Determined height, 2 = Surface Override, 3 = Default Cloud, 4 = MODIS height |
| Cloud Masks: | 0 = NoRetrieval, 1 = CloudHighConfidence, 2 = CloudLowConfidence, 3 = ClearLowConfidence, 4 = ClearHighConfidence |
FILE FORMAT UPDATES
The TC_STEREO product underwent extensive revision in November, 2002 (version
F05_0008 of the product file): most of the 1.1 km and 2.2 km resolution field
names have been changed, and they have also been re-ordered to put the most
important fields at the top of each grid. Please see the Data Products
Specifications document available from the
MISR Access Data Table for full
details.
EXTERNAL DATA SOURCES
No external data sources such as the MODIS cloud-heights and the DAO/NSIDC
snow-ice masks are used in the L2TC Processing. The snow-ice data are instead
provided by monthly, static climatological inputs from the TASC Dataset.
ALGORITHM UPDATES
There have been several updates to the algorithms described in the ATBD.
First, the 2-D histogram used for the wind retrieval now employs a
bin-expansion algorithm and we no longer average together separate wind bins,
instead we choose the bin with the smallest height range. If there is no
stereoscopically retrieved height available, the StereoHeight and SDCM are set
to NoRetrieval except in the case of clear-sky over ocean (as determined by
the value of the Radiometric Camera-by-Camera Cloud Mask - RCCM) where the
surface height is substituted for the missing stereo height. The RLRA is set to
NoRetrieval where there is no stereo height, rather than being filled in with
default values.
The ASCM is Provisional for all products with a version number of F03_0003 or greater. This version of the software went into production on August 13, 2003. The angle-by-angle and overall scene classifiers as computed from the SDCM, PreliminarySDCM, and RCCM are now of Stage 1 Validated quality (for some scene types) following the declaration of their parent products as Validated. The altitude-binned scene classifiers remain at Provisional status. The RCCM-based cloud-fractions are of Stage 1 Validated quality over water, and Beta quality over land. Similarly, the SDCM- based classifiers are Validated over ice-free ocean and Provisional elsewhere. The ASCM-based altitude-binned scene classifiers are also Provisional. The Classifiers version corresponding to the Stereo products attaining Validated status is F02_0002. This version went into production on November 12, 2002. All Classifiers data at or above this version have some products at Validated status.
REFERENCES
The physical basis of the ASCM algorithm is available in "A band-
differenced angular signature technique for cirrus cloud detection", Di
Girolamo L., and R. Davies, IEEE Trans. GeoSci. Remote Sens., vol 32, 1994.
Note that although the ASCM is referred to as a cirrus cloud mask in this
paper, it is currently being used as a general cloud mask.
CLOUD AND TOPOGRAPHIC SHADOW MASKS NOT AVAILABLE
The cloud and topographic shadow masks are currently not part of the
Classifiers product.
ANGULAR SIGNATURE CLOUD MASK
The Angular Signature Cloud Mask (ASCM) is calculated by thresholding a single
observable, namely the Band-Differenced Angular Signature (BDAS). The
thresholds for the ASCM depend on the sun-view geometry, the underlying surface
type, and season. Currently, all the thresholds are set to constant,
place-holder values based on the study of Di Girolamo and Davies (1994). These
data are scheduled to be updated later this year.
Visual inspection of 12 orbits of MISR data over 4 seasons has revealed that the current version of the ASCM overestimates the amount of clouds over snow and ice, and underestimates it elsewhere. Global cloud distribution maps from the ASCM show the known climatological cloud distributions, with the observed biases. These observations were sufficient to declare the ASCM of Provisional quality. Further quality assessment will await the delivery of a new ASCM Threshold dataset, which will be derived from the MISR data.
CLOUD CLASSIFIERS FIELDS
Since the algorithm for determining the cloud classifiers is so simple, the
quality of these products is directly derived from the incoming data.
Therefore, one is urged to pay close attention to the quality statements for
the SDCM, RCCM and ASCM. All the classifiers except the altitude-binned ones
have the same quality as their parent cloud masks. The "BestWinds"
version of the Stereo Heights and related cloud masks are used in the
calculations of all these products.
The altitude-binned classifiers remain at Provisional for now because no systematic study has been done to determine how much low cloud is being missed because of the SDCM's inability to properly detect clouds lower than the height resolution, etc. One has to take the limitations of the stereo-height retrievals into account when looking at these classifiers.
The earlier problem with the An-camera RCCM as referenced to the cloud-top heights (FR_RCCM in the Stereo product) being masked out in sunglint areas has been corrected in version F07_0011 of the stereo product. This field and the "Combined Cloud Fractions" and "RCCM_AnByHeight" data are now completely populated.
FILE FORMAT CHANGES
The TC_CLASSIFIERS product has undergone extensive revision: most of the field
names have been changed, and they have also been re-ordered to put the most
important fields at the top of each grid. Please see the Data Products
Specifications document available from the
MISR Access Data Table for full
details.
The Top-of-Atmosphere BRFs (and all accompanying parameters such as the top and side BRFs and the number of unobscured pixels), and all three texture indices are Stage 1 Validated. The Local, Restrictive and Expansive Albedos (except over snow and ice) are also Stage 1 Validated. All Albedo data with a version number of F02_0004 (which went operational November 12, 2002) or greater are at this level. This applies to both current and reprocessed data. The second and third texture indices (grey-level difference vectors) were not available until version F03_0005 of the product and they are validated as of that time. The local albedos are internally consistent regardless of the modelling method used and compare well with the BRF images, and the restrictive and expansive albedos also pass visual inspection.
The accuracy of the albedos is limited by two factors - radiometric calibration and bidirectional corrections. All information in the Calibration Quality statement applies equally well to the albedos. The main bidirectional limitation applies to clear scenes at high latitudes. At present these are not corrected for azimuthal effects and consequently grossly exaggerate the true albedo. This affects all albedo products over clear snow or ice at high latitudes.
There is no detectable bias difference between the expansive and restrictive albedos. Regionally, their rms difference is approximately 0.04 to 0.10 depending on solar zenith angle. Based on limited studies to date, the rms uncertainty due to bidirectional modeling ranges from less than 0.01 at low latitudes to 0.03 at high latitudes (cloudy scenes only).
CALIBRATION
The reader is urged to pay close attention to the quality of the radiometric
calibration as there have been some recent changes made. See the
Radiometric
Calibration section of the Level 1 Quality Statements.
CLEAR-SKY DETERMINISTIC MODELLING MISSING
The clear-sky deterministic modelling algorithm as described in the ATBD
is not yet implemented. When the scene is determined to be clear (by looking
at the SDCM), all the local albedo components are calculated using solid-angle
weighting.
FACTORS AFFECTING ALBEDO INTERPRETATION
The local albedo is defined as the unobscured reflection from the 2.2 km RLRA
and will therefore often appear to be lower than would be expected. If the
local albedos are not summarized statistically but instead are looked at on an
individual basis, only the ones that have no obscuration should be used. (This
information is available at 2.2 km resolution in the NumUnobscuredTop field.
Any number >= 64 indicates that there was no obscuration).
Small fluctuations in the value of the RLRA will directly affect the obscuration and the local albedo. For continuous scenes, a 500 m RLRA difference (due to pixel quantization and other factors) typically results in up to a 10% difference in the local albedo due to obscuration effects. This effect is generally not as noticeable in scenes with naturally discontinuous height fields.
FILE FORMAT CHANGES
The TC_ALBEDO product underwent extensive revision in November 2002, (version
F02_0004 of the product file): most of the field names have been changed, and
they have also been re-ordered to put the most important fields at the top of
each grid. Please see the Data Products Specifications document available from
the MISR Access Data Table for
full details.
LOCAL ALBEDO MODELLING ALGORITHMS
The Local Albedo calculation is first attempted by Deterministic Modelling (if
the scene is homogenous), then Stochastic Modelling and finally solid-angle
weighting. No modelling is attempted for clear-sky pixels or where the solar
zenith angle is < 25.8 degrees. The weights used in the stochastic modelling
are based on pre-launch theoretical simulations and will be updated in 2004 to
reflect real measurements.
ALGORITHM UPDATES FROM ATBD REV. D
The algorithm for reprojecting the RLRA field down to the surface ellipsoid was
found to be flawed and has been completely replaced with a new
backwards-projection algorithm that reprojects the BRF's up to the RLRA.
