|
MISR Level 2 Top-of-Atmosphere/Cloud Products Quality Statement |
Stereo: Catherine Moroney, Roger Davies, Larry Di Girolamo
Classifiers: Catherine Moroney, Roger Davies, Larry Di Girolamo
Albedo: Catherine Moroney, Roger Davies
This statement applies to MISR Level 2 TC Stereo(F07_0013), Classifiers (F04_0006), and Albedo (F04_0007) and beyond until further improvements to MISR software are made. Quality statements covering earlier time periods may be accessed through links at the bottom of this page.
The evaluation of the product quality is on-going. Please read the summary words of caution if you have not done so already.
Many of the algorithms used in the product retrievals have been developed specifically for the MISR instrument, and as such, are relatively untested. Trade-offs with the stereo-matching algorithms have been made at times to sacrifice accuracy or coverage for speed.
In spite of all the warnings, the MISR Level 2 TC Stereo, Albedo and Classifiers software which generated these products is believed to be functioning well except where noted below. This statement highlights major known problems with the products, as well as functionalities which are currently not implemented.
The Stereo Heights are now Stage 2 validated following improvements in the Wind Quality Assessment flags and a study of the stereo heights over ARM sites. The Reflecting Level Reference Altitude (RLRA) is also at this level. This label applies to all products with a version number of F07_0011 or greater. This version of the software went into production on Feb 4, 2004.
The Stereoscopically Derived Cloud Mask (SDCM) for all scene types and Cloud Motion Vectors (winds) are Stage 1 Validated for version F07_0011 of the products.
Several factors affect the quality of the stereo heights including the nature of the scene being matched, the co-registration of the different cameras in Level 1, and the accuracy of the wind retrievals.
The MISR cloud-top height retrieval is conceptually simple. An object located above the surface (such as a cloud) will appear in two different positions when viewed from different angles. The apparent change in position (parallax or disparity) as measured by pattern- matching software can be used to calculate the cloud height. The cloud motion vectors (winds) are retrieved first using three angles, then the heights are calculated using the disparity measured between two cameras and the knowledge of the winds.
The winds are calculated at 70.4km resolution and are assumed to be constant over the region. Changes in the estimated wind vectors across the domain boundaries lead to a blocky appearance in the heights at the same resolution. The Stereo Heights and Stereoscopically Derived Cloud Mask (SDCM) are calculated at 1.1 km resolution.
The SDCM is directly derived from the corresponding cloud height. If the stereo height is more than 560 m (the resolution of the stereo heights) above the terrain, the pixel is designated as cloudy. The SDCM is unable to detect clouds over land that are below this altitude.
With the addition of the WindQA flags to the product, the StereoHeights and all related fields are now produced in three different types - BestWinds, WithoutWinds and RawWinds.
The BestWinds parameters are only computed for those domains where there was a successful wind retrieval of Good or VeryGood quality. They are set to NoRetrieval otherwise. These data comprise our best guess of what the true stereo height is for each pixel. There is still some blockiness present but it is greatly diminished from previous versions of the data. The WithoutWinds data are calculated assuming a constant value of zero wind everywhere. Over clear or motionless areas, the WithoutWinds will equal the actual stereo height, everywhere else they instead yield a "relative height". The blockiness due to wind discontinuity is removed and the relative variation in the heights over small areas is correct.
The RawWinds product uses all available wind retrievals regardless of their quality with a default to zero wind when no wind measurement is available. This is the algorithm used in previous versions of the stereo product. These heights are blocky due to discontinuities and drop-outs in the wind vectors. This is intended as a diagnostic field to allow assessment of the cloud-top height improvement due to the inclusion of the WindQA flags.
Under good matching conditions with perfect registration (see Registration paragraph below), the winds are consistent with the theoretical limit of 3 m/s with a corresponding height error of 400 m. Difficulties in applying the stereo-matchers to multi-layer or low-contrast scenes limit the accuracy of the winds and heights. The stereo heights themselves are quantized in units of 560 m. The stereo-matchers lack subpixel accuracy and a single pixel of disparity difference translates into 560 m of height. In general, the stereo-matchers are accurate to within one pixel.
The Goddard Global Modeling and Assimilation Office (GMAO) has performed a detailed analysis of 6 weeks worth of wind data (Sept 1 to Oct 15, 2003) by comparing it to the expected streamlines derived from 6 hour forecasts. Several thousand observations per day were included in this study. The results are tabulated in the wind errors table. This analysis showed the quality of the winds over ocean to be better than those over land, possibly because the patterns of the land can be seen underneath the clouds, thereby confusing the stereo matchers.
A statistical study of the winds shows there to be a 2 m/s southerly bias in the NorthSouth wind components.
A new version of the wind algorithm is currently being developed and noticeable improvements in the winds are expected with the next release of the TC_STEREO product. Results from a prototype version of the code are being analyzed now and these numbers will be reported in the next version of this statement.
Quality flags for each individual wind vector are included in the TC product. These flags are computed by looking at the signal strength returned from the stereo-matchers and do not take any possible misregistration into account. The flags are set conservatively so it is possible for "good" winds to be labelled as bad, but very few poor quality winds should pass the quality test. An individual wind-vector should only be considered of Validated quality if the Orbit_QA flag indicates "good", the mean misregistration retrievals for the Df and Da cameras (over the entire swath) is zero, and the individual wind-qa flag indicates "good" or "very good". These misregistration retrievals may be cloud-contaminated, so one should look for a modal peak located at +/- one pixel error.
Comparisons between ground-based millimeter-wave cloud radar and lidar measurements of cloud top and MISR stereo-derived cloud top heights have been conducted using data from 3 sites. One site is in the Southern Great Plains (Lamont, Oklahoma), the second in the tropical western Pacific (Nauru Island), and the third at the North Slope of Alaska (Barrow). These radars and lidars are operated as part of the U.S. Department of Energy Atmospheric Radiation Measurement (ARM) program.
The MISR height retrieval is performing largely as expected when the clouds are thick enough to be identified by the pattern matchers. A summary of results is presented in the height errors table.
For moderately optically thick clouds (5 > tau > 0.5), which may lack well-defined contrast at the cloud top, the MISR retrievals tend to identify structures within the cloud (typically 500m to 1.5km deep) or different cloud layers lower in the atmosphere. Much of the mean offset shown in these summary statistics result from these moderately thick clouds. This optical depth assessment is based on a variety of retrievals using ground based radar and lidar techniques.
The pattern-matchers used in the stereo height retrievals identify the location of greatest contrast, which is not necessarily the highest point in the cloud. MISR, along with many other instruments, has difficulty in retrieving the height of optically thin clouds. The optical depth threshold at which the algorithm identifies the thin cloud rather than the underlying object depends on the contrast of the underlying feature. The reader should take this inherent limitation of the retrievals into account when comparing MISR heights to other instruments or to model results.
For the extreme case of very dark features (such as open water) near bright features such as snow, clouds with optical depth of at least 2 and perhaps as much as 5 can be missed. When boundary layer clouds such as stratocumulus or trade cumulus are present, cirrus clouds with optical depths of 2 to 5 are sometimes not retrieved. However, for cases without lower clouds or underlying bright surfaces, thin cirrus clouds with optical depths near 0.5 (over heterogeneous land) and perhaps even smaller (over dark water) are usually detected.
The Stereoscopically Derived Cloud Mask (SDCM) is calculated solely from the Stereo Heights. If the height is more than 560m above the terrain, it is called a cloud. It is therefore difficult to detect low clouds close to this altitude. The possible values of the SDCM have been relabelled to Cloud (Low/High Confidence) and Near Surface (Low/High Confidence). There is no additional information available to determine whether a pixel with a low feature height is clear or not. The best we can say is that it's near to the surface. Additional radiometric data must be used to distinguish low clouds and fog from clear sky.
The SDCM cloud/no-cloud distinction over clear-sky ocean is good. If the SDCM says Cloudy, then the likelihood of this being true is estimated to be above 95%. This ties to the fact that the Preliminary SDCM consistently returns NoRetrieval over clear-sky ocean because there are not any features for the pattern-recognition algorithms to match. Any retrieval of the SDCM (be it Cloud or NearSurface) over ocean is likely to be cloud. The SDCM (due to the 560m resolution of the stereo heights) has been observed to over-detect clouds in broken boundary layers. This is noticeable in areas dominated by trade cumulus.
The SDCM comes in four flavours - Preliminary_SDCM (BestWinds), Preliminary_SDCM (WithoutWinds), SDCM (BestWinds), and SDCM (WithoutWinds). The "BestWinds" varieties are calculated using the corresponding BestWinds stereo heights, and similarly for the WithoutWinds data. The Preliminary masks are calculated using only stereoscopically derived heights, while the "final" versions use RCCM data to fill in the missing stereo heights over clear-sky ocean.
The SDCM has been validated using a combination of visual inspection, field campaigns and other satellite-derived cloud masks. Its accuracy is estimated at more than 90% over ice-free ocean, and greater than 85% over other surface types including snow and ice. The SDCM over land has been promoted to Stage 1 Validated based on these results and given its close derivation from the Stereo Heights which are now Stage 2 Validated.
A study of the cloud mask against ARM data for the Nauru and Manus sites in the tropical western Pacific shows that the SDCM can detect clouds with an optical depth > 0.3 to 0.5.
The accuracy of the cloud motion retrieval is a key component in the cloud-top height calculation. The winds are very sensitive to any misregistration of the oblique angles, and discontinuity in the wind vectors shows up as clearly visible "blockiness" in the stereo heights (most noticeably in the RawWinds version of the heights - see next section). Drop-outs in the wind vectors result in a default wind field of zero being applied and this will also appear as blockiness in the RawWind heights. In addition, failure of the stereo-matchers will also cause poor quality winds on occasion.
Level 1 now includes an Orbit Quality (Orbit_QA) flag that assesses the registration quality of the orbit based on the Terra orbit attitude and ephemeris data quality indicators. All of the TOA/Cloud products now read in this quality flag. If the flag indicates that the registration quality of the orbit might be poor, all the BestWind height products in the TC_STEREO file are set to NoRetrieval since it is impossible to retrieve good quality winds if the registration is inaccurate. This decision is made on an orbital basis and is flagged in both the Orbit_QA and CloudMotionSource flags.
The co-registration of the Df, Da, Bf and Ba cameras has been improved (significantly so in the case of Da) with the current release of the Level 1 software. See the Georectification Page for more details. With this improvement in the registration, the Da camera has been turned back on for its use in the stereo wind retrievals. This results in better populated wind histograms and thus less noisy winds.
The registration accuracy of the Df and Da cameras in both the along and across-track directions is also reported in the product at 70.4 km resolution. These retrievals generally succeed over clear-sky land, and calculate the misregistration of the D cameras in units of 275 m pixels. There is a known problem with mistakenly calculating a spurious misregistration over cloud-contaminated areas. Thus if one is concerned about individual misregistration retrievals, the oblique views should be checked for cloudiness to ensure that the scene is indeed clear.
Multi-layer scenes and those without a great deal of contrast cause problems for the stereo-matching algorithms. The variation in cloud opacity with view angle, in particular, makes the wind retrieval (and therefore accurate height calculation) difficult. In such cases, MISR will match the layer of greatest contrast, rather than the highest heights. High, thin clouds over a lower-level cloud deck are ignored.
The stereo-matchers lack a robust blunder detection algorithm and will therefore retrieve spurious results on occasion. This results in areas of "noise" in the stereo height field. The scene is pre-screened for sufficient contrast and a failure in this test results in a NoRetrieval in the stereo heights, but sometimes low contrast scenes are matched and will result in difficulties applying the stereo matchers correctly.
Sometimes horizontal stripes of NoRetrieval values will appear in the product. See the Exceptions/Anomalies paragraph in the Level 1 Quality Statement for more details.
The Orbit_QA, CloudMotionSource, WindQuality and StereoHeightSource flags all contain key information about the source of the TC_STEREO data. Their values are listed below. The Orbit_QA flag is contained in the global file attributes, the others are available as gridded data fields at the appropriate resolution.
| Orbit_QA: | -9999.0 = NoRetrieval -1.0 = Poor Registration 0.0 = Nominal Registration |
| CloudMotionSource: | 0 = Stereo Not Attempted 1 = Wind Retrieval Failed due to poor Orbit_QA flag 2 = Stereo Attempted and Failed 3 = Stereo Succeeded for Low Cloud only 4 = Stereo Succeeded for High Cloud only 5 = Stereo Succeeded for Low and High Clouds |
| WindQuality: | 0 = NoRetrieval 1 = Bad 2 = Uncertain 3 = Good 4 = VeryGood |
| StereoHeightSource: | 0= NoRetrieval 1 = Stereoscopically Determined height 2 = Surface Override 3 = Default Cloud 4 = MODIS height |
| Cloud Masks (SDCM): | 0 = NoRetrieval 1 = CloudHighConfidence 2 = CloudLowConfidence 3 = NearSurfaceLowConfidence 4 = NearSurfaceHighConfidence |
| Cloud Masks (RCCM, ASCM): | 0 = NoRetrieval 1 = CloudHighConfidence 2 = CloudLowConfidence 3 = ClearLowConfidence 4 = ClearHighConfidence |
| Consensus Cloud Classifier: | 0 = NoRetrieval 1 = Overcast 2 = KnownCloud 3 = KnownClear |
A study of the MISR-retrieved winds against GOES data is published in the vol. 28, 2001 issue of Geophysical Research Letters (GRL). Height Comparisons made with MODIS, radar, and computationally intensive stereo matching algorithms are documented in the July 2002 issue of IEEE-TGARS and in the August 2002 issue of GRL. An additional study of the MISR heights and winds is available in the proceedings of the EUMETSAT Users' Conference (Antalya, October 2001) by G. Seiz.
Garay M.J., Mazzoni D., Davies R., and Diner D.J. 2005. The application of support vector machines to analysis of global satellite datasets from MISR. Proceedings of the Fourth Conference on Artifical Intelligence, American Meteorological Society, San Diego, CA 9-13 January 2005.
Horvath A., Davies R., Simultaneous retrieval of cloud motion and height from polar orbiter multi-angle measurements. GRL Vol 28, No. 15. 2915-2918 August 2001.
Moroney C., Davies R., Muller JP, 2002: Operational Retrieval of cloud-top heights using MISR data. IEEE Transactions on geoscience and remote sensing. 40 (7). 1532-1540 JUL 2002.
Marchand R., Ackerman T 2004: An assessment of Multi-angle Imaging Spectroradiometer (MISR) stereo-derived cloud top heights using ground-based millimeter-wavelength cloud radars. 14th International Conference on Clouds and Precipitation. Bologna Italy 19-23 July 2004.
Seiz, G., 2003. Ground and satellite-based multi-view determination of 3D cloud geometry. PhD thesis, Institute of Geodesy and Photogrammetry, ETH Zuerich, Switzerland, 2003. IGP Mitteilungen Nr. 80.
The TC_STEREO product underwent extensive revision in November 2002 (version F05_0008 of the product file): most of the 1.1 km and 2.2 km resolution field names have been changed, and they have also been re-ordered to put the most important fields at the top of each grid. Please see the Data Products Specifications document available from the MISR Access Data Table for full details.
No external data sources such as the MODIS cloud-heights and the DAO/NSIDC snow-ice masks are used in the L2TC Processing. The snow-ice data are instead provided by monthly, static climatological inputs from the TASC Dataset.
There have been several updates to the algorithms described in the ATBD. First, the 2-D histogram used for the wind retrieval now employs a bin-expansion algorithm and we no longer average together separate wind bins, instead we choose the bin with the smallest height range. If there is no stereoscopically retrieved height available, the StereoHeight and SDCM are set to NoRetrieval except in the case of clear-sky over ocean (as determined by the value of the Radiometric Camera-by-Camera Cloud Mask - RCCM) where the surface height is substituted for the missing stereo height. The RLRA is set to NoRetrieval where there is no stereo height, rather than being filled in with default values.
The ASCM is Stage 2 Validated (over ocean), and Stage 1 Validated (over land) for all products with a version number of F04_0006 or greater. Sea-ice and snow cover over land are included in these statements. This version of the software went into production on May 13, 2005. The overall scene classifiers as computed from the SDCM, PreliminarySDCM, and RCCM are now Stage 2 Validated following the declaration of their parent products as Validated. The RCCM-based angle-by-angle cloud-fractions are of Stage 2 Validated quality over water, and Stage 1 Validated over snow-free land. Similarly, the SDCM-based classifiers are Stage 1 Validated. The ASCM-based altitude-binned scene classifiers are also Stage 2 or Stage 1 Validated depending on the surface type. The altitude-based scene classifiers are all Stage 1 Validated. The Classifiers version which has all the scene classifiers at Validated status is F04_0006. This version went into production on May 13, 2005. All Classifiers data at or above this version have all products at Validated status (with the exception of the consensus cloud classifiers which remain at Beta.
The physical basis of the ASCM algorithm is available in "A band- differenced angular signature technique for cirrus cloud detection", Di Girolamo L., and R. Davies, IEEE Trans. GeoSci. Remote Sens., vol 32, 1994. Note that although the ASCM is referred to as a cirrus cloud mask in this paper, it is currently being used as a general cloud mask.
The cloud and topographic shadow masks are currently not part of the Classifiers product.
The Angular Signature Cloud Mask (ASCM) is calculated by thresholding a single observable, namely the Band-Differenced Angular Signature (BDAS). The thresholds for the ASCM depend on the sun-view geometry, the underlying surface type, and season. The ASCM ocean and land thresholds have been updated and now contain seasonally dependant values. When the current validation is complete, this statement will be updated to present the results.
The ASCM algorithm is only valid when one of the D cameras is viewing forward-scattering (defined by scatt_angle(Dcam) <= 92 degrees). Therefore the ASCM is not calculated in the equatorial regions due to the scattering angles for both D cameras being out of range.
Global cloud distribution maps made from the ASCM show the expected climatological cloud distributions. Validation via visual inspection, field campaigns and satellite data show that the ASCM has accuracies of better than 90% for ice-free ocean and greater than 85% for other surfaces including snow and ice cover. A detailed validation is underway and the results will be reported in this statement when available.
The visual discontinuities present in previous versions of the ASCM have been removed. The final ASCM is now calculated from the individual terrain-referenced versions masks for the forward and aft viewing camera pairs. This means that the ASCM is now terrain-referenced and is on a different projection from the SDCM. Therefore, doing high-resolution pixel-by-pixel comparisons of the three cloud masks is not recommended unless one fully understands the data and the projection issues.
A new cloud classifier that takes all three individual cloud masks (RCCM, SDCM and ASCM) into account has been added to the Classifiers product as of November 2004 (version F04_0005). These masks are available at 2.2km and 35.2km resolution and classify scenes into three types: "KnownClear" signifying that no cloud at all was detected in the pixel, "KnownCloud" meaning that some cloud was detected, and "Overcast" meaning that that the scene is entirely clouded over. Additionally the maximum stereo height is computed for all regions classified as Overcast. These data have not been validated yet and are therefore not suitable for scientific use. Their quality is Beta.
Since the algorithm for determining the cloud classifiers is so simple, the quality of these products is directly derived from the incoming data. Therefore, one is urged to pay close attention to the quality statements for the SDCM, RCCM and ASCM. All the classifiers except the altitude-binned ones have the same quality as their parent cloud masks. The "BestWinds" version of the Stereo Heights and related cloud masks are used in the calculations of all these products.
The altitude-binned classifiers remain at Stage 1 Validated because no systematic study has been done to determine how much low cloud is being missed because of the SDCM's inability to properly detect clouds lower than the height resolution, etc. One has to take the limitations of the stereo-height retrievals into account when looking at these classifiers.
The earlier problem with the An-camera RCCM as referenced to the cloud-top heights (FR_RCCM in the Stereo product) being masked out in sunglint areas has been corrected in version F07_0011 of the stereo product. This field and the "Combined Cloud Fractions" and "RCCM_AnByHeight" data are now completely populated.
The TC_CLASSIFIERS product has undergone extensive revision: most of the field names have been changed, and they have also been re-ordered to put the most important fields at the top of each grid. Please see the Data Products Specifications document available from the MISR Access Data Table for full details.
The Top-of-Atmosphere BRFs (and all accompanying parameters such as the top and side BRFs and the number of unobscured pixels), and all three texture indices are Stage 1 Validated. The Local, Restrictive and Expansive Albedos (except over snow and ice) are also Stage 1 Validated. All Albedo data with a version number of F02_0004 (which went operational November 12, 2002) or greater are at this level. This applies to both current and reprocessed data.
Broadband versions of the restrictive and expansive albedos were added to the product in version F04_0007 (in production as of November 28, 2004). Currently they are calculated as a linear combination of the four spectral albedos using constant co-efficients. Given the simplicity of the algorithm and visual verification of the results, they are debuting at Provisional status.
The second and third texture indices (grey-level difference vectors) were not available until version F03_0005 of the product and they are validated as of the current release (F04_0007) when normalization errors were fixed. The local albedos are internally consistent regardless of the modelling method used and compare well with the BRF images, and the restrictive and expansive albedos also pass visual inspection.
The accuracy of the albedos is limited by two factors - radiometric calibration and bidirectional corrections. All information in the Calibration Quality statement applies equally well to the albedos. At high latitudes, notably poleward of 60 degrees, the angular models currently fail due to extreme anistropy. Albedos in these regions are frequently over-estimated.
There is no detectable bias difference between the expansive and restrictive albedos. Regionally, their rms difference is approximately 0.04 to 0.10 depending on solar zenith angle. Based on limited studies to date, the rms uncertainty due to bidirectional modeling ranges from less than 0.01 at low latitudes to 0.03 at high latitudes (cloudy scenes only).
The reader is urged to pay close attention to the quality of the radiometric calibration as there have been some recent changes made. See the Radiometric Calibration section of the Level 1 Quality Statements.
The clear-sky deterministic modelling algorithm as described in the ATBD is not yet implemented. When the scene is determined to be clear (by looking at the SDCM), all the local albedo components are calculated using solid-angle weighting.
The local albedo is defined as the unobscured reflection from the 2.2 km RLRA and will therefore often appear to be lower than would be expected. If the local albedos are not summarized statistically but instead are looked at on an individual basis, only the ones that have no obscuration should be used. (This information is available at 2.2 km resolution in the NumUnobscuredTop field. Any number >= 64 indicates that there was no obscuration).
Small fluctuations in the value of the RLRA will directly affect the obscuration and the local albedo. For continuous scenes, a 500 m RLRA difference (due to pixel quantization and other factors) typically results in up to a 10% difference in the local albedo due to obscuration effects. This effect is generally not as noticeable in scenes with naturally discontinuous height fields.
The TC_ALBEDO product underwent extensive revision in November 2002, (version F02_0004 of the product file): most of the field names have been changed, and they have also been re-ordered to put the most important fields at the top of each grid. Please see the Data Products Specifications document available from the MISR Access Data Table for full details.
The Local Albedo calculation is first attempted by Deterministic Modelling (if the scene is homogenous), then Stochastic Modelling and finally solid-angle weighting. No modelling is attempted for clear-sky pixels or where the solar zenith angle is < 25.8 degrees. The weights used in the stochastic modelling are based on pre-launch theoretical simulations and will be updated in 2005 to reflect real measurements.
The algorithm for reprojecting the RLRA field down to the surface ellipsoid was found to be flawed and has been completely replaced with a new backwards-projection algorithm that reprojects the BRF's up to the RLRA.
