FIRE IFO 1 LANGLEY RESEARCH CENTER LIDAR USER'S GUIDE FIRE IFO 1 For questions, requests, and comments please contact: Jose M. Alvarez Mark A. Vaughan Mail Stop 417 Mail Stop 417 Langley Research Center or Langley Research Center Hampton, VA 23681-0001 Hampton, VA 23681-0001 (804) 864-2677 Phn (804) 864-5331 Ph (804) 864-7711 Fax (804) 864-7711 Fax Introduction This file is a User's Guide for the information we stored on the Langley DAAC in October 1994. If the information contained herein needs to be updated or corrected, we will submit the revision at a later time. The first part of the Guide explains the structure of the data files. The second part contains an explanation of the log book entries. The third section explains the lidar graphics images we submitted to the archive and the last section deals with mathematically interpreting the near-raw lidar data. NASA LANGLEY RESEARCH CENTER FIRE LIDAR FIRE IF0 1 Data Format The data was originally stored on magnetic tape. Each tape has multiple records. A record consists the information obtained from averaging a number of individual laser pulse returns. Since each data tape was transferred to an optical disk file, the file names are the names of the original tapes. Thus, some of the files contain records obtained on different days, such as "tape264.pro" which contains data obtained on three different days. The record numbers are sequential in any file and therefore, data obtained on different days may have sequential record numbers. The logbook is the best means to connect the record numbers with the date the data was obtained. The organization of data files is as follows. Since we have chosen the blank character as our field delimiter, every individual "chunk" of information is followed by a blank character or a newline and carriage return. For ease in programming, we have denoted the data type we used in writing the individual entries in the parentheses following the word "character". The first line in the file is a file header terminated by a carriage return and a newline character. The file header consists of the following information. The first group of characters(char[4]) provide the tape number followed by a blank character. The second group of characters(char[5]) provide the time zone(time used for this entire data set is GMT) followed by a blank character. The NEXT SIX GROUPS of characters(char[3]) provide the date and time the data collection commenced. Each of these six groups is separated by a blank character and the last group is followed by a blank character. The first group of characters(char[3]) provides the last two digits of the year. The second group of characters(char[3]) provides the day of the month. The third group of characters(char[3]) provides the month. The fourth group of characters(char[3]) provides the hour. The fifth group of characters(char[3]) provides the minute. The sixth group of characters(char[3]) provides the second. From the blank character following the sixth group above to the end of the line is a brief description of the type of data in the file. For example, the data contained in tape258.pro was obtained for the Air Force and FIRE as lidar data of a clear day. THIS FILE HEADER IS TERMINATED BY A NEWLINE AND CARRIAGE RETURN. The second line in the file header is a record header and every record following is preceeded by this type of header. The record header consists of the following information. The first group of characters(int[3]) provides the number of data points in each of the arrays following each record header; this group is followed by a blank character. The second group of characters(float[8.2]) provides the start time of the record in seconds since midnight(GMT) followed by a blank character. The second group of characters(float[8.2]) provides the stop time of the record in seconds since midnight(GMT) followed by a blank character. The third group of characters(int[3]) provides the number of laser pulse returns which were averaged to provide the data arrays. A blank character follows. The fourth group of characters(int[4]) provides the record number of that individual record in the data file followed by a blank character. The fifth and sixth groups of characters(char[9]) provides the start and stop times for that record. There is a blank character between these groups and a blank character following the sixth group. The last group of characters(int[1]) is a flag depicting the type of data array following. If this entry is a zero(0), then the array following contains the altitude array and two data arrays. If this entry is a one(1), then the array following contains only the two data arrays. THIS RECORD HEADER IS TERMINATED BY A NEWLINE AND CARRIAGE RETURN. The next series of numbers are the data arrays. They are all floating point numbers with two decimals following the decimal point. Reading horizontally, if the last character in the record header is a zero(0), these numbers represent the altitude, the perpendicular channel average, and the parallel channel average. If the last character in the record header is a one(1), these numbers, again reading horizontally, represent the perpendicular channel average and the parallel channel average. Each group of entries is separated by a blank characters and each line is terminated by a newline and a carriage return. Reading vertically, there are as many entries as is indicated in the first entry in the record header(this is usually 500); this provides a programmer with an easy way stop reading the data array and find the next record header. The suffix "pro" denotes the organization of the entire data file as being consistent with the format of most spreadsheets, i.e., time-sequential data runs vertically. Logbook Information This writeup is meant to accompany the LaRC IFO 1 logbook entries stored in the files named with the starting characters "IFO1_". The logbook file names reflect the manner in which the lidar data were originally stored, which is to say that the three numbers following the underscore character(_) denote the original magnetic tape number. The following tables present the dates and the times during which the data were collected. The data were all collected at Fort McCoy, Wisconsin during the first FIRE Intensive Field Operation(IFO). The Langley Lidar was at this field site from October 16 to November 2, 1986. All times quoted in this file, the data files, the logbook files and the catalog picture(GIF) files are GMT. File Name Date(s) Times IFO1_258.LOG October 16, 1986 1345-1941 IFO1_259.LOG October 17, 1986 1628-2132 October 18, 1986 1408-1946 IFO1_260.LOG October 19, 1986 1214-1731 IFO1_261.LOG October 20, 1986 1053-1553 October 21, 1986 1522-1737 IFO1_262.LOG October 22, 1986 1051-1602 IFO1_263.LOG October 22, 1986 1615-2125 IFO1_264.LOG October 23, 1986 1634-1926 October 24, 1986 1754-2035 October 27, 1986 1240-1610 IFO1_265.LOG October 27, 1986 1906-2400 October 28, 1986 0000-0200 IFO1_266.LOG October 28, 1986 0751-1441 IFO1_267.LOG October 28, 1986 1457-2331 IFO1_268.LOG October 30, 1986 1148-2136 IFO1_269.LOG October 31, 1986 1330-1700 IFO1_270.LOG November 1, 1986 1554-2333 IFO1_271.LOG November 2, 1986 1356-2006 IFO1_272.LOG November 2, 1986 2016-2195 The log files contain instrument settings, comments, and other information which is not contained in the data files. A brief description of how the data was obtained and how it was stored is necessary to understand some of the log entries. A record is made up of data collected and stored by the data acquisation system. Each record is obtained by averaging a number of individual pulse returns. A return is defined as the data obtained from the light scattered from one laser pulse. To cover the wide dynamic range of each pulse return, the data in each channel(s and p) were obtained by monitoring the output of one or two photomultipliers and combining the them. The lidar system was configured to sample light scattered from the atmosphere and from areosols in two perperdicular planes. The "p" plane denotes a plane parallel to the polarization plane of the highly linearly polarized beam emanating from the laser. The "s" plane is the plane perpendicular to the p plane and to the travel direction of the output beam. The following presents a sample portion of the log entry in file IFO1_258.ME to facilitate further explanation. Record Nos. Time COMMENTS: Record 18 aborted 11- 18 1439-1444 Wds. 1000 Avg. 64 Data/BG 4 Per. 200 - - S D1:25 D2:50/60 H1:953 H2:1008 G1:100 G2:100 P D1:25 D2:50/60 H1:1081 H2:1098 G1:100 G2:100 For these records, the lidar "p" channel utilized two photomultipliers(pmt's) operating with different cathode voltages as shown by the H1 and H2 entries in the fourth row which contains the P channel instrument settings. Likewise, H1 and H2 in the S row are the cathode voltages for that channel. The D1 settings in both channels are the number of microseconds which elapsed before the first pmt was activated. The first D2 setting is the time in microseconds which elapsed before the second pmt was electronically substituted for the first pmt in the data taking. On alternate returns the second pmt was switched on somewhat later and this delay time is the second of the D2 entries. This was done to provide a common region in each return to splice the data from the two pmts. The gain setting of the digiters in each channel is given by G1 and G2. For most of this mission the number of data words put into a record was 1000, each word being 10 bits. The Data/BG entry gives the ratio of data to background returns collected. The background returns are collected by not turning on the laser Q switch and thus contain not only background light data but also the small "signal" generated when the flash lamps fire and the system operates. Background returns are subtracted from data records thus correcting the data return for system effects and the background light. The "AVG" entry contains the number of returns collected at each of the two D2 settings. Since it takes both of the returns collected at each D2 setting to determine how the data from the two pmt's in each return is to be spliced, each pair of spliced returns(the D2 pair) is not totally independent; each member of the pair has had the data collected by each pmt spliced by determining constants obtained from both returns. Nevertheless, each record is obtained by averaging a number of individual returns where the number of returns averaged is twice the AVG setting. The bottom line is that all of the returns in that D1/D2 sequence are averaged to make up a record. The "Per" is short for laser period in milliseconds. It is the inverse of the laser pulse repetition rate. FIRE IFO 1 Langley LIDAR Graphics Images As part of the data archival material we submitted color images of the lidar data. These images were produced in a currently popular format, the Graphics Interchange Format(GIF), in order to facilitate viewing them on a pc or work station. The naming convention is that of the log entries, which is to say, the images are named according to the tape number. However, if there was too much data to fit within one image or if the tape contained more that one day, then the images are labeled a, b, c, etc. The term "Scattering" in the image files refers to "Attenuated Scattering Ratio" which is described in the Data Reference section following this section. The "Depolarization" and how this quantity is obtained is also described in the Data Reference section. Data Reference Depolarization The single scattering lidar equation assumed here is given by C * PWR P(Z) = -------- * B(Z) * T2(Z) (1) 2 Z where PWR is the transmitted laser power, C is a system constant which includes terms describing the effective receiver area and the laser pulse width, B(z) is the volume backscattering coefficient, and T2(z), the two way transmittance, is the exponential of -2 times the integral with respect to range of the extinction coefficient, X(r), from z=0, at the lidar transmitter, to some range z. Alternately, for a zenith pointing lidar, T2(z) = EXP{ -2 * INTEGRATE[ X(R), LoLim=0, UpLim=z, dR ] } (2) The signal received and stored by the data acquisition sub-system is proportional to the optical power, P, at the receiver; that is S = G * P (3) where G is the electro-optic gain, which includes transmission optics losses, detector electronics gain, electro-optic conversion factors such as photo- multiplier quantum efficiencies, and the electronic gain of any preamplifiers. Note that the received parallel and perpendicular signals share certain factors in common -- in particular the laser output, the receiver geometry prior to the polarizing cube, and the optical depth of the atmosphere. However, upon entering the polarizing cube the signals are routed through separate optical and electric pathways that will almost certainly have very different gains, if for no other reason than that the perpendicular signal in clear air requires much more amplification than the parallel signal. The linear nature of the process of converting light into a series of digitized signals may be written in more detail as S[p,s] = G[p,s] * C' * PWR * T2 * B[p,s] (4) where S[p,s] represents the signal stored in the parallel (p) and perpendicular (s) channels of the data acquisition sub-system. C' accounts for geometric and optical effects encountered BEFORE the signal is transmitted to the polarizing optics (i.e., C' = C / Z^2), T2 represents extinction effects, and PWR is once again the transmitted power. B[p,s] is the volume backscattering coefficient in the parallel and perpendicular channel, and the G[p,s] term quantifies the electro-optic signal gain for each channel incurred AFTER the beam passes through the polarizer. Assuming perfect alignment of the optical components, we can write the ratio of the signals received in the two channels as S[s] C' PWR T2 G[s] B[s] G[s] B[s] m = ---- = ------------------- = ---- * ---- = GR * D (5) S[p] C' PWR T2 G[p] B[p] G[p] B[p] where GR is the gain ratio, and D the depolarization ratio. In this case, all we need to retrieve the depolarization ratios from the measured ratios is an accurate estimate of the gain ratio. Suppose, however, that the optical axis of the polarizing cube is somewhat misaligned with that of the transmitted beam, leading to a apparent depolari- zation ratio which is artificially (and incorrectly) low. To eliminate these sorts of measurement errors we inserted a half-wave plate into the optical path of the lidar. If the polarization plane of the transmitted beam exactly matches that of the principle axis of the half-wave plate, each component of the light is transmitted unchanged to the polarizing optics. However, if the half-wave plate is misaligned, or offset, by some angle, Q, the polarization plane of the transmitted light is rotated in the same direction by a factor of 2*Q. Consequently, after transmission, the electric field vectors of the parallel and perpendicular components are given by E[p,t] = E[p,i] cos(2Q) + E[s,i] sin(2Q) (6a) E[s,t] = E[p,i] sin(2Q) - E[s,i] cos(2Q) (6b) where E[?,t] represents the field transmitted through the half-wave plate, and E[?,i] represents the incident field. Since the electric fields produced by the scattering process are assumed to uncorrelated, the powers associated with the transmitted fields are 2 2 2 2 P[p] = 0.5*( E[p,i] cos (2Q) + E[s,i] sin (2Q) ) (7a) 2 2 2 2 P[s] = 0.5*( E[p,i] sin (2Q) + E[s,i] cos (2Q) ) (7b) Recognizing that 2 E [?,i] = K * C' * PWR * T2 * B[?] (8) where K is a constant of proportionality, we can substitute (7a) and (7b) into (4) and write our expressions for the signals received in the parallel and perpendicular channels as 2 2 S[p] = 0.5 K C' G[p] PWR T2 { B[p]*cos (2Q) + B[s]*sin (2Q) } (9a) 2 2 S[s] = 0.5 K C' G[s] PWR T2 { B[p]*sin (2Q) + B[s]*cos (2Q) } (9b) If we now recompute the measured ratio given in (7) we have 2 2 2 G[s] { B[p]*sin (2Q) + B[s]*cos (2Q) } { D + tan (2Q) } m = -------------------------------------- = GR * ------------------ (10) 2 2 2 G[p] { B[p]*cos (2Q) + B[p]*sin (2Q) } { 1 + D*tan (2Q) } Our calibration scheme uses equation 11 to simultaneously determine both the gain ratio, GR, and the offset angle, Q, due to any misalignment of the optical components. We do this by rotating the half- wave plate through a series of three or more accurately measured calibration angles, acquiring some number of data records at each angle. For the "j"th calibration angle, A[j], our depolarization equation becomes 2 D + tan (2Q - 2A[j]) m[j] = GR * ---------------------- (11) 2 1 + D*tan (2Q - 2A[j]) ------------------------------------------------------------------------------ <<<<<<<<<<<<<<<<<< IMPORTANT NOTE >>>>>>>>>>>>>>>>>>>>>>>> In order to compute all of the above from the data in the logbook, the conversion constant between the angular "counts" given in the logbook and the actual physical angle must be known. It is given below. 1099 angular counts = 10.00 degrees(positive rotation) 1096 angular counts = 10.00 degrees(negative rotation) <<<<<<<<<<<<<<<< >>>>>>>>>>>>>>>>>>>>>>>>>> ------------------------------------------------------------------------------ After averaging the data for each angle, we apply a nonlinear least squares algorithm to solve the resulting system of equations in a "clear air" region of the data. In addition to calculating the system calibration constants, this procedure also determines the mean depolarization ratio of "clear air" for the altitude range where the calibration was performed. Attenuated Scattering Ratio and Other Quantities Our lidar is designed to make two simultaneous measurements: a total backscatter (i.e., total signal) profile, and a depolarization ratio profile. Both of these are derived quantities, obtained from the raw parallel (p) and perpendicular (s) channel backscatter data by using the calibration constants described here. The total signal profile (TS), which is the sum of the parallel and perpendicular contributions, is calculated as S[s,Z] TS[Z] = S[p,Z] + ---------- (12) Gain Ratio In calculating the total attenuated backscatter we in essence create a signal return proportional to what we would have obtained using a lidar that was NOT equipped to make depolarization measurements. This signal should be the quantity used to calculate optical thickness and optical extinction. We also routinely compute another quantity, the attenuated scattering ratio, obtained by dividing our total signal profile by a model "clear air" profile -- that is, the profile we would expect if we were operating our lidar in a purely molecular atmosphere. In practice this is done by calculating a constant of proportionality, D, between the lidar data and the model over some altitude range of the measured profile judged to be "clear air". (If at all possible, this clear air region is chosen to occur below any clouds or aerosol layers.) Then for each sampled altitude we compute the attenuated scattering ratio, R[i], using TotalSignal[i] R[i] = -------------- (13) D * model[i] In terms of the lidar equation (equation 1 above), the attenuated scattering ratio, R, is given by CldBeta(r) R(r) = [ 1 + ---------- ] EXP{ -2 INTEGRAL[CldExtin(r), r=CldBase,CldTop]} AirBeta(r) (14) where Beta is the volume backscatter coeficient, and Extin is the extinction coeficient. For any given data session a clear air model can be constructed using the lidar equation, atmospheric densities derived from local rawinsonde data and theoretical values for the Rayleigh backscattering and extinction coefficients. This is the quantity displayed in the data images in the archive.