.:scripts:home

Hierarchical Script Structure

The graphic below shows the structure of the process_all scripts in the ADPAA library. Each script calls the ones beneath it (for example: the process_all_dir script will call the process_all_polcast3 script, which in turn calls the process_all script, and etc.).

Master Processing Scripts

process_all_dir

This simply goes through every PostProcessing directory and processes the *.sea file. Edits are also applied. To view a list of command line arguments that will work with process_all_dir, just enter process_all_dir into the command line

When running the process_all_dir script it is a requirement to be in the general time period directory. If the script is ran in any other directory nothing will happen.

The following is the list of proper names and directories:

   MC3E                   - Spring 2011 Oklahoma Project (UND Directory Structure)
   Goodrich               - Winter 2010 North Dakota Project (UND Directory Structure)
   POLCAST3               - Summer 2010 North Dakota Project (RAL Directory Structure)
   SaudiArabia_Spring2009 - Spring 2009 Saudi Arabia Project (RAL Directory Structure)
   Mali_Summer2008        - Summer 2008 Mali Project (RAL Directory Structure)
   SaudiArabia_Summer08   - Winter 2007/2008 Saudi Arabia Project (UND Directory Structure)
   POLCAST2               - Summer 2008 North Dakota Project (RAL Directory Structure)
   Mali_Summer2006        - Summer 2006 Mali Project (RAL Directory Structure)
   Mali_Summer2007        - Summer 2007 Mali Project (RAL Directory Structure)
   SaudiArabia_Winter0708 - Winter 2007/2008 Saudi Arabia Project (RAL Directory Structure)
   Saudi                  - Winter 2007/2008 Saudi Arabia Project (UND Directory Structure)
   Mali                   - July/August/September/October 2007 (UND Directory Structure)
   Harvesting             - June/July/August 2007
   Sikorsky3              - September/October 2005
   TAMDAR_Turbulence      - August/September 2005
   Sikorsky2              - January/February 2005
   L3Com                  - November/December 2004
   MPACE                  - September/October 2004
   IOP1                   - June/July 2004
   Sikorsky               - March/April 2004
   WISP4                  - February/March 2004
   THORpex                - November/December 2004
   NACP                   - May/June 2003
   Crystal                - July 2002
   Kwajex                 - August/September 1999

EX: process_all_dir POLCAST3

     This simply goes through every PostProcessing directory and processes the *.sea file. Edits are also applied.
     To view a list of command line arguments that will work with process_all_dir, just enter process_all_dir into the command line

Applications In order to change the program so that the cloud bases and temperatures could be determined during the 2010 seeding by the Cessna340_N37360 aircraft, it was necessary to find the time intervals that the aircraft was near a target. THis is due to the lack of equipment on the aircraft that would allow it to keep track of cloud bases during flight. The process used to determine this post flight is outlined below:

1. Open the data file for the date in question under /nas/ral/NorthDakota/Summer2010/Aircraft/Cessna340_N37360/FlightData, enter the Post_Processing file, and use cplot to the open the .pol3a or .pol file
2. Plot Latitude versus Longitude in cplot
3. Locate the areas that the aircraft seems to circle on the chart
4. Use a combination of Tools->Select Time Interval and Control->Time Interval in cplot to narrow down the flight just the circling times of the aircraft. As a rule of thumb, the aircraft must circle a minimum of 15 minutes in order for that region to be considered a target.
5. Now that you have found the time interval for the target, switch your x and y axis on cplot to Pressure_Alt and Air_Temp respectively. These will produce a rather random plot, but will allow you to find the average cloud height and temperature.
6. Use Tools-> Statistics to determine the approximate cloud height and temperature.

Data Level Processing Scripts

M300 Data File (*.sea) Scripts

extract_tables

This script will extract the tables in a given *.sea file. The syntax for this is as follows:

extract_tables <-vm> input_file

getstart_info

This subroutine determines the start time and date from a *.sea file. The syntax for this is:

getstart_info <-sfm> <-end> <-v> <-vm> input_file

process_raw

Process_raw creates instrument files from the M300 *.sea binary file. The syntax for this subroutine is:

process_raw <-d> <-v> <-vm> input_file

UND 'NASA' ASCII File Scripts

addedit

The addedit is a perl program that creates an edit file.

addedit [start=n] [end=n] [id='Editor Name'] [outputfile] name='Parameter' why='Reason for Edit' inputfile

Bracket entries are optional. Single quotation entries only are mandatory. If no start and end times are given the whole entire time period of the inputfile is used. If no id parameter is given the Flight Scientist name is used.The inpute file has to be a *.raw file. IF no outputfile is given the input file is used as the base of the output file.

Ex: addedit start=64100 end=64200 name='Pitot_Nose' why='Iced up' 08_06_13_18_19_05.physical.raw

apply_edits

The apply_edits file is used to create a clean file which can be used with cplot.

apply_edits <csh_error> input_filename edit_filename output_filename

input_filename - File in the starndard NASA/UND ASCII format to which edits are applied. edit_filename - File which contain the edits to apply to the input file. output_filename - File created by appling the edits to the input file.

Including the csh_error option causes the error to be return to the csh calling program, instead of occuring in the IDL code.

combine_files

This script combines parted (different files on the same flight) files that have the same extension. To use this script call it in any ????????_? (YEARMMDD_#) named folder without specifying any arguments. The syntax for this script is:

combine_files <file_suffix>
file_suffix - Suffix after the first dot of the file to process.
              If file_suffix argument not given, default list of summary files is used.

drift_correction

The drift_correction is a python program that corrects for cloud drift due to wind speeds to show the hypothetical path a plane would follow if clouds did not drift due to wind speed. This can be particully useful when trying to determine when a cloud is passing through a cloud more than once. The syntax for running the code is:

drift_correction [start=n] [end=n] [coordinates=AIMMS] [speed=n] [direction=n] inputfile

Start and end represent the start and end times for which you want drift coordinates displayed for. Coordinates are whatever coordinates you want corrected (AIMMS or GPS). Speed and direction are optional parameters input by the user when the user does not want to rely on AIMMS to provide the wind direction and speed for the interval being corrected. If speed and direction are not provided, the program will choose the first valid AIMMS wind speed and direction in the interval selected from the inputfile The input file should be a *.sau file. The output file will be a *.drift file. this program may only be run on a computer with python 2.5 with numpy installed (as of now esxfedora and buster are the only computers with the program).

list_paras

This subroutine program can be used to determine what parameters that are required by UND "NASA" formatted files. The syntax is:

listparas <-h> <-l> inputfile
-l, --longname                         
Include full variable description

mergefield

The purpose of the mergefield program is to merge parameters of one data file with another. The syntax of the program is as such:

mergefield field=n [target=n ] file=mergefile [MVC[i]=x]+ [tolerance= ]         [fill=interpolate|older] < infile > outfile

Field is the parameter(s) being taken from the input file to be added to the mergefile. This is expressed in a number. Target is the number of the parameter being put into the output file. For example, say you wanted the temperature parameter from the March 20 2009 saudi file to be added to the pcasp raw data file. The temperature is the first parameter (column of data) in the .sau file, therefore, your field would be 1. If you want all parameters from the .sau file to be added, do not specify a field number. There are 17 parameters in the march 20 2009 raw pcasp file so the target would be the number 18 (since you want to add it as the 18th parameter. Please note that if you do not specify the target, it will automatically be set to the field number(s). File must be set to whatever file the field is being obtained from, in this case it is 09_03_20_06_46_51.sau. The infile is whatever the file the field is being added to, in this case it is 09_03_20_06_46_51.conc.pcasp.raw. The outfile is the name of the new file which is being outputted by this program, in other words, what the file that contains the combined parameters will be named. For example, I could name this new file 09_03_20_06_46_51.temp.pcasp.merge.

Overall, the command would look like this:

mergefield field=1 target=18 file=09_03_20_06_46_51.sau <09_03_20_06_46_51.conc.pcasp.raw> 09_03_20_06_46_51.temp.pcasp.merge

This would output a file named 09_03_20_06_46_51.temp.pcasp.merge to whatever directory the command is being executed from. Tolerance and fill are optional parameters and are rarely used.

pcaspscat

The pcaspscat program creates a *.550nm.scat.raw file (from whatever *.conc.pcasp.raw file in the directory the program is being executed from). This file displays atmospheric scattering, absorbtion, backscattering, and assymetry parameters all based on the pcasp channels. These parameters are all based on a wavelength of 550nm. The program is executed under the following syntax:

pcaspscat [-v] [-vm] input_file

1. v - Verbose mode.
2. vm - Execute code using IDL virtual machine.

The input file must be a *.conc.pcasp.raw file.

process_raw_file

This subset program edits the input file and creates a .1Hz file. This subset utilizes the avgfileds function which averages the fields from the source file into a target file. In this case the target file is the newly created .1Hz file. The syntax for this program is as follows:

process_raw_file inputfile

subset

The subset program is used to create a data file which takes data from another file that follows a certain numerical characteristic. For example, say I wanter all data from the March 20 2009 saudi file that had LWC greater than 1 g/cm^3 or all data with temperatures between -10 and -5 degrees celsius. I can use the subset program to create a file with that data. The syntax for the program is as follows:

subset [start=n] [end=n] [test=n] [field=n] [fill=MVC's] [mvcfield=n] [newmvc=n] [name='Parameter Names'] [unit='Units Values'] [format='parameter format'] [format_s='string format']  < inputfile > outputfile

Analysis Scripts

cplot

To start the main visulation program.

cplot data_file

The 'data_file' is usually in the NASA UND format.

plot2dc

To bring up 2dc probe images, the plot2dc program may be used. To execute simply use this syntax:

plot2dc 2DC_file

Images from the *.2dc file will be brought up for analysis.