7.3 Removing Blemishes

  The IUEDAC has several procedures for removing blemishes from extracted data (see chapter 8 for removing blemishes from LBL files). The first to be discussed is PATCH. This procedure interpolates over data with undesirable quality flag values using the nearest good points.

PATCH,F,Q,fcor,cutoff,header=h,/newsips 

F	Input fluxes to be patched.
Q	Quality flags.
fcor	Corrected fluxes (if not present, F will be overwritten).
cutoff	If not given, all fluxes for which the corresponding quality flag is negative will be interpolated. Unless the NEWSIPS keyword is set, CUTOFF is assumed to be the lowest acceptable value for the quality flags.
header	You may have a comment about the patching done added to a FITS header via this keyword.
newsips	If this keyword is set, the CUTOFF parameter will be interpreted as the sum of bit settings to avoid. You may set a value for CUTOFF with the NUFLAGS procedure, described below.

Here is an example of using the CUTOFF parameter for NEWSIPS data. To dispose of saturated pixels and reseaux, which have $\nu$ values of -1024 and -4096 respectively, ``cutoff'' could be -5120. You can use the procedure NUFLAGS to set a value for ``cutoff''. It will let you select the criteria (e.g., reseaux) to be avoided, and return the appropriate number. Here's an example of using NUFLAGS and PATCH together for NEWSIPS data:

readmx,'swp12345.mxlo',h,w,f,nu,s

nuflags,cutoff 

patch,f,nu,ff,cutoff,/newsips,header=h

You must include the /newsips keyword if ``cutoff'' is to be interpreted properly. If you pass the FITS header using the ``header'' keyword, PATCH will add a line to the HISTORY portion to reflect what was done to the data.

Another procedure which linearly interpolates across spectral regions is BLEMISH. The user marks the edges of the bad region with crosshairs.

BLEMISH,W,F,bsize,FCOR  

W	Wavelengths.
F	Fluxes to be edited.
bsize	Optional scalar specifying the interval (in the same units as W) over which F will be averaged on either side of the blemish, before interpolation.
FCOR	Edited fluxes.

You may restrict the wavelength range covered by a spectral segment with TRIM. This procedure reduces the number of points in the vectors you pass to it by eliminating all the points outside the range of wavelengths you specify. Here is its calling sequence and a few examples:

TRIM,LOW,HIGH,V1,v2,v3,v4,v5,v6,header=header

TRIM,1200,1950,W,F,Q 

TRIM,1300,1600,w,f,q,s,bkgd,net,header=h

TRIM eliminates all the points at wavelengths shorter than LOW and longer than HIGH. Generally speaking LOW and HIGH are given in the same units as your V1 vector (usually wavelengths). Up to six vectors can be specified. Such trims can be used to eliminate regions of poor data quality. For NEWSIPS data if you include the FITS header using the HEADER keyword a notation will be made in the HISTORY section about trimming done to the data.

TRIM can be used to eliminate regions of poor data quality. First determine what the lowest acceptable quality flag is (here, we set it equal to a variable ``cutoff''). The highest value will likely be the maximum value of Q, since the maximum value is likely to be the value denoting good data:

trim,cutoff,max(q),q,w,f

This discards all points with quality flags lower than cutoff. There may be gaps in your spectrum after doing this!

Procedures for patching bad data are available for LBL files. See Section 8.4 for more information.

Note that the data quality flags are not completely reliable; you may need to examine the LBL files or the raw files in order to determine whether a blemish is due to the instrument, a cosmic ray, or is a real feature in your spectrum. See the IUE Data Analysis Guide (IUE NASA Newsletter 32) for additional information.