The cross-dispersion profile is determined from the background-subtracted net flux data. Each line of data in the low-dispersion SI that is contained within the spatial limits of the extraction slit (13 lines for point sources, 23 lines for extended, trailed, multiple, and flat-field sources) is temporarily binned in wavelength according to the S/N, with bin sizes typically ranging from 1 to 10 pixels. A warning is issued if the bin size exceeds 4 pixels since this may indicate either weak or noisy spectral data or that a large fraction of the data is unusable due to conditions such as saturation or telemetry dropouts. The area under the cross-dispersion profile at each wavelength bin is then normalized to one to divide out the spectral information. After normalizing, the fraction of flux in each line of data (relative to the total flux in each wavelength bin) is smoothed by fitting splines with anywhere from 2 to 15 nodes in each line.
The number of spline nodes is determined dynamically based on the total
of the square of the S/N of the spectral data. The placement of the
nodes in wavelength space is also determined dynamically so that the
data located between each pair of nodes have an equal total of
(S/N)2. Thus regions of higher S/N have more spline nodes placed more
closely together than regions with lower S/N. All lines of data in the
cross-dispersion direction use the same number and placement of spline
nodes and there is always a minimum of two nodes placed at either end of
the spectrum. However, when only two spline nodes are used, the spectral
data is fitted with a default extraction profile. The spline fits for
each line of data are computed iteratively, rejecting data bins that
have residuals to the fit greater than 3.5
. The iteration cycle
ends when no new bins are rejected.
In the procedure outlined above, those wavelength bins that have negative or zero total flux are rejected from the computation of the spline fits. If bins on either end of the spectrum are rejected, they are replaced by the average fraction of light values for the last 10 good bins closest to the end. This is done to avoid extrapolation of the high order spline fits into wavelength regions that have no constraining data. Rejected bins that are not at either end of the spectrum but are located in the interior of spectrum are simply excluded from the computation of the spline fit and are therefore essentially interpolated over. This extrapolation/interpolation process provides a means of dealing with discontinuous data.
The cross-dispersion profile at each wavelength sample is constructed by evaluating the spline fits, clipping the negative values that may occur in the wings of the profile, and renormalizing to ensure a sum of unity at each wavelength. Thirteen spline fits (one for each of the 13 lines of the low-dispersion SI containing source flux) are used to specify the cross-dispersion profile for point source data, and likewise 23 are used for extended, trailed, or multiple sources. This profile is used when computing the extracted fluxes to weight the data so that the pixels with the largest fraction of light are given the most weight. Because the profile is derived by spline fitting each spatial line of data independently of other lines, a wide variety of profiles can be accommodated. Specifically, the algorithm is very capable of handling spectra obtained through either the large or small spectrograph apertures, trailed sources, and extended objects with complicated source structures which may cause the profile to vary dramatically as a function of wavelength. However, because the algorithm produces a single output spectrum, spectra of several sources in the aperture (multiple spectra) will not be separated. Such images will require custom extraction.