The Hubble Source Catalog (HSC) combines the tens of thousands of visit-based, general-purpose source lists in the Hubble Legacy Archive (HLA) into a single master catalog.
In the current Beta 0.3 release, the HSC contains members of the WFPC2, ACS/WFC, WFC3/UVIS, and WFC3/IR Source Extractor source lists from HLA version 7.2 that have valid detections (i.e., data with quality flag values less than 5; see HLA Source List FAQ for a definition of the flagging system for HLA source lists). Data from the ACS/HRC (High Resolution Camera) are NOT included in the HSC at the current time. Approximately 1/3 of the ACS/WFC and WFPC2 images in HLA DR7.2 are not included in the HSC due to image quality and other issues.
This is a Beta version of the HSC; you can expect some rough edges and a larger fraction of artifacts than will be present in the Version 1 release. In addition, the database may change without warning since we are working to make improvements. For the upcoming HSC Version 1 release (tentatively planned for fall 2014), we plan to include improved WFPC2 and ACS/WFC source lists using algorithms developed to generate the WFC3 source lists.
All HLA magnitudes are in the ABMAG system. Here is a discussion of the ABMAG, VEGAMAG and STMAG systems. A handy, though not exact conversion for ACS is provided in Sirianni et al. (2005) The Synphot package provides a more generic conversion mechanism for all HST instruments.
NOTE: This Beta 0.2 catalog has severe "doubling" and is missing many objects in regions with high background. It is considered mixed quality.
NOTE: This Beta 0.3 catalog of the Antennae has essentially no "doubling" and is now considerd good quality.
Example of an old Beta 0.2 (WFPC2 and ACS) catalog for the nuclear region in M83.Nearby galaxy with high background: M83 (8234_01), N > 1 version To get the image below, click on "Advanced HSC controls", then check the box for "Require NumImages > 1", then click on HSC (beta). There are 1356 HSC sources.
NOTE: This image has some "doubling" and the sources do not line up with the circles well.
Example of a new Beta 0.3 (WFPC2, ACS and WFC3) catalog for nuclear region in M83.
Example of a new Beta 0.3 (WFPC2, ACS and WFC3) catalog for outer region in M83.
NOTE: The Beta 0.3 version of the nuclear region in M83 is dramatically better, but the outer region shows a high level of non-uniformity at the corner between where the WFC3 ends and the much sparser WFPC2 source list begins. This catalog is considered mixed quality.
NOTE: This catalog is greatly improved compared to HSC Beta 0.2 due to: 1) the inclusion of WFC3 data; 2) better matching (removal of severe "doubling"). However, it is considered a mixed quality catalog due to a number of false detections near the nucleus in the bottom left of the image.
NOTE: There are missing sources in a large part of the image since no SExtractor catalog was made for this image. The HSC objects circled are based on a different image.
NOTE: Using N > 1 cleans up the edge effects pretty well, but the numerous detections along the diffraction spikes, bleeding columns, and saturated centers of the stars remain.
The primary improvements for Beta 0.3 are:) Inclusion of WFC3 sources lists, 2) improved matching (i.e., while roughly 70 percent of the Beta 0.2 catalogs were "good" quality, roughly 95 % of the Beta 0.3 catalogs are "good".
Known Problem #1 - MagAper2 magnitudes for objects on the PC (Planetary Camera) portion of the WFPC2 may differ when compared to the same object when it is on the WF (Wide Filed) portion of the WFPC2. The primary reason for this is that the Point Spread Function (PSF) on the PC is different than on the WF. While the rebinning of the PC to match the pixel size of the WF largely takes care of this effect, differences still remain, typically at the 0.2 to 0.3 magnitude level. We are currently evaluating how large this difference is for point sources and will include documentation on the effect HERE in the near future.
The Hubble Source Catalog (HSC) is based on HLA Source Extractor Bertin & Arnouts 1996 source lists. To build these source lists, the HLA first constructs a "white light" or "detection" image by combining the different filter observations within each visit for each detector. This filter-combined drizzled image provides added depth. Source Extractor is run on the white light detection image to identify each source and determine its position.
Next, the combined drizzled image for each filter used in the detection image is checked for sources at the positions indicated by the detection image. If a valid source is detected at a given position. then its properties are entered into the HLA source list appropriate for the visit, detector, and filter, with a flag value less than 5.
Sources that are found in the white light detection image, but not in a particular filter used to make the white light image, are regarded as "filter-based nondetections". These can be examined by asking for level = 1 under Detection Options on the HSC Detailed Search form.
Note that corrections for Charge Transfer Efficiency (CTE) problems have been made to the WFPC2 HLA source lists, but not the ACS nor WFC3/UVIS source lists. See the HLA CTE FAQ
The detections (and nondetections) that correspond to the same physical object (as determined by the algorithm defined in Budavari & Lubow 2012) are given a unique , are given a unique MatchID number and an associated position (MatchRA, MatchDec). Each member of the match, including nondetections, also has an assigned MemID value and a source position (SourceRA, SourceDec). The procedure dramatically improves the relative astrometry between visits, to a level of about 11 mas or better for the majority of the sources. Each source detection and nondetection has some separation distance, D, from the match position. The DSigma value is the standard deviation of the D values. Visit-based nondetections are listed at the end of the results and have blank MatchID and MemID values.
The plot shows the distribution of the DSigma values, the standard deviations of the source positions in matches for Beta 0.2. The red curve is the distribution for the current HLA astrometry. The blue curve is for the astrometry that has been corrected for the HSC based on the method described in Budavari & Lubow (2012). The median value before astrometric correction is 56 mas (the distribution extends to larger values than shown), while after astrometric correction the median value is 11 mas.
Each source detection and nondetection has some separation distance, D, from the match position. The D-Sigma value is the standard deviation of the D values.
Many of the detections in the Detailed Search form are in matches that involve a single visit and detector. These cases have D=0 and D-Sigma=0.Searches on the Detailed Search Form for only crossmatched detections are made by specifying the constraint D > 0 using the User-specified field and the Field Descriptions options.
The output of the Detailed Search Form has the following options
available via the Detection Options field.
Level 0 - Includes all detections (no nondetections). Such detections have a Det value of Y.
Level 1 - Includes detections and filter-based nondetections. Sources that are found in the white light detection image, but not in one or more of the filters included in the visit, are called "filter-based nondetections". Filter-based nondetections detections have a Det value of N and an assigned MatchID value.
Only certain properties for filter-based nondetections are reported, including the image name, exposure time, and filter. Certain properties such as fluxes and magnitudes are indeterminate and are left blank on the search form.
Level 2 - Includes detections, filter-based nondetections, and visit-based nondetections. Visit-based nondetections are cases where an image overlaps with the specified positional search constraints, but no sources are detected there. Visit-level nondetections have a Det value of N and no assigned MatchID value.
These nondetections have image information, but have blank source positions, fluxes, and magnitudes.
Nondetections have "N" in the "Det" column, and are listed last on the form.
Note: Users studying nondetections may want to use the HLA image cutout Cutout FAQ capability to confirm them.
The HSC Summary Search Form includes results for all detections for a given object on a single row. The magnitudes for different visits are averaged together.
The HSC Detailed Search Form shows each individual detection for a given object on a single row.
The HSC Summary Search Form lists aggregate properties of each match such as the number of visits and filters. In addition, the average magnitude values and their associated RMS scatter (sigma), are listed for the most frequently used filters. More information about a match can be obtained by searching the Detailed Search Form for the appropriate MatchID value (under the User-specified field), or by clicking on the blue MatchID value in the first column when displaying the html version of the requested query. Frequently used filters are listed first; other filters are listed next using a different format
The HSC Detailed Search Form displays an entry for each individual detection in a match on a single row. Each object has a unique MatchID value. Each member of the match, including nondetections, also has an assigned MemID value, a unique position (MatchRA, MatchDec), and a separation distance D from the match position. See FAQ for a more detailed discussion of how matches are defined.
By default, the most frequently used filters across all HSC sources are listed in the HSC Summary Search form:
A_F814W W2_F606W A_F606W W2_F814W A_F775W W2_F300W
where A stands for ACS/WFC and W2 stands for WFPC2.
For Beta 0.3, two additional frequently used filters have been added, namely W3_F110W W3_F160W
where W3 stands for WFC3.
Note that values for the same filter, but different instruments (e.g., A_F814W and W2_814W), are NOT averaged together, since the same aperture size has not been used for different instruments.
Following the globally most frequently used filters, two other filters within the match are listed using the format:
Filter1 Mag1 Mag_Sigma1 Filter2 Mag2 Mag_Sigma2 ...
Users can customize the list of filters by selecting from all available for ACS and WFPC2 (see 4 below).
This hybrid system of including the most frequently used filters in a standard fixed-column format, followed by a format with the appropriate filter listed in the preceding column, is a result of the great diversity of the HST filter-instrument combinations.
Click on "Field Descriptions" in the upper right of the HSC Summary or Detailed Search forms. This provides a short description of the various parameters that can be used in a query. For more detailed information (for example how is the concentration index = CI defined ?), go to the HLA FAQ.
The output definition portions of the HSC Detailed and Summary Search forms can be used to change the output from the default values.
The "Output Columns" feature can be used to modify the order in which the columns appear; remove a column, or add a new column (using the "add" button at the bottom left - the parameter will then show up in the list of columns included under "Output Columns").
The "Sort By" feature allows you to sort using a wide range of parameters. There are three pull-down menus to allow you to sort sequentially - e.g., first on angular separation, then on Match ID, then on CI (concentration index).
The "Output Coordinates" allow you to change from sexagesimal (e.g., 13 37 00.874 -29 51 55.40), to degrees (204.2536415 -29.8653877), to hours (for Right Ascension: 13.6169094 -29.8653877).
The "Output Format" can be changed from the default (HTML_Table - which is displayed in real time) to a large number of alternative formats that can be downloaded.
The "Maximum Records" and "Records per Page" can also be defined.
In the future, it may be possible to save your custom-made catalog definitions for future use.
Yes - A list of targets can be searched using the "File Upload Form" button, which is located near the top right of either the HSC Summary or Detailed Search forms. The "Local File Name" box (or "Browse" pull-down menu) allows you to provide the name of a file listing the targets you would like to include in the search. A number of different format options for the input file are allowed, as defined in the target field definition portion of the form.
The HLA Interactive Display facility can be used to overlay the HSC catalog (click on the pink "Hubble (Beta)" in the upper right part of the display). Here is an example for a part of M101.
Nearby Galaxy: M101 (10918_01), N > 1 version To get the image below, click on "Advanced HSC controls", then check the box for "Require NumImages > 1", then click on HSC (beta), then wait a while (i.e., there are 25,709 HSC sources).
You can get to the HLA from your browser using hla.stsci.edu
Other catalogs that can be overlayed, if data exist for that portion of the sky, are (HLA DAOphot, HLA SExtractor, SDSS (Sloan Digital Sky Survey), 2MASS (Two Micron All Sky Survey), GSC2 (Guide Star Catalog 2), FIRST (Faint Images of the Radio Sky at Twenty Centimeters) or GALEX (the Galaxy Evolution Explorer ultraviolet survey).
Information for individual objects can be obtained by clicking on the object. See Basic Hubble Source Catalog Walkthrough for an example. More detailed information about the options provided within the interactive display (e.g., how to see all the columns in a catalog view) is available at Interactive Display Help
The Hubble Source Catalog is composed of visit/detector-based, general-purpose source lists from the Hubble Legacy Archive (hla.stsci.edu) . While the catalog may be sufficient to accomplish the science goals for certain projects, in other cases astronomers may need to make their own catalogs to achieve the optimal performance that is possible with the data (e.g., to go deeper). In addition, the Hubble observations are inherently different than large-field surveys such as SDSS, due to the pointed, small field-of-view nature of the observations, and the wide range of instruments, filters, and detectors. Here are some of the primary limitations that users should keep in mind when using the HSC .
- Uniformity - Coverage can be very non-uniform (unlike surveys like SDSS), since a wide range of HST instruments, filters, and exposure times have been combined. We recommend that users pan out to see the full HSC field when using the Interactive Display in order to have a better feel for the uniformity of a particular dataset.
(Beta 0.3 Note) - The uniformity is actually worse in many cases for Beta 0.3 due to the inclusions of WFC3 sources lists, which are deeper than for WFPC2 and ACS. The uniformity will be improved for version 1 (tentatively fall 2014) which will include new source lists for WFPC2 and ACS that use algorithmes developed for the WFC3.
- Depth - The HSC does not go as deep as it is possible to go. This is due to a number of different reasons, ranging from using an early version of a WFPC2 catalog (see FAQ ), to the use of visit-based source lists rather than a deep mosaic image where a large number of images have been added together.
- Completeness and nondetections - Information about limiting magnitudes is not currently inlcuded in the HSC.
In addition, the current generation of HLA WFPC2 and ACS Source Extractor source lists have problems finding sources in regions with high background. The WFC3 sources lists are much better in this regard. The next generation of WFPC2 and ACS source lists, which will use the improved WFC3 algorithms, will be incorporated into the HSC when available (tentatively planned for fall 2014)
- Calibration exposures are often missing. In most cases, this is because calibration exposures (generally taken by the instrument groups) are not included in the HLA. However, there are also cases where an often repeated calibration objects (e.g., the white dwarf GRW+70) is not included in the HSC. This is largely for historical reasons (i.e., the number of exposures was so large that it took a very long time to build the HSC in this region). It is likely that the calibration exposures that are in the HLA will be included in future builds of the HSC.
- The use of visit-based rather than mosaic-based source lists. This introduces a number of limitations, including much fainter completeness limits, as discussed above. However, another important limitation imposed by this approach is that different source lists are created for each visit, hence a single, unique source list is not used. A much more efficient method would be to build a single, very deep mosaic with all existing HST observations, and build a source list from this image. Measurements at each of these positions would then be made for all of the seperate observatioins. This is sometimes called "forced photometry", since in many cases an object is not even seen on a separate exposure.
The reason this approach has not been used in the past is due to the difficulty of building good mosaic images. What has been missing is accurate offset information between the different images. This is now available, thanks to the HSC, hence we are in the process of buillding the mosaics necessary to support this approach.
- False detections - Uncorrected cosmic rays are a common cause of blank sources. Users may want to use NumImage > 1 to help filter out this and other artifacts.
Another common cause of "false detections" is the attempt by the detection software to find large, diffuse sources. In some cases this is due to the algorithm being too agressive when looking for these objects and finding noise. In other cases the objects are real, but not obvious unless observed with the right contrast stretch and field-of-view. It is not easy to filter out these potential artifacts without loosing real objects. One technique users might try is to use a size criteria (e.g., concenteration index = CI) to distinquish real vs. false sources for a particular dataset. This would be particularly reasonable in cases where the user is only interested in compact sources rather than extended objects. -
- Doubling - The HLA pipeline is designed to correct the inherant ~ 2 arcsec astrometric uncertainty (due to uncertainties in guide star positions) by comparing with SDSS and GSC2 positions. While this works in the vast majority of cases, it does not work for a few percent of the data for a variety of reasons (e.g., very crowded fields, high backgrounds, few or no SDSS or GSC2 sources in a field).
One of the steps in the HSC matching algorithm is to search for objects within a radius of 0.3" and only include these detections as potential matches. Hence, cases where the HLA has not been able to correct the postions to fit within a 0.3 " radius are left out of the potential pool of detections to be matched. This results in what is called "doubling" (though tripling or higher numbers are also possible if there are several detections with poor absolute astrometry). These cases can generally be identified by looking for a common pattern, i.e., double detections with the same separation and orientation. Here is an example for the Antennae galaxies.
(Beta 0.2 version of the HSC)
(Beta 0.3 Note) Doubling has been dramatically improved for Beta 0.3, primarily because we now make use of 2MASS sources, as well as SDSS and GSC2. An example is shown below. Compare this with the image just above.
- Mismatched sources - The HSC matching algorithm uses a friends-of-friends algorithm, together with a Bayesian method to break up long chains (see Budavari & Lubow 2012) to match detections from different images. In some cases the algorithm has been too aggressive and two very close, but physically separate objects, have been matched together. This is very rare, however.
- Bad images (and hence bad source lists) - Images taken when Hubble has lost lock on guide stars (generally after an earth occultation) are the primary cause of bad images. We attempt to remove these images from the HLA, but occasionally a bad image is missed and a corresponding bad source list is generated. A document showing these and other examples of potential bad images can be found at HLA Images FAQ. If you come across what you believe is a bad image please inform us at firstname.lastname@example.org
- Miscellaneous bad information - There is occassionally incorrect information about an object in the catalog due to a variety of errors ranging from incorrect information in the image header to undetected errors in the pipeline processing. If you suspect a problem, please send an e-mail describing the issue to the email@example.com so we can track it down.
Yes - HLA Images FAQ. Here is a figure from the document showing a variety of artifacts associated with very bright objects.
Due to the great diversity of the Hubble data, this is a hard question to answer. Our approach is to make detailed comparisons for a large number of HSC fields, and present them as "uses cases"(i.e. A comparison of the HSC with Brown et al 2006). By comparing flux differences between sources in the same match and having the same detector and filter, Fig 5(see Budavari & Lubow 2012) indicates typical flux uncertainties of about 2.5% for ACS and 5% for WFPC2, based on the width of the distributions at half the maximum values. In addition, we will produce a PASP-type article describing a wide range of HSC quality comparisons. A draft of the article will be available for Version 1.
The following figure shows the distribution of pair-wise comparisons between repreated photometric measurements in the HSC for Beta 03. The units for the x-axis are flux ratio. It is IMPORTANT to understand that the reason for the apparent increase in the typical flux differences for the WFC3 (which might be expected to be similar to ACS in photometric accuracy for an object with a given brightness) is due to the fact that the WFC3 catalogs go MUCH DEEPER than the WFPC2 and ACS catalogs. Hence, it is the deeper completeness threshold, rather than intrinsic photometric accuracy, that is reflected in the longer tail in flux differences for WFC3. The main point of this figure is to demonstrate that typical photometic uncertainties in the HSC are a few tenths of a magnitude, or better for all instruments.
The following figure shows the distribution of pair-wise comparisons between repreated astrometric matches in the HSC for Beta 03. The units for the x-axis are milli-arcsec (mas). As expected, the instruments with smaller pixels (ACS [50 mas] and WFC3/UVIS  mas) show the best astrometric accuracy, with a peak less than a few mas and typical values less than 10 mas. On the other hand, the instruments with larger pixels WFPC2 [100 mas on the WFC chips which dominate the statistics] and WFC3/IR [130 mas], have much larger astrometric uncertainties,
Send a note to firstname.lastname@example.org. Please include enough information (e.g., a screen save of the problem) to make it possible to diagnose the issue.
Yes there are. Our walkthroughs provide an introduction to HSC usage.
Here are the available walkthroughs:
The HSC step-by-step use cases are designed to provide training material, demonstrate the level of quality you will find in the HSC and illustrate limitations that you should be aware of.
Here are the available use cases: