Planet Pipeline - Search Help

So, how do you use the Planet Pipeline search form? This document contains information on utilizing an optimized form for solar system objects.

Name or Description: Describe the type of target you would like to search for
Separate multiple targets you want with commas
The form auto-suggests items based on what you are typing
Start and End Date: Use the drop-down calendar to specify a date range
Searches for any exposures taken between that range (WFPC2)
The lifetime of WFPC2 was January 1994 till May 2009
Observation Date (MJD): Search by a modified Julian date
Can also search by a range (separate with two dots: 47500 .. 49500)
Proposal ID: Find an associated proposal ID of exposures
Principal Investigator (P.I.): Search by the last name of a proposal's main researcher
Dataset (or rootname): Identifies unique exposures/images
Exposure Time: Length of the exposure (in seconds), how long it took light into the camera
Filters: Searches both WFPC2 filters for the specified filter name
WFPC2 Detector: Optional field, limit results to exposures from a specific detector/s for WFPC2
Advanced Search Options - Coordinate Search (RA/DEC): Searches around an RA/DEC are possible
Either RA, DEC, or both must be specified
Default radius is 3 arcminutes, can be adjusted in the Radius field

User Specified Field n

You may use these form elements to search on any column(s) in the mission table. First, select the field you wish to search from the pulldown menu under the "User-specified field n" heading. Then, type in the qualification in the corresponding "Field Descriptions" box. Clicking on the "Field Descriptions" link, will display information on the allowed fields including the allowed range of values.

As an example, a Kepler user might select "E(B-V)" from the pulldown menu and enter "< 0.5" in the "Field Descriptions" box.

NOTE only fields which are not already included on the search form should be selected. Specifying search criteria for a field that is listed in BOTH the form and in the User Specified field, may cause either the query to fail or return unexpected results.

Output Columns

This form element allows you to choose the columns to be displayed and their order, for the search results. A set of columns that are commonly requested has been chosen as a default.

You remove output columns by highlighting the column to be removed and then clicking on the remove button to the right of the output columns list.

You may determine the order of columns by highlighting a column and then clicking the up or down buttons to the right of the list of chosen output columns.

You may also add more columns to the list, either singly or all at once. To add individual columns, select the desired column from the pull down menu beneath the list of chosen output columns, then click the add button. The column will be added to the bottom of the output column list. To add all the available columns, click the "add all" button.

Note the output column form element has its own reset button to restore the list of output columns to the values initially displayed when the page was drawn. The reset button at the top of the form is used to reset the other form elements. Clicking the "clear form" button will restore the original defaults to all isections of the form.

One warning, the amount of memory required for a search is proportional to the number of columns requested. For large requests, users may want to reduce the number of output columns to the minimum required.

Verb

Verb is an integer parameter used by the VO community for specifying the amount of output returned for a given search request. It is only available when retrieving data as a web service but works with every MAST service. Setting verb=3 in a search request is equivalent to specifying "add all" from a search form; it will return all the available columns in the output not just the standard default fields. Currently setting verb to any other value has no effect.

Sort output by:

Choose how you want the output rows sorted. You can select up to three fields to sort on. The rows will be sorted in the order of the first sort field; if two rows have the same sort field, they will be sorted in order of the second sort field, and so on. Default sort fields may be listed, but any field from the pulldown list can be used. Specifying multiple sort fields may increase executon time. If you prefer no sorting, you can specify "null" for all 3 fields. This may speed up the query, but results will be displayed in the order in which they were originally stored in the database table.

For each field, you can select that the rows be sorted in reverse order on that field by selecting the reverse checkbox. For example, you can sort the rows with the most recent observations first by selecting Observation Date for the first sort field and selecting the reverse checkbox next to it.

One word of caution: the selected sort field can change the search results when the query finds more rows than are displayed ( i.e, when the number of found rows exceeds the value of "maximum records"). For example, for a search on a particular coordinate that finds 5,000 entries, if the search is sorted on exposure time and 1,001 rows are to be displayed (the default), then the 1,001 shortest exposures from the 5,000 found entries will be displayed which may not include the entries closest to the desired position. (It is a good idea to always sort on "ang_sep" for target or coordinate searches).

Display Coordinates

Specifies the format for displaying the primary equatorial (i.e., RA and Dec) coordinates. The options include:

Sexagesimal - The default format with Right Ascension specified as hh mm ss.sss and Declination as +/-dd mm ss.ss ( e.g., RA = 12 46 11.091, Dec = -00 30 12.08). Note an extra digit was added as of June, 2012.
Degrees - Decimal degrees for both RA and Dec with 7 significant figures to the right of the decimal point (e.g., RA=191.5461912, Dec=-0.5033333). Note 2 extra digits were added as of June, 2012. or
Hours - Decimal hours for RA and decimal degrees for Dec in same format as for decimal degrees (e.g., RA=12.7697512, Dec=-0.5033333). Note decimal hours = decimal degrees/15.0. (Note 2 extra digits were added as of June, 2012.)

Any other coordinate fields contained in the searched mission catalog will be displayed in their original format.

Maximum Records

This value specifies the maximum number of rows returned in a single query. For the standard mission search forms, the current default is 1,001, but values from 1 to 25,001 are allowed. For the file upload forms in which multiple targets, data ID's, or coordinates can be specified, the default is set to 20 rows per file entry, with allowed values ranging from 1 to 5,000.

Note when displayed as HTML, the latest search scripts will display 100 records per page. Links to the additional pages are shown on the results page. This paging feature however does require javascript to be turned on.

Users should be cautioned about retrieving a large number of records (i.e., > 10,000 - 15,000) in HTML format. This can cause memory problems for the browser (particulrly Safari) and prevent javascript commands from functioning. It may also cause the browser to freeze and require restarting. Using the output format options which download results in a file can reduce the problem.

Another option for large requests is to use "Casjobs". Casjobs requires requesting a user name and password, and submitting queries in SQL, but it allows users to submit large search results and save them online. For Kepler, the link ito Casjobs is http://mastweb.stsci.edu/kplrcasjobs/. For other missions, check the Search_Retrieve page in the left gutter.

When queries are submitted as a web service, the default number of rows returned is 2,000, but any value is allowed when max_rows is specified as a query parameter. (See the MAST Web services page for more information.)

Records per page

This parameter controls the number of records displayed per web page. By default, 100 rows or records are displayed per HTML page. Therefore if 400 records are returned, links to 4 pages will be displayed at the top and bottom of the results table. The paging feature however uses javascript, so if javascript is turned off, paging won't work and only the rows shown on the first page can be displayed. This is one reason why increasing the default value may be helpful. The current limit is 1000 rows per page which is recommended for large requests. Note this value is ignored when output formats other than HTML are selected.

Make Rows Distinct

Selecting this checkbox will restrict the display to only rows in which every output column value for a given row is unique. This option is primarily useful when only a small number of columns are displayed (i.e., using the "remove" button to remove default output columns) and when the selected columns have duplicate values. Including columns which already have unique values (e.g. Kepler ID or Data ID) will make the "Distinct" option ineffective.

As an example, a IUE or FUSE user might want to create a list of unique target names for a specific object class/category. He or she would specify the desired object class, select only "object Class" and "target name" for the output columns, click the "Make Rows Distinct" button, then click "Search".

Note that for some missions, columns such as RA, Dec and Magnitude were frequently defined by the observer and often have different values for the same target. If columns such as these are chosen as output columns, there will often be more than one row listed per object name.

Output Format

You may choose any of the following formats for displaying/storing search results. If you are using a browser (e.g., Firefox, etc) to submit a query from a MAST web form, the formats labelled "File: ..." offer a way to download results directly to your local computer. Choose any of the other format options if you want to display the results in the browser, or if you are submitting the request from a program (see MAST web services).

HTML_Table (default) - results returned as a standard HTML table including various links for retrieving data, displaying previews, literature references, plotting spectra, etc.
File: Excel_Spreadsheet - results are stored as an Excel spreadsheet file. (Note: assumes users computer/browser provides support for Excel-format files). The default file name when downloaded is "mission"_search.xls where "mission" is the mission name (e.g., fuse_search.xls).
VOTable - an XML format adopted by the Virtual Observatory (VO) project and displayed in the user's browser. Note coordinates in VOTable format are always in decimal degrees rather than sexagesimal format. For searches returning results from more than one mission and/or target, multiple "resource" tags are created. Searches with a radius of 0 will return a VOTable file listing the output fields for that particular mission/catalog. (For more information on the XML file format, see VOTable documentation.)
Comma-separated values - a simple ASCII array containing column headings followed by rows of comma-separated values. In file upload mode, a blank line is inserted between the search results to separate multiple target queries.
Space-separated values - a simple ASCII array containing column headings followed by rows of space-separated values. In file upload mode, a blank line is inserted between the search results to separate multiple target queries. Note users may want to select coordinates in decimal rather than sexagesimal format to maintain the correspondence between column headings and entries although some column headings may contain blanks as well. (Semi-colons, pipes, or commas are probably safer delimiters.)
IRAF Space-separated values with INDEFs - Like the space-separated format above except empty fields are replaced with the string "INDEF". This format is useful for IRAF-compatibility.
Semi-colon separated values - a simple ASCII array containing column headings followed by rows of semi-colon separated values. In file upload mode, a blank line is inserted between the search results to separate multiple target queries.
Tab-separated values - a simple ASCII array containing tabs for delimiters. May be useful for ingesting into Excel spreadsheets.
Pipe-separated values - a simple ASCII array containing column headings followed by rows of pipe or vertical bar separated values. (Note: not offered in file upload mode.)
Json format - Javascript Object Notation (json) is a simple machine and human-readable, name/value ASCII format supported by many programming languages. (Note: not offered in file upload mode.)
File: comma-separated values - a simple ASCII text file containing column headings followed by rows of comma-separated values. In file upload mode, a blank line is inserted between the search results to separate multiple target queries. Rather than being displayed in the browser, the results are directly downloaded to the users computer using the file name "mission"_search.txt where "mission" is the mission name (e.g., hst_search.txt). Depending on the browser settings, the user may be prompted for a file location before the file is downloaded.
File: Space-separated values - a simple ASCII text file containing column headings followed by rows of space-separated values. In file upload mode, a blank line is inserted between the search results to separate multiple target queries. Rather than being displayed in the browser, the results are directly downloaded to the users computer using the file name "mission"_search.txt where "mission" is the mission name (e.g., hst_search.txt). Depending on the browser settings, the user may be prompted for a file location before the file is downloaded.
File: IRAF Space-separated values with INDEFs - Like the space-separated format above except empty fields are replaced with the string "INDEF". This file format is useful for IRAF-compatibility.
File: Semicolon-separated values - a simple ASCII text file containing column headings followed by rows of semicolon-separated values. In file upload mode, a blank line is inserted between the search results to separate multiple target queries. Rather than being displayed in the browser, the results are directly downloaded to the users computer using the file name "mission"_search.txt where "mission" is the mission name (e.g., hst_search.txt). Depending on the browser settings, the user may be prompted for a file location before the file is downloaded.
File: Tab-separated values - a simple ASCII file containing tabs for delimiters. May be useful for ingesting into Excel spreadsheets.
File: Pipe-separated values - a simple ASCII text file containing column headings followed by rows of pipe-separated values. (Note: not offered in file upload mode.) Rather than being displayed in the browser, the results are directly downloaded to the users computer using the file name "mission"_search.txt where "mission" is the mission name (e.g., hst_search.txt). Depending on the browser settings, the user may be prompted for a file location before the file is downloaded.
File: Json format - Javascript Object Notation (json) is a simple machine and human-readable, name/value ASCII format supported by many programming languages. Rather than being displayed in the browser, the results are directly downloaded to the users computer using the file name "mission"_search.txt where "mission" is the mission name (e.g., hst_search.txt). Depending on the browser settings, the user may be prompted for a file location before the file is downloaded.
File: WGET Commands - This option is only available for certain missions. If selected, a shell script file is output which the user can execute from his own computer to download all the selected light curves or extracted spectral files with one command. The shell script file uses the "WGET" program which is available for most operating systems. Note for Kepler, two WGET options are available: "File: WGET LC commands" will create a script for downloading available light curves, "File: WGET TPF commands" will create a script for downloading target pixel files.

Note: If no entries are found for an entry a "no entries found" message is written in the selected format and the program continues. In all cases, error conditions will cause the database search to abort.