Use the K2 FFI Search form to
search and retrieve
FFI, CLL, ARP, BKG and END class files.
Searches are limited to queries based on file name, class,
start date, end date, release date and/or generation date.
General Search Options and Operators
Search values can be specified in several different ways, depending
on the data type of the field. In all cases, a single value can
be entered (although this is not recommented for floating point values).
In addition,
various operators can be included depending on the data type of the
field as described below.
The data types for each column can be displayed using the "Field Descriptions"
link at the top of all MAST search forms.
Numerical fields - Real (i.e., real,
numeric, float, or double) and Integer (i.e., long) fields can be
specified as a single value, as a single value with numerical operators
such as:
"< n", less than
">= n", greater than or equal
"!= n", not equal to
"\null", is NULL
"n..m", an inclusive range (e.g., "1990 .. 2000").
Bcause of small differences in stored floating point values, specifying
a single floating point value may not return the expected results.
Therefore, numerical or range operators are recommended with floating point
fields queries. Integers may be requested either way.
The number of digits displayed to the right of the
decimal point for floating point numbers is determined solely by the
data type. If the data type is "real" only 1 digit is displayed, and
for "numeric" 2 digits are displayed. "Float" data types have 3, and "double"
can have up to 8 or more.
String fields - String fields, also
known as "char" or "varchar" fields,
can use the following operators:
= (equal) which is the default,
!= (not equal, e.g., != SMALL),
\null for IS NULL ,
!\null for IS NOT NULL,
"*" or "%" wildcard operators (e.g. "Jup*").
By default, string searches use equals ("=") which runs faster
without wild cards but implies matches must be exact. For example, searching on
Target description = "Planet" will not
return an entry for "Planet:Jupiter". (There are some exceptions though such as
searching on Kepler Investigation ID will automatically include wild cards.)
Wildcards are allowed
and encouraged when an exact match is not desired, so in the previous
example, searching on Planet* would return all entries beginning with the word Planet.
Quotes are not necessary for string values.
Since moving to the Microsoft SQL Server database system in 2009, string
searches are no longer case-sensitive. Values will still be displayed in the same
case they were originally entered, but entries will be found regardless of the
case of the searched string (i.e., searching for HST id "go-5916" or "GO-5916"
will return the same entries).
Also, as with any data type, commas can be used to search for multiple entries.
For example, to search for all O3 and B3 stars from the Skiff catalog,
just enter O3*,B3*. Likewise, entering \null,<5 will return values that are
either null or less than 5. However, if nulls are used with comma-separated strings,
specify the strings first and in single quotes (e.g., 'hrs',\null ).
Substring fields - A few fields have a data type of "substring"
for which
wildcard operators are automatically added at the beginning and end of
the input string. For example, entering "Comets" for the K2 Object type
will search for "*comets*".
Commas still work as with other data types, so entering "Exoplanet,red" for the Kepler
condition flag (another substring field)
will search for condition flags = "*exoplanet*" or condition flags = "*red*".
Note these special cases do not apply to casjob searches.
Bibstring fields - Bibstrings are a special
string field for storing ADS bibcodes. A bibstring value has 19 characters,
and its assumed that no operators other than wild cards are included in
queries. This data type was mainly added because ".." was normally interpreted
as a range search request. For a bibstring field, specifying 2101ApJ..*
means find all entries whose bibcode begins with 2010ApJ..
Coordinate fields - Generally you can specify a variety
of formats for Right Ascension and Declination using either decimal
degrees or sexagesimal values (RA in decimal hours is
also available, but only as an output format option). The allowed search
formats are described in detail below.
Date fields - Dates can also be specified in a variety
of formats and can also use operators and inclusive range searches.
Here are some allowed/recommended examples:
> Jul 15 1994
< 01-jan-2010
Dec 1 1995 .. Dec 6 1995
01-jan-2009 .. 15-feb-2010
2009-05-11 17:51:31 (but date match must be exact)
20090115 .. 20100101
dec 2009 .. jan 2010
2009 jan .. 2010 jan
2009 .. 2010
Formats found not to work or not recommended include:
15 Jul 2005 (valid format but only returns entries with a value of 15 Jul 2005 00:00:00)
15JUL2009 (same problem as above)
15-jul-2009 (same problem as above)
2009-01 (same problem as above)
2009-10 .. 2010-01 (doesn't seem to work)
2009-10-01 .. jan-01-2010 (don't mix formats in range searches)
jan-15-2009 .. feb-01-2010 (putting month before day doesn't work with dashes!)
Dec 15 2009 .. Dec 01 2009 (earlier date should be listed first)
Often queries on a single date will fail because the database can store
datetime fields to the millisecond and the matches must be exact. It is
preferable to use a range or the <, > operators.
Note that when the time is not specified, the query will default to 00:00:00.
Therefore without specifying times, a range search would include the starting date
but exclude the ending date. Leaving off the day or month would work similarly.
To see the data type of a particular field, click on the form element
label or any of the help page links.
Note quotes are not needed for any values.
Note, searches on "null" values in fields of any data type
are now possible by entering \null.
By default, the various search criteria will be submitted
using logical AND's. Logical OR's
are not supported on most mission search forms except when using
commas within a single form element such as entering "hc230,srhlw"
for IUE program ID to return entries with ID hc230 OR srhlw..
Information on individual search form elements is listed below.
Note that specific examples given below do not necessarily apply
to all missions. The examples are merely intended to show valid
formats for data entry.
DADS archive classification. Current options include:
full frame images (ffi), long and short cadence collateral
files (CLL, CLS), artifact removal pixel files (ARP),
background pixel files (BKG), and Ancillary Engineering data (ENG).
Each pair of CCDs comprise a module and corresponds to four output channels (two per CCD). Each output is read out by a unique analog signal chain (e.g. amplifier). 2 to 24
OUTPUT is an integer ranging from 1 to 4 describing one of the four output channels. An object can have a non-zero output number if it is off, but near, a CCD. Each module has four output channels (two per CCD). Each output is read out by a unique analog signal chain (e.g. amplifier). 1 to 4
CHANNEL is an integer ranging from 1 to 84 that uniquely specifies the MODULE/OUTPUT pair.There are 21 modules, each with 4 outputs for a total of 84 channels. 1 to 84
You may use these form elements to search on any column(s)
in the mission table. First, select the field
you wish to search from the pulldown menu under the "User-specified field n" heading.
Then, type in the qualification in the corresponding "Field Descriptions" box.
Clicking on the "Field Descriptions" link, will display information on the allowed fields
including the allowed range of values.
As an example, a Kepler user might select "E(B-V)" from the pulldown menu and
enter "< 0.5" in the "Field Descriptions" box.
NOTE only fields which are not already
included on the search form should be selected. Specifying search criteria
for a field that is listed in BOTH the form and in the User Specified field, may
cause either the query to fail or return unexpected results.
This form element allows you to determine which columns
are displayed and in what order.
The initial list contains those designated as defaults, but it
may be possible to add more.
(Its possible a project decided all columns should be
displayed by default.)
To remove a column, highlight the column to be removed
then click the "remove" button to the right of the output columns list.
To remove all columns, click the "remove all" button. This is useful
when only a few output columns are desired. If a search
is submitted with all columns removed, it will display the original
set of default columns.
To add a column, select the desired
column from the pull down menu beneath the list of chosen output columns,
then click the "add" button. The column will be added to the bottom
of the output column list. To add all the available columns, click the
"add all" button.
You can change the order of columns by highlighting a column and
then clicking the up or down buttons to the right of the list of
chosen output columns. Each click moves the column by one position.
Note the "Mark" column, which allows data sets to be retrieved, will
always appear at the top of the list. If desired though it can be removed.
The output columns form element has its own "reset" button to restore
the list of output columns to the original defaults.
This is different than the "Reset" button at the top of the form
which is used to reset the other form elements.
Clicking the "clear form" button will restore the original defaults in
all sections of the form.
One warning, the amount of memory required and possibly the execution time
for a search is proportional to the amount of information returned.
For large requests, users may want to
reduce the number of output columns to the minimum required. Also consider
non-html output formats and the "Skip formatting" option.
By default, when using "file upload mode", the
message "No Records Found Matching Query" (or for non-html output,
"no rows found") is displayed for each entry with no search results.
Also, for non-html output, a line is now added for each entry
in the uploaded file
stating the entry number and target name or coordinates
used for that particular database query.
Clicking this button will prevent either of these messages
from appearing in the output which may be useful
for reducing output from large search results and/or for
parsing the CSV-style output.
If checked when using "file upload mode" to search on coordinates
or targets and when specifying
a csv-like output format, a number will be
prepended to each row of search results indicating the position
of the input target in the uploaded list of targets.
For example, if 10 entries were found for the 5th target listed in the
uploaded file, each of those 10 rows would have a 5 in the first column.
This can be useful for correlating search results to input targets
when multiple targets are searched. Note Data ID search results which
all appear in one output table will NOT have entry numbers since
they are all returned in one database query.
Verb is an integer parameter used by the VO community for specifying the
amount of output returned for a given search request. It is only
available when retrieving data as a web service but works with every
MAST service. Setting verb=3 in a search request is equivalent to
specifying "add all" from a search form;
it will return all the available columns in the output not just the standard
default fields. Currently setting verb to any other value has no effect.
Choose how you want the output rows sorted. You can select
up to three fields to sort on. The rows will be sorted in the order of
the first sort field; if two rows have the same sort field, they will be
sorted in order of the second sort field, and so on. Default sort fields
may be listed, but any field from the pulldown list can be used.
Specifying multiple sort fields may increase execution time.
If you prefer no sorting, you can specify "null" for all 3 fields.
This may speed up the query, but results will be displayed in
the order in which they were originally stored in the database table.
For each field, you can select that the rows be sorted in reverse
order on that field by selecting the reverse checkbox. For example,
you can sort the rows with the most recent observations first by selecting
Observation Date for the first sort field and selecting
the reverse checkbox next to it.
One word of caution: the selected sort field can change the search results
when the query finds more rows than are displayed ( i.e, when the number of
found rows exceeds the value of "maximum records"). For example, for a search
on a particular coordinate that finds 5,000 entries, if the search
is sorted on exposure time
and 1,001 rows are to be displayed (the default), then the 1,001
shortest exposures
from the 5,000 found entries will be displayed
which may not include the entries closest to the
desired position. (It is a good idea to always sort on "ang_sep" for target or
coordinate searches).
Finally, note that when displaying the search results in HTML, further sorting is
possible based on any of the displayed columns simply by clicking the column header.
Even columns using sexigesimal notation can now be correctly sorted. Clicking a
header a 2nd time will reverse the order. Reloading the page will return results
to the original order.
(This sorting is performed using javascript, so javascript needs to be enabled.)
Specifies the format for displaying the primary
equatorial (i.e., RA and Dec) coordinates. The options include:
Sexagesimal - The default format with Right Ascension specified
as hh mm ss.sss and Declination as +/-dd mm ss.ss
( e.g., RA = 12 46 11.091, Dec = -00 30 12.08).
Note an extra digit was added as of June, 2012.
Degrees - Decimal degrees for both RA and Dec with 7
significant figures to the right of the decimal point
(e.g., RA=191.5461912, Dec=-0.5033333).
Note 2 extra digits were added as of June, 2012.
or
Hours - Decimal hours for RA and decimal degrees for Dec
in same format as for decimal degrees (e.g., RA=12.7697512, Dec=-0.5033333).
Note decimal hours = decimal degrees/15.0.
(Note 2 extra digits were added as of June, 2012.)
Any other coordinate fields contained in the searched mission catalog will be
displayed in their original format.
This value specifies the maximum number of rows returned in a
single query. For the standard mission search forms, the current
default is 5,001, but values from 1 to 50,001 are allowed.
For the file upload forms in which multiple targets, data ID's, or
coordinates can be specified, the default is set to 20 rows per file entry,
with allowed values ranging from 1 to 5,000.
Note when displayed as HTML, the latest search scripts will display
500 records per page. Links to the additional pages are shown
on the results page.
This paging feature however does require javascript to be turned on.
Users should be cautioned about retrieving a large number of records
(i.e., > 10,000 - 15,000) in HTML format. This can cause memory
problems for the browser (particulrly Safari) and prevent javascript
commands from functioning. It may also cause the browser to freeze
and require restarting.
Using the output format options which download results in a file can reduce the
problem.
Another option for large requests is to use "Casjobs".
Casjobs requires requesting a user name and password, and submitting
queries in SQL, but it allows users to submit large search results and
save them online. For Kepler, the link ito Casjobs is
http://mastweb.stsci.edu/kplrcasjobs/.
For other missions, check the Search_Retrieve page in the left gutter.
When queries are submitted as a web service, the default number of rows
returned is 2,000, but any value is allowed when max_records is specified
as a query parameter. A practical limit might be 25,000.
(See the MAST Web services page for
more information.)
This parameter controls the number of records displayed per web page.
By default, 500 rows or records are displayed per HTML page. Therefore if
2000 records are returned, links to 4 pages will be displayed at the
top and bottom of the results table.
The paging feature however uses
javascript, so if javascript is turned off, paging won't work
and only the rows shown on the first page can be displayed. This is
one reason why increasing the default value may be helpful.
Note this value is ignored when output formats other than HTML are selected.
Selecting this checkbox will restrict the display to only rows in which
every output column value for a given row is unique. This option is primarily useful
when only a small number of columns are displayed (i.e., using the "remove"
button to remove default output columns) and when the selected columns
have duplicate values. Including columns which already have unique values (e.g.
Kepler ID or Data ID)
will make the "Distinct" option ineffective.
As an example, a IUE or FUSE user might want to create a list of unique target
names for a specific object class/category. He or she would specify the desired object
class, select only "object Class" and "target name" for the output columns,
click the "Make Rows Distinct" button, then click "Search".
Note that for some missions, columns such as RA, Dec and Magnitude were frequently defined
by the observer and often have different values for the same target.
If columns such as these are chosen as output columns, there will often
be more than one row listed per object name.
After the search results are retrieved from the database,
selecting this option will remove columns with all
null values. Zeroes are maintained.
In some cases selecting this option can reduce the execution times,
but it can also take longer depending on the
number of null columns and the number of columns selected.
Removing null columns is primarily useful for
sparsely-populated tables
when a large number of columns are requested. Note for the
HSC summary form this option is selected by default.
After the results are retrieved from the database, some
reformatting is done.
This includes converting decimal degrees to sexagesimal format,
restricting the number of significant numbers displayed for
certain data types,
changing date formats, etc. Since this processing may be applied
to every row and column, and as catalogs keep getting larger,
this step can significantly slow down large requests
(e.g., queries returning more than a few thousand rows).
Checking this box will skip these steps and thereby reduce
execution times.
You may choose any of the
following formats for displaying/storing search results.
If you are using a browser (e.g., Firefox, etc) to submit a query
from a MAST web form, the formats labelled "File: ..." offer a
way to download results directly to your local computer.
The names shown below are the actual values specified
in a GET request or as an API request. The description in parentheses is how the
option will appear on the MAST classic search forms.
HTML_Table (HTML_Table) default - results returned as a
standard HTML table including various links for retrieving data,
displaying previews, literature references, plotting spectra, etc.
Excel_Spreadsheet (Excel_Spreadsheet) - results are downloaded
as an Excel spreadsheet file.
(Note: assumes users computer/browser provides support for Excel-format files).
The default file name when downloaded is "mission"_search.xls where "mission"
is the mission name (e.g., fuse_search.xls).
VOTable (VOTable) - an XML format adopted by the Virtual Observatory (VO)
project and displayed in the user's browser. Note coordinates in VOTable
format are always in decimal degrees rather than sexagesimal format.
For searches returning results from more than one mission and/or target,
multiple "resource" tags are created. Searches with a radius of 0
will return a VOTable file listing the output fields for that particular
mission/catalog. (For more information on the XML file format,
see VOTable documentation.)
CSV (Comma-separated values) - a simple ASCII array containing
column headings followed by rows of comma-separated values. In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
SSV (Space-separated values) - a simple ASCII array containing
column headings followed by rows of space-separated values.
In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
Note users may want to select
coordinates in decimal rather than sexagesimal format to maintain the
correspondence between column headings and entries although some column
headings may contain blanks as well.
(Semi-colons, pipes, or commas are probably safer delimiters.)
IRAF (IRAF Space-separated w/INDEFs) - Like the space-separated format
above except empty fields are replaced with the string "INDEF". This format is useful
for IRAF-compatibility.
COSV (Semicolon separated values) - a simple ASCII array containing
column headings followed by rows of semi-colon separated values. In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
TSV (Tab-separated values) - a simple ASCII array containing
tabs for delimiters. May be useful for ingesting into Excel spreadsheets.
PSV (Pipe-separated values) - a simple ASCII array containing
column headings followed by rows of pipe or vertical bar separated values.
(Note: not offered in file upload mode.)
JSON (Json format) - Javascript Object Notation (json) is a simple machine and
human-readable, name/value ASCII format supported by many programming languages.
(Note: not offered in file upload mode.)
CSV_file (File: comma-separated values) -
a simple ASCII text file containing
column headings followed by rows of comma-separated values. In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
Rather than being displayed in the browser, the results are directly downloaded
to the users computer using the file name "mission"_search.txt where "mission" is
the mission name (e.g., hst_search.txt). Depending on the
browser settings, the user may be prompted for a file location before the
file is downloaded.
SSV_file (File: Space-separated values) -
a simple ASCII text file containing
column headings followed by rows of space-separated values. In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
Rather than being displayed in the browser, the results are directly downloaded
to the users computer using the file name "mission"_search.txt where "mission" is
the mission name (e.g., hst_search.txt). Depending on the
browser settings, the user may be prompted for a file location before the
file is downloaded.
IRAF_file (File: IRAF Space-separated w/INDEFs) -
Like the space-separated format
above except empty fields are replaced with the string "INDEF". This file format is useful
for IRAF-compatibility.
SSV_file (File: Semicolon-separated values) -
a simple ASCII text file containing
column headings followed by rows of semicolon-separated values. In file upload mode,
a blank line is inserted between the search results to separate multiple target queries.
Rather than being displayed in the browser, the results are directly downloaded
to the users computer using the file name "mission"_search.txt where "mission" is
the mission name (e.g., hst_search.txt). Depending on the
browser settings, the user may be prompted for a file location before the
file is downloaded.
TSV_file (File: Tab-separated values) - a simple ASCII file containing
tabs for delimiters. May be useful for ingesting into Excel spreadsheets.
PSV_file (File: Pipe-separated values) -
a simple ASCII text file containing
column headings followed by rows of pipe-separated values.
(Note: not offered in file upload mode.)
Rather than being displayed in the browser, the results are directly downloaded
to the users computer using the file name "mission"_search.txt where "mission" is
the mission name (e.g., hst_search.txt). Depending on the
browser settings, the user may be prompted for a file location before the
file is downloaded.
JSON_file (File: Json format) - Javascript Object Notation (json) is a
simple machine and
human-readable, name/value ASCII format supported by many programming languages.
Rather than being displayed in the browser, the results are directly downloaded
to the users computer using the file name "mission"_search.txt where "mission" is
the mission name (e.g., hst_search.txt). Depending on the
browser settings, the user may be prompted for a file location before the
file is downloaded.
WGET_file (File: WGET Commands) - This option is only available for
certain missions. If selected, a shell script file is
output which the user
can execute from his own computer to download all the selected
files with one command. The shell script
file uses the "WGET" program which is available for most operating systems.
Note for Kepler, two WGET options are available:
"File: WGET LC commands" will create a script for downloading available light curves,
"File: WGET TPF commands" (also available for K2) will create a script for
downloading target pixel files.
For Swift, the WGET option will create a script for downloading
the Sky coordinate images.
CURL_file (File: CURL Commands) - Like the WGET command,
this option is only available for
certain missions. If selected, a shell script file is
output which the user
can execute from his own computer to download all the selected
files with one command. The shell script
file uses the "CURL" program which is available for most operating systems.
Note for Kepler, two CURL options are available:
"File: CURL LC commands" will create a script for downloading available light curves,
"File: CURL TPF commands" (also available for K2) will create a script for
downloading target pixel files.
For Swift, the CURL option will create a script for downloading
the Sky coordinate images.
Note: If no entries are found for an entry a
"no entries found" message is written in the selected format and
the program continues.
In all cases, error conditions will cause the database search to abort.