MAST FTP Service

MAST provides an FTP server for staging data retrieved from DADS. Users can retrieve both public and exclusive access data to this stage, and download it via FTP. This server also provides access to other MAST data, such as data from different missions.

We've switched to FTPS-only

As of 26 Oct 2021, the MAST FTP server archive.stsci.edu no longers support unencrypted FTP connections. Only encrypted FTPS connections (both explicit and implicit) are be supported. Please take this opportunity to upgrade to a client that supports FTPS connections. See below for some suggestions for modern clients.

(sftp will not work, as sftp does not implement the FTP protocol.)

Which client should you use? Command-line ftp clients, as in Linux or MacOS (where you just type ftp on the command line), have not kept pace with the protocol, so they may not work with our server. We strongly recommend using a modern FTP client. GUI clients include Filezilla, Cyberduck, while for command-line or scripted use, you might consider curl, which offers both a command-line client and a library, or the Python ftplib module (see the example code below).

One additional benefit of GUI clients is that many of them, including the ones listed above, support other protocols, such as sftp. Some, also support access to cloud resources, like Dropbox or AWS, making them convenient for accessing and transferring data between diverse remote locations.

PASV Issue for External Hosts Fixed

The previous issue with external hosts using passive-mode (PASV) connections has been fixed. This was known to affect wget, Linux and MacOS command-line clients, and Python's ftplib. If you're still having problems with those clients, please let us know.

---

A Brief FAQ

How do I log in?

First, connect your FTP client to archive.stsci.edu. If the data you are downloading is not exclusive access (that is, is publicly accessible), then you can use an anonymous connection, with "anonymous" as the username and your email address as the password. (We don't do anything with the email address, so you can enter something else if you like.)

If the data you need are exclusive access, then enter your email address (the one that you used when you registered with STScI) as the username, and your registration password as the password. As an authenticated (non-anonymous) user, you will be able to see only your directory and the anonymous directories on the stage, plus any other directories on the server to which you have specifically been granted access.

Will I need to use a username and password for FTPS?

Only if your data requires authorization to download. Data staged as anonymous, or any public data on the FTP site, can still be retrieved as anonymous. FTPS will work the same way as FTP, except that transfers are encrypted.

Why are you switching to FTPS?

For the same reason we went from HTP to HTTPS: security. FTPS will secure FTP transfers in much the same way that HTTPS protects web browsing, by encrypting the channel.

Is FTPS enabled on your server now?

Yes, both FTP and FTPS are currently enabled. We encourage you to make any changes you need before unencrypted FTP is switched off. After that, the server will accept only FTPS connections.

What FTPS clients should I use?

Most modern FTP clients are enabled for FTPS, such as Filezilla, and Cyberduck. Many clients, such as these two, are GUIs that, along with FTPS, can handle cloud storage and other protocols. You may find them useful for other purposes, so they're worth looking into.

Many command-line FTP clients are not enabled for FTPS, and may never be. For a command-line client, you might consider wget, curl, or Python's ftplib.

Can FTPS be scripted?

Yes. The wget and curl clients will work, if they have been compiled with TLS.

As of Python 3.2, ftplib supports TLS connections. Here is an example:

import ftplib
...
ftps = ftplib.FTP_TLS('archive.stsci.edu')
ftps.login(user=..., passwd=...)
ftps.prot_p() # This is a really good idea :)
ftps.cwd('stage')
ftps.cwd(stagedir) # stagedir is something like 'anonymous/anonymous12345'

filenames = ftps.nlst()
for filename in filenames:
    print("getting " + filename)
    with open(filename, 'wb') as fp: 
        ftps.retrbinary('RETR {}'.format(filename), fp.write)

By the way, do we really have to tell you not to store your password in a file? Instead, have your script ask for the password on the command line. The getpass module provides a simple way to do this.

Can I use sftp to connect to your FTPS server?

No, because sftp is not an actual FTP client. It uses ssh and scp to work like an FTP client, but it cannot talk to an FTP or FTPS server. Our server is FTP/FTPS only.

How can I tell if my FTP client can do FTPS?

The easiest way is to try it! To check wget and curl without connecting, you can also check by doing "wget -V" or "curl -V". If the string "ftps" appears in the output, then the client is FTPS-enabled. For other clients, consult the client's documentation.