Data Access and Analysis Tools

The Rubin Science Platform (RSP) is the set of integrated web-based applications, services, and tools to query, visualize, subset, and analyze LSST data - including the DP0.2 simulated data products. Throughout DP0 the RSP will be in active development by Rubin Observatory staff, with access provided to delegates on a shared-risk basis. See Rubin Science Platform Documentation page for additional information.

RSP usage: risks and responsibilities

All RSP users are responsible for being aware of the risks inherent in using software in active development, and must familiarize themselves with these risks prior to using the RSP.

• RSP Risks and Caveats
  • User data: No parachutes!
  • RSP/IDF missing features
  • Warnings and admonishments

Rubin Science Platform (RSP)

During DP0, the Rubin Science Platform (RSP) provides access to Rubin Observatory data products via two services: a Notebook Aspect and a Portal Aspect. The Notebook Aspect enables programmatic interaction with, and analysis of, data products in a python environment. The Portal Aspect provides interactive query, data discovery, and visualization tools. In the Operations era, the RSP users will be able to use the Portal and Notebook aspects in tandem (whereas for DP0.2 there is no connection between the two), and will have access to a third service: the Application Programming Interface (API) aspect. A full description of the Operations-era RSP can be found in the RSP Vision Document (LSE-319), but again, some of that envisioned RSP functionality is not available for DP0 (see RSP usage: risks and responsibilities).

Above is shown the landing page of data.lsst.cloud. The Notebook and Portal Aspects can be accessed by clicking on their respective icons on this dashboard. The API aspect is not available during DP0.2.

Portal Aspect

The Portal Aspect of the RSP provides an environment for data discovery, query, filtering, and visualization. During DP0.2, only catalog data will be accessible via the Portal (not images), and in particular only catalog data generated from the coadded images, and not the single-visit images, which means there is no time-domain data accessible via the Portal during DP0.2). Note that the Portal Aspect has not been under active development recently and it is expected to evolve significantly before the first LSST annual data release.

• Introduction to the RSP Portal Aspect
  • The Portal’s user interface
  • Single Table Queries
  • ADQL Queries

Notebook Aspect

The Notebook Aspect of the RSP provides an environment from which users can access and manipulate Rubin data products on the same machines where those data products reside (i.e., “next-to-the-data” – no downloading of data to your local machine). In particular, the Notebook Aspect offers Python-based access to DP0.2 data products via a custom implementation of web-based JupyterLab Notebooks (JupyterLab documentation). A file browser and a terminal command-line interface are also available within the JupyterLab environment.

Within the RSP Notebook Aspect, users can query DP0.2 data sets, subset and retrieve catalogs, manipulate and display images, calculate derived properties, plot results, reprocess the data, and most other analyses you can imagine performing with Python on astronomical images and catalogs. A stable software environment is provided and maintained for users, which includes many commonly-used packages and the Data processing tools including the LSST Pipelines. For DP0, this environment will only support Python 3. For DP0.2, images can only be accessed via the Notebook Aspect, and not the Portal.

• Introduction to the RSP Notebook Aspect
  • How to log in, navigate, and log out of JupyterLab
  • How to use the JupyterLab terminal
  • How to use a Jupyter notebook
  • Jupyter notebook frequently asked questions
  • Troubleshooting tips
  • Additional RSP notebook documentation

API Aspect

The API (Application Programming Interface) Aspect of the RSP enables programmatic access to the Rubin data products via Virtual Observatory (VO) interfaces. Users will be able to remotely access the LSST data and DAC services using tools they’re already familiar with, e.g. TOPCAT or libraries such as Astropy. The Portal and Notebook Aspects of the RSP make use of the same APIs to internally access the LSST datasets.

The API Aspect of the RSP is very powerful and will eventually allow for federation with other astronomical archives, bringing added value to the LSST dataset.

• Introduction to the RSP API Aspect
  • The Table Access Protocol (TAP) service

Data processing tools

Documentation for the LSST Science Pipelines, a software package which is available to all RSP users via the Notebook Aspect, can be found at pipelines.lsst.io.

A brief summary of how the LSST Science Pipelines were used to create the DP0.2 data products can be found in this Data Processing Overview.

To view a list of packages available to you in the Notebook Aspect of the RSP, type pip install in a terminal.