Public Data Access Overview / JupyterLab Workspace
17 Aug 2023 Update
Dear all current Lab users: the environment has been significantly updated.
The underlying OS that your Lab session runs on has been updated from Ubuntu 18 (bionic) to Ubuntu 20 (focal). Most importantly, the available Python version has been updated to 3.11, and all Python package versions are updated. In the future, please use pip install package to add additional packages, or for better package management, create a new mamba environment. If you need a particular package or updated version, please make a request. If you previously had a custom python (e.g. miniconda/anaconda) environment installed, this has likely been broken/removed. Please see if the new default environment is suitable for your purpose, and try to use the existing installation of conda/mamba instead of installing it from scratch. The IDL version has been updated to 8.8, the Matlab version to 2022a, the Julia version to 1.9, and the R version to 4.3. JupyterLab itself has been updated to v3.6.5, with various new features. Usefully, on the upper-right of the Lab, you will now see a memory usage indicator, showing the default 10GB memory limit.
Are you more of a VSCode user? You can connect VSCode running on your computer to your remote Lab. This lets you develop your code/notebook as you normally would in VSCode, but code will execute on the remote server with direct access to the simulation data. This is a two step process: (i) create an API token on your Lab as shown above, (ii) in the VSCode command palette, select 'Jupyter: New Jupyter Notebook'. Then, under 'Select Kernel' (upper right), choose 'Existing Jupyter Server...' and enter the URL as https://data-eu.tng-project.org/lab/user/{email_address}/?token={api_token}. Then you can select any available kernel, such as Python 3.
What is it, and how does it work?
JupyterLab is the evolution of the Jupyter Notebook (previously called IPython). It is a next-generation, web-based user interface suitable for scientific data analysis. In addition to the previous 'notebook' format, JupyterLab also enables a traditional workflow based around a collection of scripts on a filesystem, text editors, a console, and command-line execution. It provides an experience nearly indistinguishable from working directly on a remote computing cluster via SSH.
Computation is language-agnostic, and the following languages are currently supported: Python 3.11, IDL, Matlab, R, Julia, C/C++, and more.
This service enables access to the complete mirror of Illustris[TNG] data hosted at the MPCDF near Munich, Germany. It works by launching a JupyterLab instance on a system at MPCDF and connecting your web browser to it. From there, all Illustris[TNG] data will be directly available for analysis. A small amount of persistent storage means under-development scripts, intermediary analysis results, and work-in-progress figures will all stick around if you log out and pick up again later. Everything is fairly intuitive, and we suggest you simply give it a try! (Or take a look at a quick demo of the interface).
Please note: this service is presently experimental and we provide no guarantees of any kind. In particular, we cannot currently guarantee any level of uninterrupted access or resource availability -- e.g. the service may be down and/or non-functional at times, it may not be possible to complete a particular computation, and all data should be considered temporary and not backed up.
Request access:
You need to log in (or register) before requesting access to the JupyterLab functionality.