How to run a PyPSA-Eur study on a server with GEMS

This tutorial is a workflow to run a PyPSA-Eur study on a remote Linux server using GEMS. PyPSA-Eur is an open-source energy system model covering the European power grid coded in PyPSA format whose native format is not compatible with Antares Modeler. The package PyPSA-to-GEMS-Converter converts PyPSA format studies into GEMS readable studies.

Configure a simple PyPSA-Eur study with a minimal settings (one area only, 3 days), download the data.
Convert to GEMS format using the Python package PyPSA-to-GEMS-Converter inside a Docker-hosted Jupyter notebook.
Run the simulation with Antares Modeler from within the same notebook.

The Jupyter environment runs inside Docker on the remote server and is accessed locally via VS Code Remote SSH.

Requirements

A remote Linux server with Docker and Docker Compose installed
VS Code with the Remote - SSH extension installed locally

The configuration of the VS Code Remote SSH linked to remote server running Jupyter docker image is detailed in this tutorial VS Code Remote SSH + Jupyter tutorial.

Part 1 — Clone the GEMS repository

These steps are run on the remote server (connect via SSH first).

The GEMS repository contains the Docker configuration and the Jupyter notebook needed for the conversion step and run the GEMS simulation of the converted study.

git clone https://github.com/AntaresSimulatorTeam/GEMS.git
cd GEMS

Part 2 — Set up and run PyPSA-Eur

These steps are run on the remote server.

This part focuses on setting up a simple PyPSA-Eur study (France only, 3 days) on the server.

2.1 Install Pixi

Pixi is the package manager used by PyPSA-Eur.

curl -fsSL https://pixi.sh/install.sh | bash
source ~/.bashrc  # or restart the terminal

2.2 Clone PyPSA-Eur

git clone https://github.com/PyPSA/pypsa-eur.git
cd pypsa-eur

2.3 Install the environment

pixi install

This reads pixi.toml and installs all dependencies into .pixi/envs/default/. The first run takes a few minutes.

2.4 Configure the study

This part is important because it determines the amount of data downloaded from PyPSA-Eur open data. In this example, to create a small study easy to convert, the study only contains the FR node and data for 3 days.

First, copy the default config:

cp config/config.default.yaml config/config.yaml

Then edit config/config.yaml to set the target countries, number of clusters, planning horizon, and other study parameters.

Example config file with oly FR node

# PyPSA-Eur: simple one-year electricity study (runs on a normal PC)
#
# Use with:  snakemake -call solve_elec_networks --configfile config/config.regional.yaml -j 2
# Resume after crash:  add  --rerun-incomplete
benchmark:
  enabled: false

countries:
  - FR

scenario:
  clusters: [1]
  opts: [""]
  sector_opts: [""]
  planning_horizons: [2050]

snapshots:
  start: "2013-01-01"
  end: "2013-01-03"
  inclusive: left

solving:
  solver:
    name: highs
    options: "highs-default"
  mem_mb: 10000

atlite:
  nprocesses: 2

sector:
  hydrogen_underground_storage_locations:
    - onshore
  regional_co2_sequestration_potential:
    enable: false
  district_heating:
    supply_temperature_approximation:
      rolling_window_ambient_temperature: 48

plotting:
  costs_threshold: 0.001
  energy_threshold: 0.1

2.5 Run the workflow

Now, we can download data from PyPSA Eur.

Warning

This download can last for 10 minutes for only one node.

pixi run snakemake -c all all -j 4

To resume after an interrupted run:

pixi run snakemake -c all all -j 4 --rerun-incomplete

Part 3 — Build docker image and Open the Jupyter notebook via Remote SSH

We have all what is needed by the docker image : - PyPSA Eur data - GEMS repo with the jupyer notebook and docker configuration

We can create now the docker image inside the server :

In VS Code, open the Command Palette (Ctrl+Shift+P) and select Remote-SSH: Connect to Host.
Access to /home/ubuntu/GEMS/doc/getting-started/tutorial-two-pypsa-eur/
Start the container by the command
```
cd docker-tuto-pypsa-gems
docker compose up --build
```
Docker Compose will build the image from docker-tuto-pypsa-gems/Dockerfile_PyPSA using tutorial-two-pypsa-eur/ as the build context and start the container. The Jupyter notebook will be exposed on port 8889.
Open your browser at http://localhost:8889.
Open tutorial_pypsa_eur.ipynb and run the cells in order to: