Calibrating HBV hydrological model using SCE run locally outside container forced with ERA5 forcing data

Calibrating HBV hydrological model using SCE run locally outside container forced with ERA5 forcing data#

In this notebook we will demonstrate how to calibrate the HBV model with Shuffled Complex Evolution (SCE) and works as an example for calibration methods that use a function that needs to be passed to an optimization algorithm.

Instead of an ensemble of models, we will need to construct a function that does the entire model run.

We will use two new packages not used before:

the sceua package implements the optimisation algorithm. Infortunately we run into a dependency problem: ESMValTool, which is part of eWaterCycle, requires numpy 1.26.4, sceua requires numpy >= 2. So we install, and subsequently de-install sceua in this notebook.
the hydrobm package implements common metrics used in hydrology, including the kge, which we use here.

This assumes you have already seen the notebook on how to do Monte Carlo calibration in eWaterCycle

# General python
import warnings
warnings.filterwarnings("ignore", category=UserWarning)

import numpy as np
from pathlib import Path
import pandas as pd
import matplotlib.pyplot as plt
import xarray as xr
import json

# Niceties
from rich import print
from tqdm import tqdm

# General eWaterCycle
import ewatercycle
import ewatercycle.models
import ewatercycle.forcing

# We need the sceua package. If that is not available on your machine,
# uncomment the line below to install it

!pip install sceua

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: sceua in /home/mmelotto/.local/lib/python3.12/site-packages (0.1.0)
Collecting numpy>=2 (from sceua)
  Using cached numpy-2.3.5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (62 kB)
Requirement already satisfied: scipy>=1.15 in /home/mmelotto/.local/lib/python3.12/site-packages (from sceua) (1.16.3)
Using cached numpy-2.3.5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (16.6 MB)
Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 1.26.4
    Uninstalling numpy-1.26.4:
      Successfully uninstalled numpy-1.26.4
  WARNING: The scripts f2py and numpy-config are installed in '/home/mmelotto/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
numba 0.60.0 requires numpy<2.1,>=1.22, but you have numpy 2.3.5 which is incompatible.
esmvalcore 2.11.1 requires numpy!=1.24.3,<2.0.0, but you have numpy 2.3.5 which is incompatible.
Successfully installed numpy-2.3.5

# Sceua package
import sceua

# We make use of the hydrobm package to calculate metrics
!pip install hydrobm

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: hydrobm in /home/mmelotto/.local/lib/python3.12/site-packages (1.0.2)
Requirement already satisfied: numpy in /home/mmelotto/.local/lib/python3.12/site-packages (from hydrobm) (2.3.5)
Requirement already satisfied: pandas in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from hydrobm) (2.2.3)
Requirement already satisfied: scipy in /home/mmelotto/.local/lib/python3.12/site-packages (from hydrobm) (1.16.3)
Requirement already satisfied: xarray in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from hydrobm) (2024.11.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from pandas->hydrobm) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from pandas->hydrobm) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from pandas->hydrobm) (2024.2)
Requirement already satisfied: packaging>=23.2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from xarray->hydrobm) (24.2)
Requirement already satisfied: six>=1.5 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas->hydrobm) (1.16.0)

from hydrobm.metrics import calculate_metric

# Parameters
region_id = None
settings_path = "settings.json"

# Load settings
# Read from the JSON file
with open(settings_path, "r") as json_file:
    settings = json.load(json_file)

print(settings)

{
    'caravan_id': 'hysets_02OJ024',
    'calibration_start_date': '1985-08-01T00:00:00Z',
    'calibration_end_date': '1995-07-31T00:00:00Z',
    'future_start_date': '2029-08-01T00:00:00Z',
    'future_end_date': '2049-07-31T00:00:00Z',
    'base_path': '/home/mmelotto/my_data/workshop_canada',
    'path_caravan': '/home/mmelotto/my_data/workshop_canada/forcing_data/hysets_02OJ024/caravan',
    'path_shape': '/home/mmelotto/my_data/workshop_canada/forcing_data/hysets_02OJ024/caravan/hysets_02OJ024.shp',
    'path_ERA5': '/home/mmelotto/my_data/workshop_canada/forcing_data/hysets_02OJ024/ERA5',
    'path_CMIP6': '/home/mmelotto/my_data/workshop_canada/forcing_data/hysets_02OJ024/CMIP6',
    'path_output': '/home/mmelotto/my_data/workshop_canada/output_data/hysets_02OJ024',
    'hbv_parameters_manual': [7.377, 1.0, 40.0, 0.5, 0.169, 1.0, 0.1, 0.01, 7.393]
}

Pre-generated observations of discharge from caravan#

Here we re-load the disharge data we generated in this notebook.

# Load the caravan forcing object
caravan_data_object = ewatercycle.forcing.sources['CaravanForcing'].load(directory=settings['path_caravan'])
print(caravan_data_object)

CaravanForcing(
    start_time='1985-08-01T00:00:00Z',
    end_time='1995-07-31T00:00:00Z',
    directory=PosixPath('/home/mmelotto/my_data/workshop_canada/forcing_data/hysets_02OJ024/caravan'),
    shape=PosixPath('/home/mmelotto/my_data/workshop_canada/forcing_data/hysets_02OJ024/caravan/hysets_02OJ024.shp'
),
    filenames={
        'evspsblpot': 'hysets_02OJ024_1985-08-01_1995-07-31_evspsblpot.nc',
        'tasmin': 'hysets_02OJ024_1985-08-01_1995-07-31_tasmin.nc',
        'tas': 'hysets_02OJ024_1985-08-01_1995-07-31_tas.nc',
        'pr': 'hysets_02OJ024_1985-08-01_1995-07-31_pr.nc',
        'tasmax': 'hysets_02OJ024_1985-08-01_1995-07-31_tasmax.nc',
        'Q': 'hysets_02OJ024_1985-08-01_1995-07-31_Q.nc'
    }
)

Pre-generated ERA5 forcing data for HBV model#

Here we load the ERA5 data we generated in this notebook

load_location = Path(settings['path_ERA5']) / "work" / "diagnostic" / "script" 
ERA5_forcing_object = ewatercycle.forcing.sources["LumpedMakkinkForcing"].load(directory=load_location)

print(ERA5_forcing_object)

LumpedMakkinkForcing(
    start_time='1985-08-01T00:00:00Z',
    end_time='1995-07-31T00:00:00Z',
    directory=PosixPath('/home/mmelotto/my_data/workshop_canada/forcing_data/hysets_02OJ024/ERA5/work/diagnostic/sc
ript'),
    shape=PosixPath('/home/mmelotto/my_data/workshop_canada/forcing_data/hysets_02OJ024/ERA5/work/diagnostic/script
/hysets_02OJ024.shp'),
    filenames={
        'pr': 'OBS6_ERA5_reanaly_1_day_pr_1985-1995.nc',
        'tas': 'OBS6_ERA5_reanaly_1_day_tas_1985-1995.nc',
        'rsds': 'OBS6_ERA5_reanaly_1_day_rsds_1985-1995.nc',
        'evspsblpot': 'Derived_Makkink_evspsblpot.nc'
    }
)

Calibration basics and objective function#

In model calibration, we are looking for a set of parameters such that when the model is run with that set of parameters, we get the best model output. What “best” means differs per application or research question. In general, we like some model outputs to be as close as possible to observations. For this purpose we create an objective function that takes the model output of interest and observations as inputs and calculates some score that shows goodness of fit. Here we use a RMS difference function:

def calibrationObjective(model_output, observation, start_calibration, end_calibration, metric_name = 'rmse'):
    '''A function that takes in two dataFrames, interpolates the model output to the
    observations and calculates the average absolute difference between the two. '''

    # Combine the two in one dataFrame and interpolate, to make sure times match
    hydro_data = pd.concat([model_output.reindex(observation.index, method = 'ffill'), observation], axis=1,
                           keys=['model','observation'])

    # Only select the calibration period
    hydro_data = hydro_data[hydro_data.index > pd.to_datetime(pd.Timestamp(start_calibration).date())]
    hydro_data = hydro_data[hydro_data.index < pd.to_datetime(pd.Timestamp(end_calibration).date())]

    obs = hydro_data['observation'].to_numpy()
    sim = hydro_data['model'].to_numpy()

    metric_value  = calculate_metric(obs,sim,metric_name)

    return metric_value

    # match metric:
    #     case 'rmse':
    #         # Calculate mean absolute difference
    #         squareDiff = (sim - obs)**2
    #         rmse = np.sqrt(np.mean(squareDiff))
    #         return rmse
    #     case 'nse':
    #         # Calculate Nash Shutcliff Efficiency
    #         nse = 1 - np.sum((obs - sim) ** 2) / np.sum((obs - np.mean(obs)) ** 2)
    #         return nse
    #     case 'kge':
    #         # Calculate Kling Gupta Efficiency
    #         if (np.std(obs) == 0) or (np.std(sim) == 0):
    #             r = 0
    #         else:
    #             r = np.corrcoef(obs, sim)[0, 1]
    #         alpha = np.std(sim) / np.std(obs)
    #         beta = np.mean(sim) / np.mean(obs)
    #         kge = 1 - np.sqrt((r - 1) ** 2 + (alpha - 1) ** 2 + (beta - 1) ** 2)
    #         return kge

Create a function that runs the entire model and evaluates it#

# Create an array with parameter values.

# First set minimum and maximum values on the parameters
p_min_initial = np.array([0,   0.2,  40,    .5,   .001,   1,     .01,  .0001,   0.01])
p_max_initial = np.array([8,    1,  800,   4,    .3,     10,    .1,   .01,   10.0])

p_initial = (p_min_initial + p_min_initial)/2

# Print parameter names and values for first ensemble member
param_names = ["Imax", "Ce", "Sumax", "Beta", "Pmax", "Tlag", "Kf", "Ks", "FM"]
print(list(zip(param_names, np.round(p_initial, decimals=3))))

[
    ('Imax', 0.0),
    ('Ce', 0.2),
    ('Sumax', 40.0),
    ('Beta', 0.5),
    ('Pmax', 0.001),
    ('Tlag', 1.0),
    ('Kf', 0.01),
    ('Ks', 0.0),
    ('FM', 0.01)
]

# Set initial state values
#               Si,  Su, Sf, Ss, Sp
s_0 = np.array([0,  100,  0,  5,  0])

# Create a dataframe for the observations
ds_observation = xr.open_mfdataset([caravan_data_object['Q']]).to_pandas()
ds_observation = ds_observation['Q']

def runModel(params, forcing_object, observations, initial_state, start_calibration, end_calibration):
    # Create model object, notice the forcing object.
    model = ewatercycle.models.HBVLocal(forcing=forcing_object)

    # Create config file in model.setup()
    config_file, _ = model.setup(parameters=params, initial_storage=initial_state,
                                start_time = start_calibration, end_time = end_calibration)
    # Initialize model
    model.initialize(config_file)
    # Run model, capture calculated discharge and timestamps
    Q_m = []
    time = []
    while model.time < model.end_time:
        model.update()
        Q_m.append(model.get_value("Q")[0])
        time.append(pd.Timestamp(model.time_as_datetime))
    # Finalize model (shuts down container, frees memory)
    model.finalize()

    # Make a pandas series
    model_output = pd.Series(data=Q_m, name="Modelled discharge", index=time)

    return model_output

def runModelReturnObjective(params, forcing_object, observations, initial_state, start_calibration, end_calibration):

    model_output = runModel(params, forcing_object, observations, initial_state, start_calibration, end_calibration)

    kge = calibrationObjective(model_output, observations, start_calibration, end_calibration, metric_name = 'kge')

    # kge is best if high, so we do one minus kge to make sure that sce optimizes correctly
    # since sce always minimizes

    score = 1 - kge

    return score

Find best parameter set#

By calculating the objective function for each model output, we can search the combination of parameters with the lowest objective function.

# Define parameter bounds as a sequence of (min, max) pairs
bounds = [(p_min_initial[n], p_max_initial[n]) for n in range(len(p_min_initial))]

print(bounds)

[
    (0.0, 8.0),
    (0.2, 1.0),
    (40.0, 800.0),
    (0.5, 4.0),
    (0.001, 0.3),
    (1.0, 10.0),
    (0.01, 0.1),
    (0.0001, 0.01),
    (0.01, 10.0)
]

# Run optimization
result = sceua.minimize(runModelReturnObjective, bounds, args=(caravan_data_object, ds_observation, s_0, settings['calibration_start_date'],
                                                              settings['calibration_end_date']), seed=42, max_workers=1)

# Access the optimization results
best_params = result.x
best_function_value = result.fun
num_iterations = result.nit
num_function_evaluations = result.nfev

print(best_function_value)

0.6912783106713029

model_output = runModel(best_params, ERA5_forcing_object, ds_observation, s_0, 
                                              settings['calibration_start_date'], settings['calibration_end_date'])

# Create a dataframe for the observations
ds_observation = xr.open_mfdataset([caravan_data_object['Q']]).to_pandas()

# Make a plot of the model output of the minimum value
ds_observation["Q"].plot(label="Observed discharge")
ax = model_output.plot(lw=2.5,label="Modelled discharge")

plt.legend()
plt.grid()
plt.ylabel("Discharge [mm/d]")
plt.xlabel("Year")
plt.show()

../../_images/745ebb050ccbaff59c29f161723f46eb9905cc216b9173be3b735fbaf976e699.png

df_best = model_output

df_best.index = df_best.index.tz_localize("UTC")

# Make a plot of the model output of the minimum value
ds_observation["Q"].plot(label='Observed discharge')
ax = df_best.plot(lw=2.5, label="Modelled discharge")

plt.legend()
plt.grid()
plt.title(f"{settings["caravan_id"]}")
plt.ylabel("Discharge [mm/d]")
plt.xlabel("Year")
plt.show()

../../_images/ccd85f5963a156bee252930ff597c416b33e518fda3fa8b8146826c1244567b4.png

Save results#

We want to save these results to file to be able to load them in other studies.

settings["SCE_calibration_parameters"] = list(best_params)
print(list(best_params))

[
    8.0,
    0.9221151284150986,
    43.075768252525116,
    0.7030120292069942,
    0.001,
    1.0184850975935131,
    0.1,
    0.006890405982621616,
    3.2446361176000305
]

# Write to a JSON file
with open("settings.json", "w") as json_file:
    json.dump(settings, json_file, indent=4)

# Remove all temporary directories made by the optimization algo.

!rm -rf hbvlocal_*

# Re-install correct numpy for ESMVAlTool
!pip install esmvaltool

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: esmvaltool in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (2.11.0)
Requirement already satisfied: aiohttp in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (3.11.9)
Requirement already satisfied: cartopy in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.24.0)
Requirement already satisfied: cdo in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.6.0)
Requirement already satisfied: cdsapi in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.7.5)
Requirement already satisfied: cf-units in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (3.3.0)
Requirement already satisfied: cftime in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.6.4)
Requirement already satisfied: cmocean in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (4.0.3)
Requirement already satisfied: dask in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2024.11.2)
Requirement already satisfied: distributed in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2024.11.2)
Requirement already satisfied: ecmwf-api-client in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.6.3)
Requirement already satisfied: eofs in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.0.0)
Requirement already satisfied: ESMPy in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (8.4.2)
Requirement already satisfied: esmvalcore in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2.11.1)
Requirement already satisfied: esmf-regrid>=0.7.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.11.0)
Requirement already satisfied: fiona in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.10.1)
Requirement already satisfied: fire in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.7.0)
Requirement already satisfied: fsspec in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2024.10.0)
Requirement already satisfied: GDAL in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (3.9.2)
Requirement already satisfied: jinja2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (3.1.4)
Requirement already satisfied: joblib in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.4.2)
Requirement already satisfied: lime in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.2.0.1)
Requirement already satisfied: mapgenerator>=1.0.5 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.0.7)
Requirement already satisfied: matplotlib<3.9.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (3.8.4)
Requirement already satisfied: natsort in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (8.4.0)
Requirement already satisfied: nc-time-axis in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.4.1)
Requirement already satisfied: netCDF4 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.7.1)
Requirement already satisfied: numba in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.60.0)
Requirement already satisfied: numpy!=1.24.3 in /home/mmelotto/.local/lib/python3.12/site-packages (from esmvaltool) (2.3.5)
Requirement already satisfied: openpyxl in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (3.1.5)
Requirement already satisfied: packaging in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (24.2)
Requirement already satisfied: pandas!=2.2.0,!=2.2.1,!=2.2.2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2.2.3)
Requirement already satisfied: progressbar2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (4.5.0)
Requirement already satisfied: psyplot in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.5.1)
Requirement already satisfied: psy-maps in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.5.0)
Requirement already satisfied: psy-reg in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.5.0)
Requirement already satisfied: psy-simple in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.5.1)
Requirement already satisfied: pyproj>=2.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (3.7.0)
Requirement already satisfied: pys2index in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.1.5)
Requirement already satisfied: python-dateutil in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2.9.0.post0)
Requirement already satisfied: pyyaml in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (6.0.2)
Requirement already satisfied: rasterio in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.4.1)
Requirement already satisfied: requests in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2.32.3)
Requirement already satisfied: ruamel.yaml in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.18.6)
Requirement already satisfied: scikit-image in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.24.0)
Requirement already satisfied: scikit-learn>=1.4.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (1.5.2)
Requirement already satisfied: scipy in /home/mmelotto/.local/lib/python3.12/site-packages (from esmvaltool) (1.16.3)
Requirement already satisfied: scitools-iris>=3.6.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (3.11.0)
Requirement already satisfied: seaborn in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.13.2)
Requirement already satisfied: seawater in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (3.3.5)
Requirement already satisfied: shapely>=2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2.0.6)
Requirement already satisfied: xarray>=0.12.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2024.11.0)
Requirement already satisfied: xesmf>=0.7.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (0.8.7)
Requirement already satisfied: xgboost>1.6.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2.1.2)
Requirement already satisfied: xlsxwriter in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (3.2.0)
Requirement already satisfied: zarr in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvaltool) (2.18.3)
Requirement already satisfied: ConfigArgParse in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from mapgenerator>=1.0.5->esmvaltool) (1.7)
Requirement already satisfied: lxml in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from mapgenerator>=1.0.5->esmvaltool) (5.3.0)
Requirement already satisfied: plotly in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from mapgenerator>=1.0.5->esmvaltool) (5.24.1)
Requirement already satisfied: chart-studio in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from mapgenerator>=1.0.5->esmvaltool) (1.1.0)
Requirement already satisfied: config in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from mapgenerator>=1.0.5->esmvaltool) (0.5.1)
Requirement already satisfied: contourpy>=1.0.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from matplotlib<3.9.0->esmvaltool) (1.3.1)
Requirement already satisfied: cycler>=0.10 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from matplotlib<3.9.0->esmvaltool) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from matplotlib<3.9.0->esmvaltool) (4.55.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from matplotlib<3.9.0->esmvaltool) (1.4.7)
Requirement already satisfied: pillow>=8 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from matplotlib<3.9.0->esmvaltool) (11.0.0)
Requirement already satisfied: pyparsing>=2.3.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from matplotlib<3.9.0->esmvaltool) (3.2.0)
Requirement already satisfied: pytz>=2020.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from pandas!=2.2.0,!=2.2.1,!=2.2.2->esmvaltool) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from pandas!=2.2.0,!=2.2.1,!=2.2.2->esmvaltool) (2024.2)
Requirement already satisfied: certifi in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from pyproj>=2.1->esmvaltool) (2024.8.30)
Requirement already satisfied: six>=1.5 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from python-dateutil->esmvaltool) (1.16.0)
Requirement already satisfied: threadpoolctl>=3.1.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from scikit-learn>=1.4.0->esmvaltool) (3.5.0)
Requirement already satisfied: xxhash in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from scitools-iris>=3.6.1->esmvaltool) (3.5.0)
Requirement already satisfied: pyshp>=2.3 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from cartopy->esmvaltool) (2.3.1)
Requirement already satisfied: click>=8.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from dask->esmvaltool) (8.1.7)
Requirement already satisfied: cloudpickle>=3.0.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from dask->esmvaltool) (3.1.0)
Requirement already satisfied: partd>=1.4.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from dask->esmvaltool) (1.4.2)
Requirement already satisfied: toolz>=0.10.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from dask->esmvaltool) (1.0.0)
Requirement already satisfied: cf-xarray>=0.5.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from xesmf>=0.7.1->esmvaltool) (0.10.0)
Requirement already satisfied: sparse>=0.8.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from xesmf>=0.7.1->esmvaltool) (0.15.4)
Requirement already satisfied: llvmlite<0.44,>=0.43.0dev0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from numba->esmvaltool) (0.43.0)
Collecting numpy!=1.24.3 (from esmvaltool)
  Using cached numpy-2.0.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from aiohttp->esmvaltool) (2.4.4)
Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from aiohttp->esmvaltool) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from aiohttp->esmvaltool) (24.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from aiohttp->esmvaltool) (1.5.0)
Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from aiohttp->esmvaltool) (6.1.0)
Requirement already satisfied: propcache>=0.2.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from aiohttp->esmvaltool) (0.2.0)
Requirement already satisfied: yarl<2.0,>=1.17.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from aiohttp->esmvaltool) (1.18.0)
Requirement already satisfied: datapi in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from cdsapi->esmvaltool) (0.1.1)
Requirement already satisfied: tqdm in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from cdsapi->esmvaltool) (4.67.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from requests->esmvaltool) (3.4.0)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from requests->esmvaltool) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from requests->esmvaltool) (2.2.3)
Requirement already satisfied: locket>=1.0.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from distributed->esmvaltool) (1.0.0)
Requirement already satisfied: msgpack>=1.0.2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from distributed->esmvaltool) (1.1.0)
Requirement already satisfied: psutil>=5.8.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from distributed->esmvaltool) (6.1.0)
Requirement already satisfied: sortedcontainers>=2.0.5 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from distributed->esmvaltool) (2.4.0)
Requirement already satisfied: tblib>=1.6.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from distributed->esmvaltool) (3.0.0)
Requirement already satisfied: tornado>=6.2.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from distributed->esmvaltool) (6.4.2)
Requirement already satisfied: zict>=3.0.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from distributed->esmvaltool) (3.0.0)
Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from jinja2->esmvaltool) (3.0.2)
Requirement already satisfied: dask-jobqueue in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (0.9.0)
Requirement already satisfied: esgf-pyclient>=0.3.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (0.3.1)
Requirement already satisfied: filelock in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (3.16.1)
Requirement already satisfied: geopy in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (2.4.1)
Requirement already satisfied: humanfriendly in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (10.0)
Requirement already satisfied: isodate>=0.7.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (0.7.2)
Requirement already satisfied: nested-lookup in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (0.2.25)
  Using cached numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Requirement already satisfied: prov in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (2.0.0)
Requirement already satisfied: py-cordex in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (0.9.0)
Requirement already satisfied: pybtex in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (0.24.0)
Requirement already satisfied: stratify>=0.3 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (0.3.0)
Requirement already satisfied: yamale in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esmvalcore->esmvaltool) (5.2.1)
Requirement already satisfied: click-plugins>=1.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from fiona->esmvaltool) (1.1.1)
Requirement already satisfied: cligj>=0.5 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from fiona->esmvaltool) (0.7.2)
Requirement already satisfied: termcolor in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from fire->esmvaltool) (2.5.0)
Requirement already satisfied: networkx>=2.8 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from scikit-image->esmvaltool) (3.4.2)
Requirement already satisfied: imageio>=2.33 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from scikit-image->esmvaltool) (2.36.1)
Requirement already satisfied: tifffile>=2022.8.12 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from scikit-image->esmvaltool) (2024.9.20)
Requirement already satisfied: lazy-loader>=0.4 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from scikit-image->esmvaltool) (0.4)
Requirement already satisfied: et-xmlfile in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from openpyxl->esmvaltool) (2.0.0)
Requirement already satisfied: python-utils>=3.8.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from progressbar2->esmvaltool) (3.9.1)
Requirement already satisfied: docrep>=0.3 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from psyplot->esmvaltool) (0.3.2)
Requirement already satisfied: funcargparse in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from psyplot->esmvaltool) (0.2.5)
Requirement already satisfied: statsmodels in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from psy-reg->esmvaltool) (0.14.4)
Requirement already satisfied: affine in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from rasterio->esmvaltool) (2.4.0)
Requirement already satisfied: ruamel.yaml.clib>=0.2.7 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from ruamel.yaml->esmvaltool) (0.2.8)
Requirement already satisfied: asciitree in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from zarr->esmvaltool) (0.3.3)
Requirement already satisfied: numcodecs>=0.10.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from zarr->esmvaltool) (0.14.1)
Requirement already satisfied: fasteners in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from zarr->esmvaltool) (0.17.3)
Requirement already satisfied: defusedxml in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esgf-pyclient>=0.3.1->esmvalcore->esmvaltool) (0.7.1)
Requirement already satisfied: requests-cache in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esgf-pyclient>=0.3.1->esmvalcore->esmvaltool) (1.2.1)
Requirement already satisfied: webob in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esgf-pyclient>=0.3.1->esmvalcore->esmvaltool) (1.8.9)
Requirement already satisfied: myproxyclient in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from esgf-pyclient>=0.3.1->esmvalcore->esmvaltool) (2.1.1)
Requirement already satisfied: typing_extensions>3.10.0.2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from python-utils>=3.8.1->progressbar2->esmvaltool) (4.12.2)
Requirement already satisfied: Cython in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from stratify>=0.3->esmvalcore->esmvaltool) (3.0.11)
Requirement already satisfied: retrying>=1.3.3 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from chart-studio->mapgenerator>=1.0.5->esmvaltool) (1.3.3)
Requirement already satisfied: multiurl>=0.3.2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from datapi->cdsapi->esmvaltool) (0.3.3)
Requirement already satisfied: geographiclib<3,>=1.52 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from geopy->esmvalcore->esmvaltool) (2.0)
Requirement already satisfied: tenacity>=6.2.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from plotly->mapgenerator>=1.0.5->esmvaltool) (9.0.0)
Requirement already satisfied: rdflib>=4.2.1 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from prov->esmvalcore->esmvaltool) (7.1.1)
Requirement already satisfied: pooch in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from py-cordex->esmvalcore->esmvaltool) (1.8.2)
Requirement already satisfied: latexcodec>=1.0.4 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from pybtex->esmvalcore->esmvaltool) (2.0.1)
Requirement already satisfied: patsy>=0.5.6 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from statsmodels->psy-reg->esmvaltool) (1.0.1)
Requirement already satisfied: pyOpenSSL in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from myproxyclient->esgf-pyclient>=0.3.1->esmvalcore->esmvaltool) (24.2.1)
Requirement already satisfied: platformdirs>=2.5.0 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from pooch->py-cordex->esmvalcore->esmvaltool) (4.3.6)
Requirement already satisfied: cattrs>=22.2 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from requests-cache->esgf-pyclient>=0.3.1->esmvalcore->esmvaltool) (24.1.2)
Requirement already satisfied: url-normalize>=1.4 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from requests-cache->esgf-pyclient>=0.3.1->esmvalcore->esmvaltool) (1.4.3)
Requirement already satisfied: cryptography<44,>=41.0.5 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from pyOpenSSL->myproxyclient->esgf-pyclient>=0.3.1->esmvalcore->esmvaltool) (43.0.3)
Requirement already satisfied: cffi>=1.12 in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from cryptography<44,>=41.0.5->pyOpenSSL->myproxyclient->esgf-pyclient>=0.3.1->esmvalcore->esmvaltool) (1.17.1)
Requirement already satisfied: pycparser in /opt/conda/envs/ewatercycle2/lib/python3.12/site-packages (from cffi>=1.12->cryptography<44,>=41.0.5->pyOpenSSL->myproxyclient->esgf-pyclient>=0.3.1->esmvalcore->esmvaltool) (2.22)
Using cached numpy-1.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.0 MB)
Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 2.3.5
    Uninstalling numpy-2.3.5:
      Successfully uninstalled numpy-2.3.5
  WARNING: The script f2py is installed in '/home/mmelotto/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
sceua 0.1.0 requires numpy>=2, but you have numpy 1.26.4 which is incompatible.
Successfully installed numpy-1.26.4