1. Validating terrestrial water storage and evaporation

Important parts of the water balance and assessment of model performance are the terrestrial water storage (TWS) and evaporation (E). In this example, TWS is validated against GRACE/GRACE-FO data and E is benchmarked with GLEAM data. pcrglobwb_utils facilitates the validation process by providing scores per polygon of a shp-file.

1.1. Preambule

Loading required packages and showing package versions used.

%matplotlib inline
import pcrglobwb_utils
import xarray as xr
import pandas as pd
import numpy as np
import geopandas as gpd
import matplotlib
import matplotlib.pyplot as plt
import rasterio as rio
import spotpy as sp
import os, sys
import datetime

print('this notebook was created using Python', str((sys.version)), 'on a', str(sys.platform),'on', datetime.datetime.now())

this notebook was created using Python 3.8.5 | packaged by conda-forge | (default, Jul 31 2020, 01:53:45) [MSC v.1916 64 bit (AMD64)] on a win32 on 2021-12-03 17:03:53.367184

pcrglobwb_utils.utils.print_versions()

pcrglobwb_utils version 0.3.3
pandas version 1.2.4
xarray version 0.18.2
numpy version 1.20.3
geopandas version 0.9.0
rasterio version 1.1.5
rioxarray version 0.4.2

1.2. Select the area

pcrglobwb_utils applies the TWS and E validation for an area as user-specified by means of a shp-file containing one or more polygons. Per polygon, both simulated and observed data is averaged per time step. With the resulting time series, performance metrics can be computed and visualized. For the example here, the water provinces located in Tanzania are used. Hence, the first step is to load the shp-file.

TZA_waterProvinces = pcrglobwb_utils.spatial_validation.validate_per_shape(shp_fo='example_data/Tanzania_shp/waterProvinces.geojson',
                                                                           shp_key='watprovID')

reading shp-file C:\Users\hoch0001\Documents\_code\pcrglobwb_utils\examples\example_data\Tanzania_shp\waterProvinces.geojson

Quickly inspect the extent information of the loaded data.

TZA_waterProvinces.extent_gdf.head()

	OBJECTID	MAJORBASIN	COUNTRY	REGION	watprov	watprovID	km2	Shape_Leng	Shape_Area	geometry
0	12	Africa, East Central Coast	Tanzania	Iringa	Tanzania_Iringa_Africa, East Central Coast	25	88460.9	22.002607	7.195788	POLYGON ((36.29721 -5.12083, 36.29583 -5.12503...
1	13	Africa, East Central Coast	Tanzania	Lindi	Tanzania_Lindi_Africa, East Central Coast	28	85004.1	15.485085	6.865913	MULTIPOLYGON (((40.19511 -10.26096, 40.19125 -...
2	14	Africa, East Central Coast	Tanzania	Manyara	Tanzania_Manyara_Africa, East Central Coast	29	36772.8	13.405812	2.976842	POLYGON ((37.17537 -2.76838, 37.17193 -2.76958...
3	15	Africa, East Central Coast	Tanzania	Mbeya	Tanzania_Mbeya_Africa, East Central Coast	30	17342.4	8.245606	1.414085	POLYGON ((34.15102 -7.38556, 34.15304 -7.39140...
4	16	Africa, East Central Coast	Tanzania	Morogoro	Tanzania_Morogoro_Africa, East Central Coast	31	146718.0	29.906623	11.605295	MULTIPOLYGON (((39.25203 -8.00781, 39.24966 -8...

1.2.1. TWS validation

For validating simulated TWS, netCDF-files of both simulation and observation need to be provided. Per polygon, time series are retrieved and evaluated. This yields a dataframe with R and RMSE value per polygon.

watprov_gdf = TZA_waterProvinces.against_GRACE(PCR_nc_fo='example_data/GRACE/totalWaterStorageThickness_monthAvg_output_2010_Tanzania.nc',
                                               GRACE_nc_fo='example_data/GRACE/GRACE_data_2010_Tanzania.nc')

reading GRACE file C:\Users\hoch0001\Documents\_code\pcrglobwb_utils\examples\example_data\GRACE\GRACE_data_2010_Tanzania.nc
reading PCR-GLOBWB file C:\Users\hoch0001\Documents\_code\pcrglobwb_utils\examples\example_data\GRACE\totalWaterStorageThickness_monthAvg_output_2010_Tanzania.nc
extract raw data from nc-files
clipping nc-files to extent of shp-file
computing R and RMSE for polygon with key identifier watprovID 25
computing R and RMSE for polygon with key identifier watprovID 28
computing R and RMSE for polygon with key identifier watprovID 29
computing R and RMSE for polygon with key identifier watprovID 30
computing R and RMSE for polygon with key identifier watprovID 31
computing R and RMSE for polygon with key identifier watprovID 32
computing R and RMSE for polygon with key identifier watprovID 33
computing R and RMSE for polygon with key identifier watprovID 443
computing R and RMSE for polygon with key identifier watprovID 444
computing R and RMSE for polygon with key identifier watprovID 445
computing R and RMSE for polygon with key identifier watprovID 1021
computing R and RMSE for polygon with key identifier watprovID 1226
computing R and RMSE for polygon with key identifier watprovID 1227
computing R and RMSE for polygon with key identifier watprovID 1228
computing R and RMSE for polygon with key identifier watprovID 1229

As pcrglobwb_utils returns a geo-dataframe, the R and RMSE values can also be plotted.

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(20, 10))
watprov_gdf.plot(column='R', ax=ax1, cmap='magma', legend=True, legend_kwds={'orientation':'horizontal', 'label':'R'})
watprov_gdf.plot(column='RMSE', ax=ax2, cmap='viridis', legend=True, legend_kwds={'orientation':'horizontal', 'label':'RMSE'})
plt.tight_layout();

1.2.2. E benchmark

The same workflow is followed when benchmarking simulated E with GLEAM data.

watprov_gdf = TZA_waterProvinces.against_GLEAM(PCR_nc_fo='example_data/GLEAM/totalEvaporation_monthTot_output_2010_Tanzania.nc',
                                               GLEAM_nc_fo='example_data/GLEAM/GLEAM_data_2010_Tanzania.nc')

reading GLEAM file C:\Users\hoch0001\Documents\_code\pcrglobwb_utils\examples\example_data\GLEAM\GLEAM_data_2010_Tanzania.nc
reading PCR-GLOBWB file C:\Users\hoch0001\Documents\_code\pcrglobwb_utils\examples\example_data\GLEAM\totalEvaporation_monthTot_output_2010_Tanzania.nc
extract raw data from nc-files
clipping nc-files to extent of shp-file
computing R and RMSE for polygon with key identifier watprovID 25
computing R and RMSE for polygon with key identifier watprovID 28
computing R and RMSE for polygon with key identifier watprovID 29
computing R and RMSE for polygon with key identifier watprovID 30
computing R and RMSE for polygon with key identifier watprovID 31
computing R and RMSE for polygon with key identifier watprovID 32
computing R and RMSE for polygon with key identifier watprovID 33
computing R and RMSE for polygon with key identifier watprovID 443
computing R and RMSE for polygon with key identifier watprovID 444
computing R and RMSE for polygon with key identifier watprovID 445
computing R and RMSE for polygon with key identifier watprovID 1021
computing R and RMSE for polygon with key identifier watprovID 1226
computing R and RMSE for polygon with key identifier watprovID 1227
computing R and RMSE for polygon with key identifier watprovID 1228
computing R and RMSE for polygon with key identifier watprovID 1229

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(20, 10), sharey=True)
watprov_gdf.plot(column='R', ax=ax1, cmap='magma', legend=True, legend_kwds={'orientation':'horizontal', 'label':'R'})
watprov_gdf.plot(column='RMSE', ax=ax2, cmap='viridis', legend=True, legend_kwds={'orientation':'horizontal', 'label':'RMSE'})
plt.tight_layout()

1.3. Command line functionality

Similar process, but less interactive is the command line functionality. Using click it is possible to evaluate model output with any other gridded dataset per polygon.

pcrglobwb_utils.eval.POLY('example_data/Tanzania_shp/waterProvinces.geojson', 'example_data/GLEAM/totalEvaporation_monthTot_output_2010_Tanzania.nc','example_data/GLEAM/GLEAM_data_2010_Tanzania.nc', './_OUT/TEST', 'watprovID', 'E', 'total_evaporation', conversion_factor=100)

INFO -- start.
INFO -- pcrglobwb_utils version 0.3.3.
INFO -- saving output to folder C:\Users\hoch0001\Documents\_code\pcrglobwb_utils\examples\_OUT\TEST
INFO -- reading observed variable E from example_data/GLEAM/GLEAM_data_2010_Tanzania.nc
INFO -- reading simulated variable total_evaporation from example_data/GLEAM/totalEvaporation_monthTot_output_2010_Tanzania.nc
INFO -- reading polygons from C:\Users\hoch0001\Documents\_code\pcrglobwb_utils\examples\example_data\Tanzania_shp\waterProvinces.geojson
INFO -- evaluating each polygon
INFO -- storing dictionary to C:\Users\hoch0001\Documents\_code\pcrglobwb_utils\examples\_OUT\TEST\output_dict.csv.
INFO -- storing polygons to C:\Users\hoch0001\Documents\_code\pcrglobwb_utils\examples\_OUT\TEST\output_polygons.geojson.
INFO -- done.
INFO -- run time: 0:00:05.927825.

import shutil
shutil.rmtree('./_OUT/TEST')