SWOT Hydrology Science Application Tutorial on Local Machine

Retrieving SWOT attributes (WSE, width, slope) and plotting a longitudinal profile along a river or over a basin

Requirement

This tutorial can be run on a local environment. For the AWS cloud version of this notebook, click here.

Earthdata Login

An Earthdata Login account is required to access data, as well as discover restricted data, from the NASA Earthdata system. Thus, to access NASA data, you need Earthdata Login. Please visit https://urs.earthdata.nasa.gov to register and manage your Earthdata Login account. This account is free to create and only takes a moment to set up.

This code runs using SWOT Level 2 Data Products (Version C, aka 2.0).

Notebook Authors: Arnaud Cerbelaud, Jeffrey Wade, & Cassie Nickles, NASA Jet Propulsion Laboratory - California Institute of Technology

Learning Objectives

1. Retrieve SWOT hydrological attributes on river reaches. Query reaches by:
   • River name
   • Downstream tracing from reach id (e.g. headwater to outlet) for river longitudinal profiles
   • Upstream tracing from reach id (e.g. outlet to full river network) for watershed analysis
2. Plot a time series of WSE, width, slope data on the filtered data
3. Visualize an interactive map of WSE, width, slope data on the filtered data

Import Packages

import xarray as xr
import pandas as pd
import geopandas as gpd
import numpy as np
import matplotlib.pyplot as plt
import hvplot.xarray
import earthaccess
import zipfile

pd.set_option('display.max_columns', None) #make sure all columns displayed for shapefiles

C:\Users\nickles\Anaconda3\envs\env-swot-wrkshp\lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Authenticate

Authenticate your Earthdata Login (EDL) information using the earthaccess python package as follows:

earthaccess.login() # Login with your EDL credentials if asked

<earthaccess.auth.Auth at 0x175fd8679a0>

1. Retrieve SWOT hydrological attributes on river reaches within the AWS cloud (Cal/Val data)

What data should we download?

• Optional step: Get the .kmz file of SWOT passes/swaths for the and import it into Google Earth for visualization
• Determine which pass number corresponds to the river/basin you want to look at! #### Search for multiple days of data

# Enter pass number(s)
pass_number    = ["354", "091"]   #Susquehanna River in NA
# e.g. 341, 576, 298 for Connecticut in NA, "236", "514", "542", "085", "363", "057", "335", "029" for Rhine in EU

# Enter continent code
continent_code = "NA"     # e.g. "AF", "NA", "EU", "SI", "AS", "AU", "SA", "AR", "GR"

# Retrieves granulev and links list from the passes we want, in this case by passing to `earthdata.search_data` function the data collection shortname and temporal bounds
links_list = []
for p in range(len(pass_number)):
    river_results = earthaccess.search_data(short_name = 'SWOT_L2_HR_RIVERSP_2.0', 
                                        temporal = ('2024-01-25 00:00:00', '2024-05-16 23:59:59'),
                                        granule_name = "*Reach*_" + pass_number[p] + "_" + continent_code + "*")
    for r in range(len(river_results)):
        river_link = earthaccess.results.DataGranule.data_links(river_results[r], access='external')[0]
        links_list.append(river_link)

Granules found: 5
Granules found: 5

#links_list

Download files to new data_downloads folder

earthaccess.download(links_list, "./data_downloads")

QUEUEING TASKS | : 100%|██████████| 10/10 [00:00<00:00, 998.98it/s]
PROCESSING TASKS | : 100%|██████████| 10/10 [00:06<00:00,  1.45it/s]
COLLECTING RESULTS | : 100%|██████████| 10/10 [00:00<00:00, 9804.36it/s]

['data_downloads\\SWOT_L2_HR_RiverSP_Reach_010_354_NA_20240206T150945_20240206T150952_PIC0_01.zip',
 'data_downloads\\SWOT_L2_HR_RiverSP_Reach_011_354_NA_20240227T115450_20240227T115458_PIC0_01.zip',
 'data_downloads\\SWOT_L2_HR_RiverSP_Reach_012_354_NA_20240319T083953_20240319T084000_PIC0_01.zip',
 'data_downloads\\SWOT_L2_HR_RiverSP_Reach_013_354_NA_20240409T052714_20240409T052715_PIC0_01.zip',
 'data_downloads\\SWOT_L2_HR_RiverSP_Reach_014_354_NA_20240430T020958_20240430T021000_PIC0_01.zip',
 'data_downloads\\SWOT_L2_HR_RiverSP_Reach_010_091_NA_20240128T055759_20240128T055802_PIC0_01.zip',
 'data_downloads\\SWOT_L2_HR_RiverSP_Reach_011_091_NA_20240218T024304_20240218T024307_PIC0_01.zip',
 'data_downloads\\SWOT_L2_HR_RiverSP_Reach_012_091_NA_20240309T232808_20240309T232810_PIC0_01.zip',
 'data_downloads\\SWOT_L2_HR_RiverSP_Reach_013_091_NA_20240330T201303_20240330T201314_PIC0_01.zip',
 'data_downloads\\SWOT_L2_HR_RiverSP_Reach_014_091_NA_20240420T165809_20240420T165820_PIC0_01.zip']

Unzip selected files

The native format for this data is a .zip file, and we want the .shp file within the .zip file, so we must first extract the data to open it. First, we’ll programmatically get the filename we just downloaded, and then extract all data to the data_downloads folder.

filenames = []
for l in range(len(links_list)):
    filenames.append(links_list[l].split("/")[-1])

for f in range(len(filenames)):
    with zipfile.ZipFile(f'data_downloads/{filenames[f]}', 'r') as zip_ref:
        zip_ref.extractall('data_downloads')

# replace zip file names with .shp ending
filename_shps = [filename.replace('zip', 'shp') for filename in filenames]

#initialize opened list of files
SWOT_HR_shps = []

# Loop through queried granules to open and stack all acquisition dates
for j in range(len(filename_shps)):
    SWOT_HR_shps.append(gpd.read_file(f'data_downloads/{filename_shps[j]}')) 

Aggregate unzipped files into dataframe

# Combine granules from all acquisition dates into one dataframe
SWOT_HR_df = gpd.GeoDataFrame(pd.concat(SWOT_HR_shps, ignore_index=True))

# Sort dataframe by reach_id and time
SWOT_HR_df = SWOT_HR_df.sort_values(['reach_id', 'time'])

SWOT_HR_df

	reach_id	time	time_tai	time_str	p_lat	p_lon	river_name	wse	wse_u	wse_r_u	wse_c	wse_c_u	slope	slope_u	slope_r_u	slope2	slope2_u	slope2_r_u	width	width_u	width_c	width_c_u	area_total	area_tot_u	area_detct	area_det_u	area_wse	d_x_area	d_x_area_u	layovr_val	node_dist	loc_offset	xtrk_dist	dschg_c	dschg_c_u	dschg_csf	dschg_c_q	dschg_gc	dschg_gc_u	dschg_gcsf	dschg_gc_q	dschg_m	dschg_m_u	dschg_msf	dschg_m_q	dschg_gm	dschg_gm_u	dschg_gmsf	dschg_gm_q	dschg_b	dschg_b_u	dschg_bsf	dschg_b_q	dschg_gb	dschg_gb_u	dschg_gbsf	dschg_gb_q	dschg_h	dschg_h_u	dschg_hsf	dschg_h_q	dschg_gh	dschg_gh_u	dschg_ghsf	dschg_gh_q	dschg_o	dschg_o_u	dschg_osf	dschg_o_q	dschg_go	dschg_go_u	dschg_gosf	dschg_go_q	dschg_s	dschg_s_u	dschg_ssf	dschg_s_q	dschg_gs	dschg_gs_u	dschg_gssf	dschg_gs_q	dschg_i	dschg_i_u	dschg_isf	dschg_i_q	dschg_gi	dschg_gi_u	dschg_gisf	dschg_gi_q	dschg_q_b	dschg_gq_b	reach_q	reach_q_b	dark_frac	ice_clim_f	ice_dyn_f	partial_f	n_good_nod	obs_frac_n	xovr_cal_q	geoid_hght	geoid_slop	solid_tide	load_tidef	load_tideg	pole_tide	dry_trop_c	wet_trop_c	iono_c	xovr_cal_c	n_reach_up	n_reach_dn	rch_id_up	rch_id_dn	p_wse	p_wse_var	p_width	p_wid_var	p_n_nodes	p_dist_out	p_length	p_maf	p_dam_id	p_n_ch_max	p_n_ch_mod	p_low_slp	geometry
623	71353200021	-1.000000e+12	-1.000000e+12	no_data	55.167022	-83.792685	no_data	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	15.682142	0.390864	-1.000000e+12	-1.000000e+12	34738.1	8.658179e+02	4879.7	865.8	34738.1	-1.000000e+12	-1.000000e+12	0.0000	49.762508	6067.85339	9582.71875	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	29360570	29360570	3	503349248	0.859529	2	-999	1	0	0.000000	2	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	1	1	71353200031, no_data, no_data, no_data	71353300013, no_data, no_data, no_data	6.30	13.561736	105.0	2821.089	75	19531.502	15088.937191	-1.000000e+12	0	3	1	0	LINESTRING (-83.72202 55.19853, -83.72249 55.1...
1246	71353200021	-1.000000e+12	-1.000000e+12	no_data	55.167022	-83.792685	no_data	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	7.486161	0.326650	-1.000000e+12	-1.000000e+12	19855.9	8.663920e+02	3447.3	866.4	19855.9	-1.000000e+12	-1.000000e+12	0.0000	43.631674	5903.60669	9575.10742	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	29360570	29360570	3	503349248	0.826382	2	-999	1	0	0.000000	2	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	-1.000000e+12	1	1	71353200031, no_data, no_data, no_data	71353300013, no_data, no_data, no_data	6.30	13.561736	105.0	2821.089	75	19531.502	15088.937191	-1.000000e+12	0	3	1	0	LINESTRING (-83.72202 55.19853, -83.72249 55.1...
0	71353200021	7.605474e+08	7.605474e+08	2024-02-06T15:09:45Z	55.167022	-83.792685	no_data	1.224520e+01	1.229139e+02	1.229139e+02	-1.000000e+12	-1.000000e+12	6.604933e-04	8.264110e-04	8.264055e-04	2.123580e-03	-1.000000e+12	7.787265e+03	472.798720	5.509722	-1.000000e+12	-1.000000e+12	1426750.9	1.662653e+04	1372903.5	16626.5	1426750.9	-1.000000e+12	-1.000000e+12	1.0717	351.300666	5925.45970	10148.90430	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	8389051	8389051	1	49166	0.037741	2	-999	1	12	0.160000	1	-4.559296e+01	-4.601600e-06	-8.702870e-02	-7.983203e-03	-7.488814e-03	5.243788e-03	-2.306894e+00	-7.915965e-02	-5.575578e-03	6.624387e-02	1	1	71353200031, no_data, no_data, no_data	71353300013, no_data, no_data, no_data	6.30	13.561736	105.0	2821.089	75	19531.502	15088.937191	-1.000000e+12	0	3	1	0	LINESTRING (-83.72202 55.19853, -83.72249 55.1...
2288	71353200021	7.677582e+08	7.677582e+08	2024-04-30T02:09:59Z	55.167022	-83.792685	no_data	1.022890e+01	2.153190e+00	2.151310e+00	-1.000000e+12	-1.000000e+12	8.898481e-04	3.383387e-04	3.383254e-04	9.786866e-04	-1.000000e+12	1.072564e+00	1062.914025	22458.887768	-1.000000e+12	-1.000000e+12	16038243.0	3.388807e+08	15509073.6	338880746.9	16038243.0	-1.000000e+12	-1.000000e+12	3.5170	301.749245	0.72457	8744.38086	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	8651192	8651192	2	540686	0.032994	2	-999	0	47	0.626667	1	-4.559886e+01	-7.991000e-07	5.839980e-02	-5.397310e-03	-5.449628e-03	4.173906e-04	-2.333552e+00	-2.406165e-02	-4.880356e-03	-5.004102e-02	1	1	71353200031, no_data, no_data, no_data	71353300013, no_data, no_data, no_data	6.30	13.561736	105.0	2821.089	75	19531.502	15088.937191	-1.000000e+12	0	3	1	0	LINESTRING (-83.72202 55.19853, -83.72249 55.1...
1	71353200031	7.605474e+08	7.605474e+08	2024-02-06T15:09:45Z	55.065842	-83.910348	no_data	2.504020e+01	2.226200e-01	2.036100e-01	-1.000000e+12	-1.000000e+12	4.786071e-04	4.781869e-05	4.772449e-05	6.136892e-04	-1.000000e+12	6.422207e-03	774.105581	2.399666	-1.000000e+12	-1.000000e+12	12726254.2	3.945038e+04	12059256.4	39450.4	12726254.2	-1.000000e+12	-1.000000e+12	2.7142	443.137074	0.21234	18331.82422	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	8389049	8389049	1	14	0.052411	2	-999	0	65	0.792683	1	-4.564788e+01	-6.814000e-06	-8.709270e-02	-7.972184e-03	-7.460387e-03	5.252804e-03	-2.304898e+00	-7.869844e-02	-5.578940e-03	1.252514e-01	1	1	71353200041, no_data, no_data, no_data	71353200021, no_data, no_data, no_data	18.60	14.102569	78.0	521.854	82	35971.449	16439.946275	-1.000000e+12	0	2	1	0	LINESTRING (-83.82561 55.11666, -83.82606 55.1...
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
4138	77111400131	7.597367e+08	7.597367e+08	2024-01-28T05:58:03Z	9.248879	-83.595282	no_data	4.537900e+02	5.563720e+01	5.563713e+01	-1.000000e+12	-1.000000e+12	1.030139e-02	7.506768e-04	7.506708e-04	1.068534e-02	-1.000000e+12	2.372435e+02	73.773174	1.902120	-1.000000e+12	-1.000000e+12	233620.2	6.023514e+03	232041.6	6023.5	233620.2	-1.000000e+12	-1.000000e+12	1.6579	83.991911	2112.25447	-8317.41406	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	8651192	8651192	2	575498	0.006757	0	-999	1	3	0.065217	0	1.390648e+01	-4.393370e-05	1.777141e-01	2.190485e-02	2.213364e-02	1.969452e-03	-2.190920e+00	-2.548915e-01	-5.267987e-03	1.273845e-01	1	1	77111400156, no_data, no_data, no_data	77111400121, no_data, no_data, no_data	465.75	310.206781	42.0	305.147	46	144265.798	9111.713368	-1.000000e+12	0	1	1	0	LINESTRING (-83.56668 9.23370, -83.56695 9.233...
5364	77111400131	7.615394e+08	7.615394e+08	2024-02-18T02:43:07Z	9.248879	-83.595282	no_data	4.686492e+02	1.177957e+01	1.177922e+01	-1.000000e+12	-1.000000e+12	5.947913e-03	5.867003e-04	5.866927e-04	6.269888e-03	-1.000000e+12	5.281537e+00	76.280653	1.685040	-1.000000e+12	-1.000000e+12	502898.4	1.110903e+04	486712.1	11109.0	502898.4	-1.000000e+12	-1.000000e+12	5.6604	120.007974	1015.69909	-8555.04297	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	8651192	8651192	2	575502	0.032186	0	-999	1	12	0.260870	0	1.391410e+01	-4.556600e-06	8.129047e-02	-1.756892e-02	-1.808319e-02	1.699407e-03	-2.191133e+00	-3.351565e-01	-1.424369e-02	9.252236e-02	1	1	77111400156, no_data, no_data, no_data	77111400121, no_data, no_data, no_data	465.75	310.206781	42.0	305.147	46	144265.798	9111.713368	-1.000000e+12	0	1	1	0	LINESTRING (-83.56668 9.23370, -83.56695 9.233...
6597	77111400131	7.633421e+08	7.633421e+08	2024-03-09T23:28:11Z	9.248879	-83.595282	no_data	4.712829e+02	1.583400e+00	1.580840e+00	-1.000000e+12	-1.000000e+12	-1.447151e-02	3.505161e-04	3.505033e-04	1.686153e-03	-1.000000e+12	2.747979e-01	225.756844	936.700742	-1.000000e+12	-1.000000e+12	1524839.8	6.326801e+06	1524839.8	6326800.6	1524839.8	-1.000000e+12	-1.000000e+12	5.4087	256.615979	41.22357	-7121.99072	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	25428408	25428408	2	575498	0.000000	0	-999	1	11	0.239130	0	1.399303e+01	-1.014890e-05	-2.465366e-01	-6.451763e-04	1.575324e-03	1.251583e-03	-2.184477e+00	-2.664711e-01	-1.627269e-02	2.222131e-01	1	1	77111400156, no_data, no_data, no_data	77111400121, no_data, no_data, no_data	465.75	310.206781	42.0	305.147	46	144265.798	9111.713368	-1.000000e+12	0	1	1	0	LINESTRING (-83.56668 9.23370, -83.56695 9.233...
7840	77111400131	7.651448e+08	7.651448e+08	2024-03-30T20:13:15Z	9.248879	-83.595282	no_data	5.033688e+02	1.079103e+01	1.079066e+01	-1.000000e+12	-1.000000e+12	-5.918540e-03	3.357183e-04	3.357049e-04	2.376273e-03	-1.000000e+12	4.037809e+00	45.977785	2.350709	-1.000000e+12	-1.000000e+12	200550.9	1.025357e+04	200550.9	10253.6	200550.9	-1.000000e+12	-1.000000e+12	0.7090	268.654031	396.36985	-7184.39990	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	25428408	25428408	2	576514	0.000000	0	-999	1	3	0.065217	0	1.400514e+01	-2.760910e-05	1.245001e-01	1.271093e-02	1.286979e-02	7.459380e-04	-2.183828e+00	-2.000982e-01	-2.024420e-02	1.023069e-02	1	1	77111400156, no_data, no_data, no_data	77111400121, no_data, no_data, no_data	465.75	310.206781	42.0	305.147	46	144265.798	9111.713368	-1.000000e+12	0	1	1	0	LINESTRING (-83.56668 9.23370, -83.56695 9.233...
9083	77111400131	7.669475e+08	7.669475e+08	2024-04-20T16:58:21Z	9.248879	-83.595282	no_data	4.509162e+02	1.195103e+01	1.195069e+01	-1.000000e+12	-1.000000e+12	-1.309620e-04	8.698252e-04	8.698200e-04	-1.090830e-03	-1.000000e+12	9.954200e+00	143.277459	2.845567	-1.000000e+12	-1.000000e+12	968977.4	1.924441e+04	968977.4	19244.4	968977.4	-1.000000e+12	-1.000000e+12	1.6762	263.612865	641.24569	-7429.97461	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	-1.000000e+12	-1.000000e+12	-1.000000e+12	-999	25428408	25428408	2	573450	0.000000	0	-999	1	11	0.239130	0	1.402630e+01	-2.812970e-05	1.647883e-01	-1.099033e-02	-1.080735e-02	2.686874e-04	-2.192133e+00	-2.167554e-01	-2.049551e-02	1.166847e-01	1	1	77111400156, no_data, no_data, no_data	77111400121, no_data, no_data, no_data	465.75	310.206781	42.0	305.147	46	144265.798	9111.713368	-1.000000e+12	0	1	1	0	LINESTRING (-83.56668 9.23370, -83.56695 9.233...

9084 rows × 127 columns

Exploring the dataset

What acquisition dates and rivers do our downloaded files cover?

print('Available dates are:')
print(np.unique([i[:10] for i in SWOT_HR_df['time_str']]))
print('Available rivers are:')
print(np.unique([i for i in SWOT_HR_df['river_name']]))

Available dates are:
['2024-01-28' '2024-02-06' '2024-02-18' '2024-02-27' '2024-03-09'
 '2024-03-19' '2024-03-30' '2024-04-09' '2024-04-20' '2024-04-30'
 'no_data']
Available rivers are:
['Abitibi River' 'Abitibi River; Moose River' 'Albany River'
 'Allegheny River' 'Anacostia River; Lower Potomac River'
 'Appomattox River' 'Attawapiskat River' 'Black River'
 'Black River; Cape Fear River' 'Black River; Oswego River'
 'Blanche River' 'Brunswick River; Cape Fear River' 'Cape Fear River'
 'Cape Fear River; Haw River' 'Cape Fear River; Northeast Cape Fear River'
 'Cashie River' 'Castor River' 'Cattaraugus Creek' 'Chenango River'
 'Chickahominy River' 'Chickahominy River; James River'
 'Chickahominy River; Warwick River' 'Chuckatuck Creek'
 'Chuckatuck Creek; Elizabeth River'
 'Chuckatuck Creek; Jones Creek; Pagan River'
 'Chuckatuck Creek; Nansemond River' 'Clyde River' 'Conestoga River'
 'Conodoguinet Creek' 'Conodoguinet Creek; Susquehanna River'
 'Cooper River' 'Crowe River' 'Dan River' 'Dan River; Roanoke River'
 'Deep River' 'East Branch Eighteenmile Creek'
 'East Branch Eighteenmile Creek; Niagara River' 'Elizabeth River'
 'Elizabeth River; Western Branch Elizabeth River' 'Ellicott Creek'
 'Frederick House River' 'Ganargua Creek' 'Genesee River' 'Goose Creek'
 'Grasse River' 'Great Pee Dee River' 'Great Pee Dee River; Sampit River'
 'Great Pee Dee River; Waccamaw River' 'Haw River' 'James River'
 'Jones Creek' 'Jones Creek; Pagan River' 'Juniata River' 'Koksoak River'
 'Kwetabohigan River' 'Kwetabohigan River; Moose River'
 'Little Wicomico River' 'Lower Potomac River'
 'Lower Potomac River; Upper Potomac River' 'Madawaska River'
 'Mattagami River' 'Mattagami River; Moose River' 'Mattawa River'
 'Missinabi River' 'Mississipi River' 'Mississippi River' 'Mohawk River'
 'Moira River' 'Monocacy River' 'Monocacy River; Upper Potomac River'
 'Moose River' 'Nanjemoy Creek' 'Nansemond River' 'Neuse River'
 'Niagara River' 'Nomini Creek' 'North Branch Susquehanna River'
 'North Driftwood River' 'North French River' 'North Santee River'
 'Northeast Cape Fear River' 'Oneida River' 'Oswegatchie River'
 'Oswegatchie River; Saint Lawrence River' 'Oswego River' 'Otonabee River'
 'Ottawa River' 'Pagan River' 'Pagan River; Warwick River'
 'Racquette River' 'Racquette River; Saint Lawrence River'
 'Rappahannock River' 'Raquette River' 'Raystown Branch Juniata River'
 'Roanoke River' 'Ruisseau de la Brasserie' 'Saint Lawrence River'
 'Saint Regis River' 'Saint Regis River; West Branch of the Saint Regis R'
 'Santee River' 'Seneca River' 'Shenandoah River'
 'Shenandoah River; South Fork Shenandoah River' 'Sleepy Creek'
 'South Fork Shenandoah River' 'South Nation River' 'South Santee River'
 'St. Johns River' 'Susquehanna River' 'Swatara Creek' 'Tonawanda Creek'
 'Trent River' 'Upper Potomac River' 'Waccamaw River' 'Wando River'
 'Warwick River' 'West Branch Cooper River'
 'West Branch Oswegatchie River' 'West Branch Susquehanna River'
 'West Branch of the Saint Regis R' 'Western Branch Elizabeth River'
 'no_data']

Filter dataframe by river name of interest and plot selected reaches:

Note: Some rivers have multiple names, hence using the contains function

# Enter river name
river = "Susquehanna River"  # e.g. "Rhine", "Connecticut River"

## Filter dataframe
SWOT_HR_df_river = SWOT_HR_df[(SWOT_HR_df.river_name.str.contains(river))]

# Plot geopandas dataframe with 'explore' by reach id
SWOT_HR_df_river[['reach_id','river_name','geometry']].explore('reach_id', style_kwds=dict(weight=6))

Make this Notebook Trusted to load map: File -> Trust Notebook

2. River longitudinal profile: trace reaches downstream of given starting reach using rch_id_dn field

WARNING: This works as long as the data is exhaustive (no missing SWORD reaches)

First, let’s set up a dictionary relating all reaches in the dataset to their downstream neighbor

Note: rch_dn_dict[rch_id] gives a list of all the reaches directly downstream from rch_id

# Format rch_id_dn for dictionary. Rch_id_dn allows for multiple downstream reaches to be stored
# Also removes spaces in attribute field
rch_id_dn = [[x.strip() for x in SWOT_HR_df.rch_id_dn[j].split(',')] for j in range(0,len(SWOT_HR_df.rch_id_dn))]

# Filter upstream reach ids to remove 'no_data'
rch_id_dn_filter = [[x for x in dn_id if x.isnumeric()] for dn_id in rch_id_dn]

# Create lookup dictionary for river network topology: Downstream
rch_dn_dict = {SWOT_HR_df.reach_id[i]: rch_id_dn_filter[i] for i in range(len(SWOT_HR_df))}

Then, starting from a given reach, let’s trace all connected downstream reaches

Find the Reach IDs on www.swordexplorer.com

# Enter reach_id from which we will trace downstream    (e.g. headwaters of the Connecticut River)
rch_dn_st = {"Connecticut River": '73120000691',
             "Rhine": '23267000651',
             "Susquehanna River": '73160300071'
            }

# Initialize list to store downstream reaches, including starting reach
rch_dn_list = [rch_dn_st[river]]
# Retrieve first downstream id of starting reach and add to list
rch_dn_next = rch_dn_dict[rch_dn_st[river]][0]

# Trace next downstream reach until we hit the outlet (or here the last reach on file)
while len(rch_dn_next) != 0:
    # Add reach to list if value exists
    if len(rch_dn_next) != 0:
        rch_dn_list.append(rch_dn_next)
    # Recursively retrieve first downstream id of next reach
    # Catch error if reach isn't in downloaded data
    try:
        rch_dn_next = rch_dn_dict[rch_dn_next][0]
    except:
        break

Finally, we filtered our downloaded data by the traced reaches to create a plot

# Filter downloaded data by downstream traced reaches
SWOT_dn_trace = SWOT_HR_df[SWOT_HR_df.reach_id.isin(rch_dn_list)]

# Remove reaches from rch_dn_list that are not present in SWOT data
rch_dn_list = [rch for rch in rch_dn_list if rch in SWOT_HR_df.reach_id.values]

SWOT_dn_trace[['reach_id','river_name','geometry']].explore('river_name', style_kwds=dict(weight=6))

Make this Notebook Trusted to load map: File -> Trust Notebook

Note: If not all of your data appears, there might be a SWORD database error

Let’s check by singling out the reach it left off on.

For example, in the Connecticut River, it looks like reach ‘73120000531’ at the time of this run (4/1/24) has ‘no_data’ in the rch_id_up and rch_id_dn fields when there should be values. To help out the SWORD database creators, we can report this discrepancy by clicking the Report Reach button at https://www.swordexplorer.com/. SWORD should be updated quarterly.

SWOT_HR_df[SWOT_HR_df.reach_id == '73120000531']

3. Watershed analysis: trace reaches upstream of starting reach using rch_id_up field

WARNING: This works as long as the data is exhaustive (no missing SWORD reaches)

First, let’s set up a dictionary relating all reaches in the dataset to their upstream neighbor

Note: rch_up_dict[rch_id] gives a list of all the reaches directly upstream from rch_id

# Format rch_id_up for dictionary. Rch_id_up allows for multiple upstream reaches to be stored
# Also removes spaces in attribute field
rch_id_up = [[x.strip() for x in SWOT_HR_df.rch_id_up[j].split(',')] for j in range(0,len(SWOT_HR_df.rch_id_up))]

# Filter upstream reach ids to remove 'no_data'
rch_id_up_fil = [[x for x in ups_id if x.isnumeric()] for ups_id in rch_id_up]

# Create lookup dictionary for river network topology: Upstream
rch_up_dict = {SWOT_HR_df.reach_id[i]: rch_id_up_fil[i] for i in range(len(SWOT_HR_df))}

Then, starting from a given reach, let’s trace all connected upstream reaches

This adds a bit of complexity, as we need to keep track of multiple branches upstream of the starting reach. Find outlet Reach IDs on www.swordexplorer.com

# Enter reach_id from which we will trace upstream    (e.g. outlet of the River)
rch_up_st = {"Connecticut River": '73120000021',
             "Rhine": '23265000051',
             "Susquehanna River": '73160100011'
            }

# Initialize list to store traced upstream reaches, including starting reach
rch_up_list = [rch_up_st[river]]
# Retrieve ids of reaches upstream of starting reach and add to list
rch_up_next = rch_up_dict[rch_up_st[river]]
# For upstream tracing, we need to set a list of next upstream ids to start while loop
rch_next_id = rch_up_next

# Loop until no more reaches to trace
while len(rch_next_id) != 0:
    # Initialize list to store next upstream ids
    rch_next_id = []
    # Loop through next upstream ids for given reach
    for rch_up_sel in rch_up_next:
        # Get values of existing upstream ids of rch_up_next reaches
        # If reach isn't in SWOT data (usually ghost reaches), continue to next reach
        try:
            rch_next_id = rch_next_id + rch_up_dict[rch_up_sel]
        except:
            continue
        # Append id to list
        rch_up_list.append(rch_up_sel)
    # If reaches exist, add to list for next cycle of tracing
    if len(rch_next_id) != 0:
        rch_up_next = rch_next_id

Finally, we filtered our downloaded data by the traced reaches to create a plot

# Filter downloaded data by upstream traced reaches
SWOT_up_trace = SWOT_HR_df[SWOT_HR_df.reach_id.isin(rch_up_list)]

# Remove reaches from rch_up_list that are not present in SWOT data
rch_up_list = [rch for rch in rch_up_list if rch in SWOT_HR_df.reach_id.values]

SWOT_up_trace[['reach_id','river_name','geometry']].explore('river_name', style_kwds=dict(weight=6))

Make this Notebook Trusted to load map: File -> Trust Notebook

Note: If not all of your data appears, there might be a SWORD database error

Let’s check by singling out the reach it left off on.

For example, in the Connecticut River, it looks like reach ‘73120000131’ at the time of this run (4/1/24) has ‘no_data’ in the rch_id_up field when there should be values. To help out the SWORD database creators, we can report this discrepancy by clicking the Report Reach button at https://www.swordexplorer.com/. SWORD should be updated quarterly.

SWOT_HR_df[SWOT_HR_df.reach_id == '73120000131']

4. Visualize and plot a time series of WSE/width/slope longitudinal profiles

Let’s create a time series dataframe from the downstream filtered database

# Retrieve all possible acquisition dates (keeping only YYYY-MM-DD)
dates = np.unique([i[:10] for i in [x for x in SWOT_HR_df['time_str'] if x!='no_data']])

# Create a new database for time series analysis with unique reach_ids
SWOT_dn_trace_time = SWOT_dn_trace.set_index('reach_id').groupby(level=0) \
                                  .apply(lambda df: df.reset_index(drop=True)) \
                                  .unstack().sort_index(axis=1, level=1)

SWOT_dn_trace_time.columns = ['{}_{}'.format(x[0],dates[x[1]]) for x in SWOT_dn_trace_time.columns]

Plot a longitudinal profile for selected SWOT variable

# Explore variables you could choose to plot
for var in ["wse","slope","width","len"]:
    print(SWOT_dn_trace.columns[SWOT_dn_trace.columns.str.contains(var)])

Index(['wse', 'wse_u', 'wse_r_u', 'wse_c', 'wse_c_u', 'area_wse', 'p_wse',
       'p_wse_var'],
      dtype='object')
Index(['slope', 'slope_u', 'slope_r_u', 'slope2', 'slope2_u', 'slope2_r_u'], dtype='object')
Index(['width', 'width_u', 'width_c', 'width_c_u', 'p_width'], dtype='object')
Index(['p_length'], dtype='object')

# Enter variable of interest for plotting
varstr = "wse"

The error that outputs the following cell is because of the SWORD database discrepancy noted above. When this is corrected, only the graph should appear without the error.

# Find cumulative length on the longitudinal profile
length_list    = np.nan_to_num([SWOT_dn_trace.p_length[SWOT_dn_trace.reach_id == rch].mean()/1000 for rch in rch_dn_list])
cumlength_list = np.cumsum(length_list)

## Plot a longitudinal profile from the downstream tracing database

## Plot a longitudinal profile from the downstream tracing database
plt.figure(figsize=(12,8))
for t in dates:
    
    # Store the quantity of interest (wse, width etc.) at time t
    value = SWOT_dn_trace_time.loc[rch_dn_list,varstr+'_'+t]
    
    # Remove set negative values (bad observations) to NaN and forward fill NaNs
    value[value < 0] = np.nan
    value = value.ffill()
    
    # Plot the data
    plt.plot(cumlength_list, value, label = varstr+'_'+t)
    
plt.xlabel('Downstream Distance (km)')
plt.ylabel(varstr)
plt.legend()

Map the longitudinal profile of selected SWOT variable

# Choose a date
date = dates[1]

#Set one column as the active geometry in the new database
SWOT_dn_trace_time = SWOT_dn_trace_time.set_geometry("geometry_"+date)

#Set cleaner colorbar bounds for better visualization
vmin = np.percentile([i for i in SWOT_dn_trace_time[varstr+'_'+date] if i>0],5)
vmax = np.percentile([i for i in SWOT_dn_trace_time[varstr+'_'+date] if i>0],95)

# Interactive map
SWOT_dn_trace_time.explore(varstr+'_'+date,
                           vmin = vmin,
                           vmax = vmax,
                           cmap = "Blues", #"Blues",
                           control_scale = True,
                           tooltip = varstr+'_'+date,  # show "varstr+'_'+dates[0]" value in tooltip (on hover)
                           popup = True,  # show all values in popup (on click)
                           #tiles = "CartoDB positron",  # use "CartoDB positron" tiles
                           style_kwds=dict(weight=10)
                          )

Make this Notebook Trusted to load map: File -> Trust Notebook

Supplemental

We can also map the upstream traced database

SWOT_up_trace_time = SWOT_up_trace.set_index('reach_id').groupby(level=0) \
                                  .apply(lambda df: df.reset_index(drop=True)) \
                                  .unstack().sort_index(axis=1, level=1)

SWOT_up_trace_time.columns = ['{}_{}'.format(x[0],dates[x[1]]) for x in SWOT_up_trace_time.columns]

SWOT_up_trace_time = SWOT_up_trace_time.set_geometry("geometry_"+date)

#Set cleaner colorbar bounds for better visualization
vmin = np.percentile([i for i in SWOT_up_trace_time[varstr+'_'+date] if i>0],5)
vmax = np.percentile([i for i in SWOT_up_trace_time[varstr+'_'+date] if i>0],95)

# Interactive map
SWOT_up_trace_time.explore(varstr+'_'+date,
                           vmin = vmin,
                           vmax = vmax,
                           cmap = "Blues", #"Blues",
                           control_scale = True,
                           tooltip = varstr+'_'+date,  # show "varstr+'_'+dates[0]" value in tooltip (on hover)
                           popup = True,  # show all values in popup (on click)
                           tiles = "CartoDB positron",  # use "CartoDB positron" tiles
                           style_kwds=dict(weight=5)
                          )

Make this Notebook Trusted to load map: File -> Trust Notebook