import datetime
import pathlib
import earthaccess
import matplotlib.pyplot as plt
import netCDF4 as nc
import numpy as np
import pandas as pd
From the PO.DAAC Cookbook, to access the GitHub version of the notebook, follow this link.
Exploring gauges and river discharge in the SWORD of Science (SoS) dataset
Mapping all discharge algorithms, integrator results, and gauge data
Summary
Looking at discharge in the SoS
It can be helpful to plot the flow law parameter estimation (FLPE) algorithm discharge alongside the integrator (MOI) discharge produced for that algorithm PLUS overlapping in situ gauge data. Note that not all rivers have gauge data associated with them. In this notebook we will look at the steps to plot SoS discharge values produced from running the Confluence workflow alongside in situ gauge data gathered and stored in the priors.
Granule structure (background)
The SWORD of Science (SoS) is a community-driven dataset produced for and from the execution of the Confluence workflow in the cloud which enables quick data access and compute on SWOT data. Data granules contain two files, priors and results. The priors file contains prior information, such as in-situ gage data and model output that is used to generate the discharge products. The results file contains the resulting river discharge data products.
The cloud-based workflow (“Confluence”) that produces the SoS will produce discharge parameter estimates which the SWOT mission will use to produce discharge. This discharge will be stored in the SWOT shapefiles as the official SWOT discharge. However, the Confluence workflow produces discharge time series alongside the discharge parameter estimates in order to preview what will eventually stored in the SWOT shapefiles. Users can reference the SoS for the latest discharge time series recognizing that the official SWOT discharge data product lives in the SWOT shapefiles.
The SoS is organized by continent following SWOT River Database (SWORD) structure and naming conventions. It is indexed on the same reach and node identifier dimensions found in SWORD. Time series data is stored by cycle and pass on an observation dimension.
More information is available in the SWOT-Confluence Github repository: * Documentation for priors * Documentation for results
Results are organized into groups corresponding to modules in the SWOT-Confluence processing software. Modules are described in the Confluence Module Documentation.
You can explore the SoS further in this notebook: https://podaac.github.io/tutorials/notebooks/datasets/SWOT_L4_DAWG_SOS_DISCHARGE.html
Locate data for a river that has gauges
We will select the Ohio River as we know it has gauge data associated with it from the USGS but feel free to modify the constants below for your river of choice!
Table of Gauge Agencies by Continent
The following list the continent with associated gauge agency and group name of the gauge agency as it is stored in the SoS.
Continent | Group Name | Gauge Agency |
---|---|---|
Africa | GRDC | Global Runoff Data Centre |
Asia | GRDC | Global Runoff Data Centre |
Asia | MLIT | Ministry of Land, Infrastructure, Transport, Tourism |
Europe | GRDC | Global Runoff Data Centre |
Europe | DEFRA | Department of Environment Food & Rural Affairs |
Europe | EAU | Hub’Eau France |
North America | GRDC | Global Runoff Data Centre |
North America | USGS | United State Geological Survey |
North America | WSC | Water Survey Canada |
Oceania | GRDC | Global Runoff Data Centre |
Oceania | ABOM | Australian Government Bureau of Meteorology |
South America | GRDC | Global Runoff Data Centre |
South America | DGA | Direccion General de Aguas |
South America | Hidroweb | Hidroweb |
Requirements
1. Compute environment
This tutorial can be run in the following environments: - Local compute environment e.g. laptop, server: this tutorial can be run on your local machine
2. Earthdata Login
An Earthdata Login account is required to access data, as well as discover restricted data, from the NASA Earthdata system. Thus, to access NASA data, you need Earthdata Login. Please visit https://urs.earthdata.nasa.gov to register and manage your Earthdata Login account. This account is free to create and only takes a moment to set up.
Learning Objectives
- To locate in situ gauge data stored in the SoS.
- To locate overlap between in situ observations and times where discharge values were produced.
- Plot gauge data alongside river discharge.
Import Packages
Authenticate
Authenticate your Earthdata Login (EDL) information using the earthaccess
python package as follows:
# Login with your EDL credentials if asked earthaccess.login()
<earthaccess.auth.Auth at 0x107fc5df0>
Search and Access SoS data
Locate the SoS data of interest and then download for access.
# Search and locate granules
= earthaccess.search_data(
granule_info ="SWOT_L4_DAWG_SOS_DISCHARGE",
short_name=("2023-04-07", "2023-04-26"),
temporal
) granule_info
Granules found: 3
[Collection: {'Version': '1', 'ShortName': 'SWOT_L4_DAWG_SOS_DISCHARGE'}
Spatial coverage: {'HorizontalSpatialDomain': {'Geometry': {'BoundingRectangles': [{'WestBoundingCoordinate': -21.794, 'SouthBoundingCoordinate': 25.382, 'EastBoundingCoordinate': 25.382, 'NorthBoundingCoordinate': 81.115}]}}}
Temporal coverage: {'RangeDateTime': {'EndingDateTime': '2023-04-25T20:01:59.000Z', 'BeginningDateTime': '2023-04-07T22:49:35.000Z'}}
Size(MB): 983.0999364852905
Data: ['https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/SWOT_L4_DAWG_SOS_DISCHARGE/eu_sword_v15_SOS_unconstrained_0001_20240228T205029_results.nc', 'https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/SWOT_L4_DAWG_SOS_DISCHARGE/eu_sword_v15_SOS_unconstrained_0001_20240228T205029_priors.nc'],
Collection: {'Version': '1', 'ShortName': 'SWOT_L4_DAWG_SOS_DISCHARGE'}
Spatial coverage: {'HorizontalSpatialDomain': {'Geometry': {'BoundingRectangles': [{'WestBoundingCoordinate': -81.139, 'SouthBoundingCoordinate': -52, 'EastBoundingCoordinate': -52, 'NorthBoundingCoordinate': 11.097}]}}}
Temporal coverage: {'RangeDateTime': {'EndingDateTime': '2023-04-26T12:04:55.000Z', 'BeginningDateTime': '2023-04-08T01:51:07.000Z'}}
Size(MB): 1700.4334163665771
Data: ['https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/SWOT_L4_DAWG_SOS_DISCHARGE/sa_sword_v15_SOS_unconstrained_0001_20240228T205034_results.nc', 'https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/SWOT_L4_DAWG_SOS_DISCHARGE/sa_sword_v15_SOS_unconstrained_0001_20240228T205034_priors.nc'],
Collection: {'Version': '1', 'ShortName': 'SWOT_L4_DAWG_SOS_DISCHARGE'}
Spatial coverage: {'HorizontalSpatialDomain': {'Geometry': {'BoundingRectangles': [{'WestBoundingCoordinate': -166.397, 'SouthBoundingCoordinate': 8.09, 'EastBoundingCoordinate': 8.09, 'NorthBoundingCoordinate': 82.311}]}}}
Temporal coverage: {'RangeDateTime': {'EndingDateTime': '2023-04-26T13:28:35.000Z', 'BeginningDateTime': '2023-04-08T05:36:12.000Z'}}
Size(MB): 1613.2776679992676
Data: ['https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/SWOT_L4_DAWG_SOS_DISCHARGE/na_sword_v15_SOS_unconstrained_0001_20240228T205032_results.nc', 'https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/SWOT_L4_DAWG_SOS_DISCHARGE/na_sword_v15_SOS_unconstrained_0001_20240228T205032_priors.nc']]
# Enter a directory path to store downloaded data in
= pathlib.Path("data_downloads")
downloads_dir =True, exist_ok=True)
downloads_dir.mkdir(parents
# Select a priors and results pair to explore
= [[link for link in earthaccess.results.DataGranule.data_links(granule)] for granule in granule_info]
download_links print("Select a priors and results file to explore:")
for downloads in download_links:
for download in downloads:
if "priors" in download: print(download)
Select a priors and results file to explore:
https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/SWOT_L4_DAWG_SOS_DISCHARGE/eu_sword_v15_SOS_unconstrained_0001_20240228T205029_priors.nc
https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/SWOT_L4_DAWG_SOS_DISCHARGE/sa_sword_v15_SOS_unconstrained_0001_20240228T205034_priors.nc
https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/SWOT_L4_DAWG_SOS_DISCHARGE/na_sword_v15_SOS_unconstrained_0001_20240228T205032_priors.nc
# Select Europe ("eu") priors file to work with
= "https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-protected/SWOT_L4_DAWG_SOS_DISCHARGE/eu_sword_v15_SOS_unconstrained_0001_20240228T205029_priors.nc"
priors_link
# Select results
= priors_link.replace("priors", "results")
results_link
earthaccess.download(priors_link, downloads_dir) earthaccess.download(results_link, downloads_dir)
File eu_sword_v15_SOS_unconstrained_0001_20240228T205029_priors.nc already downloaded
File eu_sword_v15_SOS_unconstrained_0001_20240228T205029_results.nc already downloaded
['data_downloads/eu_sword_v15_SOS_unconstrained_0001_20240228T205029_results.nc']
# Open downloaded files to access SoS granule data
= priors_link.split('/')[-1]
priors_download = results_link.split('/')[-1]
results_download
= nc.Dataset(downloads_dir.joinpath(priors_download), format="NETCDF4")
priors = nc.Dataset(downloads_dir.joinpath(results_download), format="NETCDF4") results
Locate gauge and rive discharge data.
We can now locate gauge and river discharge data from the SoS using either the data read directly from S3 or downloaded to your local computer.
# Constants
# Select a river
= "Rhine"
RIVER_NAME
# Select a gauge agency
= "EAU" GAUGE_AGENCY
Locate overlapping identifiers
Locate overlapping identifiers for reach and gauge data.
# Locate overlapping reach identifier
= results['reaches']['river_name'][:]
river_names = np.where(river_names == RIVER_NAME)
river_indexes
= results["reaches"]["reach_id"][river_indexes]
river_reach print(f"Number of {RIVER_NAME} reach identifiers: {len(river_reach)}")
= priors[GAUGE_AGENCY][f"{GAUGE_AGENCY}_reach_id"][:]
gauge_reach print(f"Number of Gauge reach identifiers: {len(gauge_reach)}")
= np.intersect1d(gauge_reach, river_reach)
reach_overlap print("Overlapping reaches:")
print(reach_overlap)
# Select the first reach
= reach_overlap[0]
reach_id print(f"Reach id selected: {reach_id}")
Number of Rhine reach identifiers: 146
Number of Gauge reach identifiers: 243
Overlapping reaches:
[23267000071 23267000081 23267000121 23267000501]
Reach id selected: 23267000071
Locate gauge discharge and in situ observation time
Locate discharge and save the in situ observation time for the reach of interest.
# Get reach index for gauge data
= np.where(gauge_reach == reach_id)
reach_gauge_index
# Get discharge and filter out missing values
= priors[GAUGE_AGENCY][f"{GAUGE_AGENCY}_q"]._FillValue
missing = priors[GAUGE_AGENCY][f"{GAUGE_AGENCY}_q"][reach_gauge_index].filled()[0]
gauge_discharge = np.where(gauge_discharge != missing)
nonmissing_indexes_g = gauge_discharge[nonmissing_indexes_g]
gauge_discharge print(f"Number of gauge discharge values: {len(gauge_discharge)}.")
# Get time and filter out missing values
= priors[GAUGE_AGENCY][f"{GAUGE_AGENCY}_qt"][reach_gauge_index].filled().astype(int)[0]
gauge_time = gauge_time[nonmissing_indexes_g]
gauge_time print(f"Number of gauge time values: {len(gauge_time)}.")
# Convert time from ordinal value
= [ datetime.datetime.fromordinal(gt).strftime("%Y%m%d") for gt in gauge_time ] gauge_time
Number of gauge discharge values: 3722.
Number of gauge time values: 3722.
Locate SWOT time
Locate the observations times for SWOT used in the discharge parameter estimation processing.
# Locate the reach identifier
= np.where(results['reaches']['reach_id'][:] == reach_id)
reach_q_index print("Reach Index: ", reach_q_index)
# Retrieve SWOT observation times and filter out missing values
= results['reaches']['time'][reach_q_index][0]
time
# Locate indexes where data exists
= results['reaches']['time'].missing_value
missing = np.where(time != missing)
nonmissing_indexes = time[nonmissing_indexes]
time
# Convert to discharge time to same format as gauge agency time
= datetime.datetime(2000,1,1,0,0,0) # SWOT timestamp delta
swot_ts = [ (swot_ts + datetime.timedelta(seconds=st)).strftime("%Y%m%d") for st in time ]
discharge_time print(f"Number of SWOT time values: {len(time)}.")
print(discharge_time)
Reach Index: (array([13158]),)
Number of SWOT time values: 13.
['20230407', '20230408', '20230409', '20230410', '20230416', '20230417', '20230418', '20230419', '20230420', '20230422', '20230423', '20230424', '20230425']
Locate algorithm discharge
Locate the algorithm discharge for a corresponding reach identifier that has gauge data. We will use HiVDI for this demonstration.
def get_algorithm_discharge(results, algorithm, variable, reach_index, nonmissing_indexes, time_shape):
"""Locate and return discharge for a specific algorithm from the results SoS file."""
if algorithm == "neobam":
= results[algorithm][variable]["q1"][reach_q_index][0]
discharge1 = results[algorithm][variable]["q2"][reach_q_index][0]
discharge2 = results[algorithm][variable]["q2"][reach_q_index][0]
discharge3 = np.mean([discharge1, discharge2, discharge3])
discharge_q
else:
= results[algorithm][variable][reach_index][0]
discharge_q
if isinstance(discharge_q, np.ndarray) and discharge_q.shape[0] > 1:
= discharge_q[nonmissing_indexes] # Filter for where data exists in time
discharge_q == missing)] = np.nan # Set missing to NaN value
discharge_q[np.where(discharge_q # No data was generated
else:
= np.empty((time_shape))
discharge_q
discharge_q.fill(np.nan)
return discharge_q
# HiVDI
= get_algorithm_discharge(results, "hivdi", "Q", reach_q_index, nonmissing_indexes, time.shape)
hivdi_q print(f"Number of HiVDI discharge values: {hivdi_q.shape[0]}")
# MetroMan
= get_algorithm_discharge(results, "metroman", "allq", reach_q_index, nonmissing_indexes, time.shape)
metroman_q print(f"Number of MetroMan discharge values: {hivdi_q.shape[0]}")
# MOMMA
= get_algorithm_discharge(results, "momma", "Q", reach_q_index, nonmissing_indexes, time.shape)
momma_q print(f"Number of MOMMA discharge values: {hivdi_q.shape[0]}")
# NeoBAM
= get_algorithm_discharge(results, "neobam", "q", reach_q_index, nonmissing_indexes, time.shape)
neobam_q print(f"Number of NeoBAM discharge values: {hivdi_q.shape[0]}")
# SAD
= get_algorithm_discharge(results, "sad", "Qa", reach_q_index, nonmissing_indexes, time.shape)
sad_q print(f"Number of SAD discharge values: {hivdi_q.shape[0]}")
# SIC4DVar
= get_algorithm_discharge(results, "sic4dvar", "Q_da", reach_q_index, nonmissing_indexes, time.shape)
sic4dvar_q print(f"Number of SIC4DVar discharge values: {hivdi_q.shape[0]}")
Number of HiVDI discharge values: 13
Number of MetroMan discharge values: 13
Number of MOMMA discharge values: 13
Number of NeoBAM discharge values: 13
Number of SAD discharge values: 13
Number of SIC4DVar discharge values: 13
Locate integrator (MOI) discharge
Locate the integrator discharge produced for the algorithm for the reach of interest that has gauge data. As mentioned, we will use HiVDI for this demonstration.
def locate_moi_discharge(results, algorithm, reach_index, nonmissing_indexes):
"""Locate and return MOI results for specific algorithm."""
= results["moi"][algorithm]["q"][reach_index][0]
moi_q = moi_q[nonmissing_indexes]
moi_q
# Set missing MOI to NaN
= results["moi"][algorithm]["q"].missing_value
missing_moi == missing_moi] = np.nan
moi_q[moi_q
return moi_q
# Locate MOI discharge results for discharge algorithm making sure to filter out missing values
# HiVDI
= locate_moi_discharge(results, "hivdi", reach_q_index, nonmissing_indexes)
moi_hivdi print(f"Number of integrator HiVDI discharge values: {len(moi_hivdi)}.")
# MetroMan
= locate_moi_discharge(results, "metroman", reach_q_index, nonmissing_indexes)
moi_metro print(f"Number of integrator MetroMan discharge values: {len(moi_hivdi)}.")
# MOMMA
= locate_moi_discharge(results, "momma", reach_q_index, nonmissing_indexes)
moi_momma print(f"Number of integrator MOMMA discharge values: {len(moi_momma)}.")
# NeoBAM
= locate_moi_discharge(results, "geobam", reach_q_index, nonmissing_indexes)
moi_neo print(f"Number of integrator NeoBAM discharge values: {len(moi_neo)}.")
# SAD
= locate_moi_discharge(results, "sad", reach_q_index, nonmissing_indexes)
moi_sad print(f"Number of integrator SAD discharge values: {len(moi_sad)}.")
# SIC4DVar
= locate_moi_discharge(results, "sic4dvar", reach_q_index, nonmissing_indexes)
moi_sic print(f"Number of integrator Sic4DVar discharge values: {len(moi_sic)}.")
Number of integrator HiVDI discharge values: 13.
Number of integrator MetroMan discharge values: 13.
Number of integrator MOMMA discharge values: 13.
Number of integrator NeoBAM discharge values: 13.
Number of integrator SAD discharge values: 13.
Number of integrator Sic4DVar discharge values: 13.
Locate overlapping observations
We will need to locate the discharge time series (FLPE and MOI) for the reach of interest and then determine if there are overlapping in situ observations with SWOT observations.
# Find overlapping time between in situ and SWOT observations
= list(set(discharge_time).intersection(set(gauge_time)))
obs_overlap
obs_overlap.sort()print("Days of observation overlap: ", obs_overlap)
print()
# Get indexes of overlap for gauge, algorithm and integrator
= np.where(np.in1d(gauge_time, obs_overlap))[0]
gauge_overlap_index = gauge_overlap_index[6:]
gauge_overlap_index = np.where(np.in1d(discharge_time, obs_overlap))[0]
discharge_overlap_index = discharge_overlap_index[6:] discharge_overlap_index
Days of observation overlap: ['20230407', '20230408', '20230409', '20230410', '20230416', '20230417', '20230418', '20230419', '20230420', '20230422', '20230423', '20230424', '20230425']
# Retrieve time and discharge values for indexes
= np.array(gauge_time)[gauge_overlap_index]
gauge_time print("Gauge time: ", gauge_time)
= np.array(gauge_discharge)[gauge_overlap_index]
gauge_discharge print(f"Number of Gauge discharge values: {gauge_discharge.shape[0]}")
Gauge time: ['20230418' '20230419' '20230420' '20230422' '20230423' '20230424'
'20230425']
Number of Gauge discharge values: 7
# Retrieve swot time values
= np.array(discharge_time)[discharge_overlap_index]
discharge_time print(f"SWOT time:\n", discharge_time)
SWOT time:
['20230418' '20230419' '20230420' '20230422' '20230423' '20230424'
'20230425']
# Retrieve discharge algorithm values
# HiVDI
= hivdi_q[discharge_overlap_index]
hivdi_q print(f"Number of HiVDI discharge values: {hivdi_q.shape[0]}")
# MetroMan
= metroman_q[discharge_overlap_index]
metroman_q print(f"Number of MetroMan discharge values: {hivdi_q.shape[0]}")
# MOMMA
= momma_q[discharge_overlap_index]
momma_q print(f"Number of MOMMA discharge values: {hivdi_q.shape[0]}")
# NeoBAM
= neobam_q[discharge_overlap_index]
neobam_q print(f"Number of NeoBAM discharge values: {hivdi_q.shape[0]}")
# SAD
= sad_q[discharge_overlap_index]
sad_q print(f"Number of SAD discharge values: {hivdi_q.shape[0]}")
# SIC4DVar
= sic4dvar_q[discharge_overlap_index]
sic4dvar_q print(f"Number of SIC4DVar discharge values: {hivdi_q.shape[0]}")
Number of HiVDI discharge values: 7
Number of MetroMan discharge values: 7
Number of MOMMA discharge values: 7
Number of NeoBAM discharge values: 7
Number of SAD discharge values: 7
Number of SIC4DVar discharge values: 7
# Retrieve MOI values
# HiVDI
= moi_hivdi[discharge_overlap_index]
moi_hivdi print(f"Number of integrator HiVDI discharge values: {len(moi_hivdi)}.")
# MetroMan
= moi_metro[discharge_overlap_index]
moi_metro print(f"Number of integrator MetroMan discharge values: {len(moi_hivdi)}.")
# MOMMA
= moi_momma[discharge_overlap_index]
moi_momma print(f"Number of integrator MOMMA discharge values: {len(moi_momma)}.")
# NeoBAM
= moi_neo[discharge_overlap_index]
moi_neo print(f"Number of integrator NeoBAM discharge values: {len(moi_neo)}.")
# SAD
= moi_sad[discharge_overlap_index]
moi_sad print(f"Number of integrator SAD discharge values: {len(moi_sad)}.")
# SIC4DVar
= moi_sic[discharge_overlap_index]
moi_sic print(f"Number of integrator Sic4DVar discharge values: {len(moi_sic)}.")
Number of integrator HiVDI discharge values: 7.
Number of integrator MetroMan discharge values: 7.
Number of integrator MOMMA discharge values: 7.
Number of integrator NeoBAM discharge values: 7.
Number of integrator SAD discharge values: 7.
Number of integrator Sic4DVar discharge values: 7.
Plotting results for comparison
Let’s plot all discharge time series to better visualize the differences and compare the FLPE, MOI, and gauge discharge values.
def plot_discharge(axs, discharge_time, discharge_q, color, name, line_list, line_list_names):
"""Plot discharge values and return list of plot lines."""
if np.all(np.isnan(discharge_q)):
return # Cannot plot NaN values
else:
=color)
axs.scatter(discharge_time, discharge_q, colorif "MOI" in name:
= axs.plot(discharge_time, discharge_q, color=color, linestyle="dashed")
line, else:
= axs.plot(discharge_time, discharge_q, color=color)
line,
line_list.append(line) line_list_names.append(name)
= plt.subplots(figsize = (25,6))
fig, axs
## Build legend data
= []
line_list = []
line_list_names
## Gauge
"grey", "Gauge", line_list, line_list_names)
plot_discharge(axs, discharge_time, gauge_discharge,
## FLPE & MOI
"blue", "HiVDI", line_list, line_list_names)
plot_discharge(axs, discharge_time, hivdi_q, "blue", "MOI HiVDI", line_list, line_list_names)
plot_discharge(axs, discharge_time, moi_hivdi,
"orange", "MetroMan", line_list, line_list_names)
plot_discharge(axs, discharge_time, metroman_q, "orange", "MOI MetroMan", line_list, line_list_names)
plot_discharge(axs, discharge_time, moi_metro,
"red", "MOMMA", line_list, line_list_names)
plot_discharge(axs, discharge_time, momma_q, "red", "MOI MOMMA", line_list, line_list_names)
plot_discharge(axs, discharge_time, moi_momma,
"purple", "NeoBAM", line_list, line_list_names)
plot_discharge(axs, discharge_time, neobam_q, "purple", "MOI NeoBAM", line_list, line_list_names)
plot_discharge(axs, discharge_time, moi_neo,
"goldenrod", "SAD", line_list, line_list_names)
plot_discharge(axs, discharge_time, sad_q, "goldenrod", "MOI SAD", line_list, line_list_names)
plot_discharge(axs, discharge_time, moi_sad,
"seagreen", "SIC4DVar", line_list, line_list_names)
plot_discharge(axs, discharge_time, sic4dvar_q, "seagreen", "MOI SIC4DVar", line_list, line_list_names)
plot_discharge(axs, discharge_time, moi_sic,
# Plot details
='center right')
axs.legend(line_list, line_list_names, loc'Time', fontsize=10)
axs.set_ylabel('Discharge', fontsize=10)
axs.set_xlabel('Discharge', fontsize=10)
axs.set_title(
plt
Disclaimer: Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not constitute or imply its endorsement by the United States Government or the Jet Propulsion Laboratory, California Institute of Technology.