From the PO.DAAC Cookbook, to access the GitHub version of the notebook, follow this link.

The spatial Correlation between sea surface temperature anomaly and sea surface height anomaly in the Indian Ocean – A demo using ECCO

Author: Edward Armstrong and Jinbo Wang

Date: 2022-02-15

Objective

This tutorial will use data from the Estimating the Climate and Circulation of the Ocean (ECCO) model to derive spatial correlations through time for two regions of the Indian Ocean. The goal is to investigate the correlative characteristics of the Indian Ocean Dipole and how the east and west regions behave differently. This investigation was motivated by Fig 2 a,b in the paper by Wang et al. (2016).

IO_Dipole.png

• Wang, H., Murtugudde, R. & Kumar, A. Evolution of Indian Ocean dipole and its forcing mechanisms in the absence of ENSO. Clim Dyn 47, 2481–2500 (2016). https://doi.org/10.1007/s00382-016-2977-y

Input datasets

1. ECCO Ocean Temperature and Salinity - Monthly Mean 0.5 Degree (Version 4 Release 4). DOI: https://doi.org/10.5067/ECG5M-OTS44
2. ECCO Sea Surface Height - Daily Mean 0.5 Degree (Version 4 Release 4b). DOI: https://doi.org/10.5067/ECG5D-SSH4B

Services and software used

1. NASA Harmony netcdf-to-Zarr
2. xarray, requests, json, pandas, numpy, matplotlib, s3fs
3. Various python utlities including the NASA harmony-py package

# Import various libraries

from matplotlib import pylab as plt
import xarray as xr
import numpy as np
import pandas as pd
import requests
import json
import time
import s3fs
import re

from datetime import datetime
from harmony import Client, Collection, Environment, Request

Identify the ShortName of the dataset of interest.

The shortname is an unique pointer for each NASA dataset.

ShortName = "ECCO_L4_SSH_05DEG_MONTHLY_V4R4B"

Set start and end dates

start_date = "1992-01-01"
end_date   = "2002-12-31"

# break it down into Year, Month, Day (and minutes and seconds if desired) 
# as inputs to harmony.py call using datetime()
start_year = 2002
start_month = 1
start_day = 1

end_year = 2017
end_month = 12
end_day = 31

Spatial bounds (Region of Interest) – Not used

westernmost_longitude = 100.
easternmost_longitude = 150.
northermost_latitude  = 30.
southernmost_latitude = 0.

Find the concept_id

response = requests.get(
    url='https://cmr.earthdata.nasa.gov/search/collections.umm_json', 
    params={'provider': "POCLOUD",
            'ShortName': ShortName,
            'page_size': 1}
)

ummc = response.json()['items'][0]

ccid = ummc['meta']['concept-id']

ccid

'C2129189405-POCLOUD'

NetCDF to Zarr transformation

Setup the harmony-py service call and execute a request

# using the the harmony.py service, set up the request and exectue it
ecco_collection = Collection(id=ccid)
time_range = {'start': datetime(start_year, start_month, start_day), 'stop': datetime(end_year, end_month, end_day)}
print(time_range)

harmony_client = Client(env=Environment.PROD)

# in this example set concatentae to 'False' because the monthly input time steps vary slightly
# (not always centered in the middle of month)
ecco_request = Request(collection=ecco_collection, temporal=time_range, format='application/x-zarr', concatenate='False')

# sumbit request and monitor job
ecco_job_id = harmony_client.submit(ecco_request)
print('\n Waiting for the job to finish. . .\n')
ecco_response = harmony_client.result_json(ecco_job_id, show_progress=True)
print("\n. . .DONE!")

{'start': datetime.datetime(2002, 1, 1, 0, 0), 'stop': datetime.datetime(2017, 12, 31, 0, 0)}

Waiting for the job to finish. . .


. . .DONE!

 [ Processing: 100% ] |###################################################| [|]

You can also wrap the creation of the Harmony request URL into one function. Shown here for legacy purposes (does not execute a Harmony request):

def get_harmony_url(ccid,start_date,end_date):
    """
    Parameters:
    ===========
    ccid: string
        concept_id of the datset
    date_range: list
        [start_data, end_date] 
        
    Return:
    =======
    url: the harmony URL used to perform the netcdf to zarr transformation
    """
    
    base = f"https://harmony.earthdata.nasa.gov/{ccid}"
    hreq = f"{base}/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset"
    rurl = f"{hreq}?format=application/x-zarr"

    #print(rurl)

    subs = '&'.join([f'subset=time("{start_date}T00:00:00.000Z":"{end_date}T23:59:59.999Z")']) 
    #subs = subs + '&' + '&'.join([f'subset=lat({southernmost_latitude}:{northermost_latitude})'])
    #subs = subs + '&' + '&'.join([f'subset=lon({westernmost_longitude}:{easternmost_longitude})'])

    rurl = f"{rurl}&{subs}"
    return rurl

ccid='C2129189405-POCLOUD'
print(get_harmony_url(ccid,start_date,end_date))

# this is the way you would execute it
# response = requests.get(url=rurl).json()

https://harmony.earthdata.nasa.gov/C2129189405-POCLOUD/ogc-api-coverages/1.0.0/collections/all/coverage/rangeset?format=application/x-zarr&subset=time("1992-01-01T00:00:00.000Z":"2002-12-31T23:59:59.999Z")

Access the staged cloud datasets over native AWS interfaces

print(ecco_response['message'])
with requests.get(ecco_response['links'][2]['href']) as r:
    creds = r.json()

print( creds.keys() )
print("AWS credentials expire on: ", creds['Expiration'] )

The job has completed successfully. Contains results in AWS S3. Access from AWS us-west-2 with keys from https://harmony.earthdata.nasa.gov/cloud-access.sh
dict_keys(['AccessKeyId', 'SecretAccessKey', 'SessionToken', 'Expiration'])
AWS credentials expire on:  2022-05-03T08:45:27.000Z

List zarr datasets with s3fs

s3_dir = ecco_response['links'][3]['href']

print("root directory:", s3_dir)

s3_urls = [u['href'] for u in ecco_response['links'][4:-1]]

# sort the URLs in time order
s3_urls.sort()

print(s3_urls[0])

s3 = s3fs.S3FileSystem(
    key=creds['AccessKeyId'],
    secret=creds['SecretAccessKey'],
    token=creds['SessionToken'],
    client_kwargs={'region_name':'us-west-2'},
)

len(s3.ls(s3_dir))

root directory: s3://harmony-prod-staging/public/harmony/netcdf-to-zarr/0164d106-f8f0-431e-926c-c7d3f5f0575e/
s3://harmony-prod-staging/public/harmony/netcdf-to-zarr/0164d106-f8f0-431e-926c-c7d3f5f0575e/SEA_SURFACE_HEIGHT_mon_mean_2001-12_ECCO_V4r4b_latlon_0p50deg.zarr

193

Open datasets with xarray()

ds0 = xr.open_zarr(s3.get_mapper(s3_urls[0]), decode_cf=True, mask_and_scale=True)
ds0

<xarray.Dataset>
Dimensions:         (time: 1, latitude: 360, longitude: 720, nv: 2)
Coordinates:
  * latitude        (latitude) float32 -89.75 -89.25 -88.75 ... 89.25 89.75
    latitude_bnds   (latitude, nv) float32 dask.array<chunksize=(360, 2), meta=np.ndarray>
  * longitude       (longitude) float32 -179.8 -179.2 -178.8 ... 179.2 179.8
    longitude_bnds  (longitude, nv) float32 dask.array<chunksize=(720, 2), meta=np.ndarray>
  * time            (time) datetime64[ns] 2001-12-16T12:00:00
    time_bnds       (time, nv) datetime64[ns] dask.array<chunksize=(1, 2), meta=np.ndarray>
Dimensions without coordinates: nv
Data variables:
    SSH             (time, latitude, longitude) float32 dask.array<chunksize=(1, 360, 720), meta=np.ndarray>
    SSHIBC          (time, latitude, longitude) float32 dask.array<chunksize=(1, 360, 720), meta=np.ndarray>
    SSHNOIBC        (time, latitude, longitude) float32 dask.array<chunksize=(1, 360, 720), meta=np.ndarray>
Attributes: (12/57)
    Conventions:                  CF-1.8, ACDD-1.3
    acknowledgement:              This research was carried out by the Jet Pr...
    author:                       Ou Wang and Ian Fenty
    cdm_data_type:                Grid
    comment:                      Fields provided on a regular lat-lon grid. ...
    coordinates_comment:          Note: the global 'coordinates' attribute de...
    ...                           ...
    time_coverage_duration:       P1M
    time_coverage_end:            2002-01-01T00:00:00
    time_coverage_resolution:     P1M
    time_coverage_start:          2001-12-01T00:00:00
    title:                        ECCO Sea Surface Height - Monthly Mean 0.5 ...
    uuid:                         cd67c8ad-0f9e-4cf7-a3c3-4ac4d6016ea5

xarray.Dataset

• Dimensions:
  • time: 1
  • latitude: 360
  • longitude: 720
  • nv: 2
• Coordinates: (6)
  • latitude
    (latitude)
    float32
    -89.75 -89.25 ... 89.25 89.75

    axis :

    Y

    bounds :

    latitude_bnds

    comment :

    uniform grid spacing from -89.75 to 89.75 by 0.5

    coverage_content_type :

    coordinate

    long_name :

    latitude at grid cell center

    standard_name :

    latitude

    units :

    degrees_north

    array([-89.75, -89.25, -88.75, ...,  88.75,  89.25,  89.75], dtype=float32)

  • latitude_bnds
    (latitude, nv)
    float32
    dask.array<chunksize=(360, 2), meta=np.ndarray>

    coverage_content_type :

    coordinate

    long_name :

    latitude bounds grid cells

    Array Chunk Bytes 2.81 kiB 2.81 kiB Shape (360, 2) (360, 2) Count 2 Tasks 1 Chunks Type float32 numpy.ndarray

  • longitude
    (longitude)
    float32
    -179.8 -179.2 ... 179.2 179.8

    axis :

    X

    bounds :

    longitude_bnds

    comment :

    uniform grid spacing from -179.75 to 179.75 by 0.5

    coverage_content_type :

    coordinate

    long_name :

    longitude at grid cell center

    standard_name :

    longitude

    units :

    degrees_east

    array([-179.75, -179.25, -178.75, ...,  178.75,  179.25,  179.75],
          dtype=float32)

  • longitude_bnds
    (longitude, nv)
    float32
    dask.array<chunksize=(720, 2), meta=np.ndarray>

    coverage_content_type :

    coordinate

    long_name :

    longitude bounds grid cells

    Array Chunk Bytes 5.62 kiB 5.62 kiB Shape (720, 2) (720, 2) Count 2 Tasks 1 Chunks Type float32 numpy.ndarray

  • time
    (time)
    datetime64[ns]
    2001-12-16T12:00:00

    axis :

    T

    bounds :

    time_bnds

    coverage_content_type :

    coordinate

    long_name :

    center time of averaging period

    standard_name :

    time

    array(['2001-12-16T12:00:00.000000000'], dtype='datetime64[ns]')

  • time_bnds
    (time, nv)
    datetime64[ns]
    dask.array<chunksize=(1, 2), meta=np.ndarray>

    comment :

    Start and end times of averaging period.

    coverage_content_type :

    coordinate

    long_name :

    time bounds of averaging period

    Array Chunk Bytes 16 B 16 B Shape (1, 2) (1, 2) Count 2 Tasks 1 Chunks Type datetime64[ns] numpy.ndarray

• Data variables: (3)
  • SSH
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(1, 360, 720), meta=np.ndarray>

    comment :

    Dynamic sea surface height anomaly above the geoid, suitable for comparisons with altimetry sea surface height data products that apply the inverse barometer (IB) correction. Note: SSH is calculated by correcting model sea level anomaly ETAN for three effects: a) global mean steric sea level changes related to density changes in the Boussinesq volume-conserving model (Greatbatch correction, see sterGloH), b) the inverted barometer (IB) effect (see SSHIBC) and c) sea level displacement due to sea-ice and snow pressure loading (see sIceLoad). SSH can be compared with the similarly-named SSH variable in previous ECCO products that did not include atmospheric pressure loading (e.g., Version 4 Release 3). Use SSHNOIBC for comparisons with altimetry data products that do NOT apply the IB correction.

    coverage_content_type :

    modelResult

    long_name :

    Dynamic sea surface height anomaly

    standard_name :

    sea_surface_height_above_geoid

    units :

    m

    valid_max :

    1.4207719564437866

    valid_min :

    -1.8805772066116333

    Array Chunk Bytes 0.99 MiB 0.99 MiB Shape (1, 360, 720) (1, 360, 720) Count 2 Tasks 1 Chunks Type float32 numpy.ndarray

  • SSHIBC
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(1, 360, 720), meta=np.ndarray>

    comment :

    Not an SSH itself, but a correction to model sea level anomaly (ETAN) required to account for the static part of sea surface displacement by atmosphere pressure loading: SSH = SSHNOIBC - SSHIBC. Note: Use SSH for model-data comparisons with altimetry data products that DO apply the IB correction and SSHNOIBC for comparisons with altimetry data products that do NOT apply the IB correction.

    coverage_content_type :

    modelResult

    long_name :

    The inverted barometer (IB) correction to sea surface height due to atmospheric pressure loading

    units :

    m

    valid_max :

    0.5245633721351624

    valid_min :

    -0.3014216423034668

    Array Chunk Bytes 0.99 MiB 0.99 MiB Shape (1, 360, 720) (1, 360, 720) Count 2 Tasks 1 Chunks Type float32 numpy.ndarray

  • SSHNOIBC
    (time, latitude, longitude)
    float32
    dask.array<chunksize=(1, 360, 720), meta=np.ndarray>

    comment :

    Sea surface height anomaly above the geoid without the inverse barometer (IB) correction, suitable for comparisons with altimetry sea surface height data products that do NOT apply the inverse barometer (IB) correction. Note: SSHNOIBC is calculated by correcting model sea level anomaly ETAN for two effects: a) global mean steric sea level changes related to density changes in the Boussinesq volume-conserving model (Greatbatch correction, see sterGloH), b) sea level displacement due to sea-ice and snow pressure loading (see sIceLoad). In ECCO Version 4 Release 4b the model is forced with atmospheric pressure loading. SSHNOIBC does not correct for the static part of the effect of atmosphere pressure loading on sea surface height (the so-called inverse barometer (IB) correction). Use SSH for comparisons with altimetry data products that DO apply the IB correction.

    coverage_content_type :

    modelResult

    long_name :

    Sea surface height anomaly without the inverted barometer (IB) correction

    units :

    m

    valid_max :

    1.4550364017486572

    valid_min :

    -1.6654272079467773

    Array Chunk Bytes 0.99 MiB 0.99 MiB Shape (1, 360, 720) (1, 360, 720) Count 2 Tasks 1 Chunks Type float32 numpy.ndarray

Attributes: (57)

Conventions :

CF-1.8, ACDD-1.3

acknowledgement :

This research was carried out by the Jet Propulsion Laboratory, managed by the California Institute of Technology under a contract with the National Aeronautics and Space Administration.

author :

Ou Wang and Ian Fenty

cdm_data_type :

Grid

comment :

Fields provided on a regular lat-lon grid. They have been mapped to the regular lat-lon grid from the original ECCO lat-lon-cap 90 (llc90) native model grid. SSH (dynamic sea surface height) = SSHNOIBC (dynamic sea surface without the inverse barometer correction) - SSHIBC (inverse barometer correction). The inverted barometer correction accounts for variations in sea surface height due to atmospheric pressure variations.

coordinates_comment :

Note: the global 'coordinates' attribute describes auxillary coordinates.

creator_email :

ecco-group@mit.edu

creator_institution :

NASA Jet Propulsion Laboratory (JPL)

creator_name :

ECCO Consortium

creator_type :

group

creator_url :

https://ecco-group.org

date_created :

2021-09-26T15:28:22

date_issued :

2021-09-26T15:28:22

date_metadata_modified :

2021-09-26T15:28:22

date_modified :

2021-09-26T15:28:22

geospatial_bounds_crs :

EPSG:4326

geospatial_lat_max :

90.0

geospatial_lat_min :

-90.0

geospatial_lat_resolution :

0.5

geospatial_lat_units :

degrees_north

geospatial_lon_max :

180.0

geospatial_lon_min :

-180.0

geospatial_lon_resolution :

0.5

geospatial_lon_units :

degrees_east

history :

Errata (Version 4, Release 4b) for Version 4, Release 4

id :

10.5067/ECG5M-SSH4B

institution :

NASA Jet Propulsion Laboratory (JPL)

instrument_vocabulary :

GCMD instrument keywords

keywords :

EARTH SCIENCE SERVICES > MODELS > EARTH SCIENCE REANALYSES/ASSIMILATION MODELS, EARTH SCIENCE > OCEANS > SEA SURFACE TOPOGRAPHY > SEA SURFACE HEIGHT

keywords_vocabulary :

NASA Global Change Master Directory (GCMD) Science Keywords

license :

Public Domain

metadata_link :

https://cmr.earthdata.nasa.gov/search/collections.umm_json?ShortName=ECCO_L4_SSH_05DEG_MONTHLY_V4R4B

naming_authority :

gov.nasa.jpl

platform :

ERS-1/2, TOPEX/Poseidon, Geosat Follow-On (GFO), ENVISAT, Jason-1, Jason-2, CryoSat-2, SARAL/AltiKa, Jason-3, AVHRR, Aquarius, SSM/I, SSMIS, GRACE, DTU17MDT, Argo, WOCE, GO-SHIP, MEOP, Ice Tethered Profilers (ITP)

platform_vocabulary :

GCMD platform keywords

processing_level :

L4

product_name :

SEA_SURFACE_HEIGHT_mon_mean_2001-12_ECCO_V4r4b_latlon_0p50deg.nc

product_time_coverage_end :

2018-01-01T00:00:00

product_time_coverage_start :

1992-01-01T12:00:00

product_version :

Version 4, Release 4b (Errata for Version 4, Release 4)

program :

NASA Physical Oceanography, Cryosphere, Modeling, Analysis, and Prediction (MAP)

project :

Estimating the Circulation and Climate of the Ocean (ECCO)

publisher_email :

podaac@podaac.jpl.nasa.gov

publisher_institution :

PO.DAAC

publisher_name :

Physical Oceanography Distributed Active Archive Center (PO.DAAC)

publisher_type :

institution

publisher_url :

https://podaac.jpl.nasa.gov

references :

ECCO Consortium, Fukumori, I., Wang, O., Fenty, I., Forget, G., Heimbach, P., & Ponte, R. M. 2020. Synopsis of the ECCO Central Production Global Ocean and Sea-Ice State Estimate (Version 4 Release 4). doi:10.5281/zenodo.3765928

source :

The ECCO V4r4b state estimate was produced by fitting a free-running solution of the MITgcm (checkpoint 66g) to satellite and in situ observational data in a least squares sense using the adjoint method

standard_name_vocabulary :

NetCDF Climate and Forecast (CF) Metadata Convention

summary :

This product (Version 4, Release 4b) is the errata for Version 4 Release 4. The errata document is available at https://ecco-group.org/docs/ECCO_V4r4_errata.pdf. This dataset provides monthly-averaged dynamic sea surface height interpolated to a regular 0.5-degree grid from the ECCO Version 4 Release 4 (V4r4b) ocean and sea-ice state estimate. Estimating the Circulation and Climate of the Ocean (ECCO) state estimates are dynamically and kinematically-consistent reconstructions of the three-dimensional, time-evolving ocean, sea-ice, and surface atmospheric states. ECCO V4r4b is a free-running solution of a global, nominally 1-degree configuration of the MIT general circulation model (MITgcm) that has been fit to observations in a least-squares sense. Observational data constraints used in V4r4b include sea surface height (SSH) from satellite altimeters [ERS-1/2, TOPEX/Poseidon, GFO, ENVISAT, Jason-1,2,3, CryoSat-2, and SARAL/AltiKa]; sea surface temperature (SST) from satellite radiometers [AVHRR], sea surface salinity (SSS) from the Aquarius satellite radiometer/scatterometer, ocean bottom pressure (OBP) from the GRACE satellite gravimeter; sea-ice concentration from satellite radiometers [SSM/I and SSMIS], and in-situ ocean temperature and salinity measured with conductivity-temperature-depth (CTD) sensors and expendable bathythermographs (XBTs) from several programs [e.g., WOCE, GO-SHIP, Argo, and others] and platforms [e.g., research vessels, gliders, moorings, ice-tethered profilers, and instrumented pinnipeds]. V4r4b covers the period 1992-01-01T12:00:00 to 2018-01-01T00:00:00.

time_coverage_duration :

P1M

time_coverage_end :

2002-01-01T00:00:00

time_coverage_resolution :

P1M

time_coverage_start :

2001-12-01T00:00:00

title :

ECCO Sea Surface Height - Monthly Mean 0.5 Degree (Version 4 Release 4b)

uuid :

cd67c8ad-0f9e-4cf7-a3c3-4ac4d6016ea5