Welcome to the NetCDF Converter Service documentation!

netcdf-convert.py

Functions related to converting a NetCDF file to other formats.

net2cog.netcdf_convert._write_cogtiff(output_directory: str, nc_xarray: DataTree, variable_path: str, logger: Logger) → str | None[source]

This function converts a variable inside a NetCDF file into a cloud optimized geotiff.

Parameters:

  • output_directory (str) –

    Path to temporary directory where output GeoTIFFs will be stored before being staged in S3. example :/home/dockeruser/converter/podaac/netcdf_converter/temp/

    netcdf_converter/ RSS_smap_SSS_L3_8day_running_2020_037_FNL_v04.0_test

  • nc_xarray (xarray.DataTree) – xarray DataTree loaded from NetCDF file. This represents the whole file.

  • variable_path (str) – Full of the variable within the file to convert.

  • logger (logging.Logger) – Python Logger object for emitting log messages.

Notes

  • Assumption that 0 is always on the prime meridian/equator.

  • The output name for converted GeoTIFFs is <variable path>.tif, with any slashes replaced with underscores.

net2cog.netcdf_convert.get_all_data_variables(root_datatree: DataTree, logger: Logger) → list[str][source]

Traverse tree and retrieve all data variables in all groups.

Parameters:

root_datatree (xarray.DataTree) – DataTree object representing the root group of the NetCDF-4 file.

Returns:

A list of paths to all variables in the data_vars property of any node in the DataTree. These variables are filtered to remove any variables that are 1-D, attribute-only (e.g., CRS definitions), dtype = string(S1/S2), or variable without dimensions.

Return type:

list[str]

net2cog.netcdf_convert.get_crs_from_grid_mapping(nc_xarray: DataTree, variable_path: str, logger: Logger) → CRS[source]

Check the metadata attributes for the variable to find the associated grid mapping variable. If the grid mapping variable, as referred to in the grid_mapping CF-Convention metadata attribute, does not exist then default to “+proj=latlong”.

Parameters:

  • nc_xarray (xarray.DataTree) – xarray DataTree loaded from NetCDF file. This represents the whole file.

  • variable_path (str) – Full of the variable within the file to convert.

  • logger (logging.Logger) – Python Logger object for emitting log messages.

Returns:

Returns a CRS object corresponding to a grid mapping variable.

Return type:

csr

net2cog.netcdf_convert.netcdf_converter(input_nc_file: Path, output_directory: Path, var_list: list[str], logger: Logger) → List[str][source]

Primary function for beginning NetCDF conversion using rasterio, rioxarray and xarray

Parameters:

  • input_nc_file (pathlib.Path) – Path to NetCDF file to process

  • output_directory (pathlib.Path) – Path to temporary directory into which results will be placed before staging in S3.

  • var_list (list[str]) – List of variable names to be converted to various single band cogs, ex: [‘gland’, ‘fland’, ‘sss_smap’]. For hierarchical granules, these names will be full paths to the variable location in the file, omitting the leading slash, e.g.: ‘Grid/precipitationCal’. If this list is empty, it is assumed that all data variables have been requested.

  • logger (logging.Logger) – Python Logger object for emitting log messages.

Notes

Currently uses local file paths, no s3 paths

net2cog.netcdf_convert.process_dimension_error_exception(nc_xarray: DataTree, variable_path: str, logger: Logger, temp_file_name: str)[source]

Handles an InvalidDimensionOrder exception by attempting to swap the dimensions of a NetCDF variable to match the expected spatial layout for raster conversion.

This function applies a dimension swap strategy using swap_dims to correct issues where the variable’s dimensions are not in a valid order for rasterization (e.g., time-first or non-spatial-first layouts). It then retries writing the variable to a temporary GeoTIFF file. If the conversion fails again, it logs the error and raises a Net2CogError.

Parameters:

  • nc_xarray (xarray.DataTree) – DataTree object representing the root group of the NetCDF-4 file.

  • variable_path (str) – Variable path is present in DataTree

  • logger (logging.Logger) – Python Logger object for emitting log messages.

  • temp_file_name (str) – rio.to_raster outputs the processed .tif file to temp location

net2cog.netcdf_convert.process_invalid_dimension_order_exception(nc_xarray: DataTree, variable_path: str, logger: Logger, temp_file_name: str)[source]

This function uses the error message to identify a suitable handler function that can transform the input DataTree to resolve the issue. It then retries the raster conversion using the corrected data. If the error persists or another exception occurs, it logs the failure and raises a Net2CogError.

Parameters:

  • nc_xarray (xarray.DataTree) – DataTree object representing the root group of the NetCDF-4 file.

  • variable_path (str) – Variable path is present in DataTree

  • logger (logging.Logger) – Python Logger object for emitting log messages.

  • temp_file_name (str) – rio.to_raster outputs the processed .tif file to temp location

net2cog.netcdf_convert.process_missing_spatial_dimension_error_exception(nc_xarray: DataTree, variable_path: str, logger: Logger, temp_file_name: str)[source]

Handles an MissingSpatialDimensionError exception by attempting to reorder then rename coordinates to standard ‘x’ and ‘y’ required by rasterio

This function uses a dimension renaming strategy with .rename to fix cases where the variable’s y/x dimensions are not found. By rename coordinates to standard ‘x’ and ‘y’ required by rasterio, the issue can be resolved. It then retries writing the variable to a temporary GeoTIFF file. If the conversion fails again, it logs the error and raises a Net2CogError.

Parameters:

  • nc_xarray (xarray.DataTree) – DataTree object representing the root group of the NetCDF-4 file.

  • variable_path (str) – Variable path is present in DataTree

  • logger (logging.Logger) – Python Logger object for emitting log messages.

  • temp_file_name (str) – rio.to_raster outputs the processed .tif file to temp location

net2cog.netcdf_convert.process_value_error_exception(nc_xarray: DataTree, variable_path: str, error_message: str, logger: Logger, temp_file_name: str)[source]

This function uses the error message to identify a suitable handler function that can transform the input DataTree to resolve the issue. It then retries the raster conversion using the corrected data. If the error persists or another exception occurs, it logs the failure and raises a Net2CogError.

Parameters:

  • nc_xarray (xarray.DataTree) – DataTree object representing the root group of the NetCDF-4 file.

  • variable_path (str) – Variable path is present in DataTree

  • error_message (str) – The ValueError exception message

  • logger (logging.Logger) – Python Logger object for emitting log messages.

  • temp_file_name (str) – rio.to_raster outputs the processed .tif file to temp location

netcdf_convert_harmony.py

Implementation of harmony-service-lib that invokes the netcdf converter.

class net2cog.netcdf_convert_harmony.NetcdfConverterService(message, catalog=None, config=None)[source]

See https://github.com/nasa/harmony-service-lib-py for documentation and examples.

__init__(message, catalog=None, config=None)[source]

Constructs the adapter

Parameters:

  • message (harmony_service_lib.Message) – The Harmony input which needs acting upon

  • catalog (pystac.Catalog) – A STAC catalog containing the files on which to act

  • config (harmony_service_lib.util.Config) – The configuration values for this runtime environment.

process_item(item: Item, source: Source) → Item[source]

Performs net2cog on input STAC Item’s data, returning an output STAC item

Parameters:

  • item (pystac.Item) – the item that should be coggified

  • source (harmony_service_lib.message.Source) – the input source defining the item

Returns:

a STAC item describing the output

Return type:

pystac.Item

stage_output_and_create_output_stac(source_asset_basename: str, output_files: list[str], input_stac_item: Item) → Item[source]

Iterate through all generated COGs and stage the results in S3. Also add a unique pystac.Asset for each COG to the pystac.Item returned to Harmony.

Parameters:

  • output_files (list[str]) – the file names of generated COGs to be staged. It is assumed that the file names adhere to the convention of <variable_name>.tif.

  • input_stac_item (pystac.Item) – the input STAC for the request. This is the basis of the output STAC, which will replace the pystac.Assets with generated COGs.

Returns:

a STAC item describing the output. If there are multiple variables, this STAC item will have multiple assets.

Return type:

pystac.Item

net2cog.netcdf_convert_harmony.main()[source]

Parse command line arguments and invoke the service to respond to them.

Return type:

None

Indices and tables