Land/ice container

Builds the daily land/ice mask from OSI-SAF sea-ice concentration, and the ice-edge SST points that go with it.

This MATLAB application automates landmask and ice operations for MUR SST input data. It processes data for two grid resolutions (p01 at 0.01° and p011 at 0.011°) using OSI SAF sea ice datasets and has been containerized for easy deployment.

Overview

The application:

  • Processes land ice operations for a specified year and day-of-year
  • Handles both P01 and P011 grid resolutions automatically in each run
  • Uses OSI SAF FTP endpoints for sea ice data access
  • Compiled as a standalone MATLAB executable for containerized deployment
  • No MATLAB license required to run the compiled version
  • Takes every static input file as an explicit named flag — no directory is bind-mounted whole and scanned; the calling orchestrator (run_mur_pipeline.py locally, run_mur_maap.py on MAAP) resolves and passes each of the six static files individually
  • Each --*-file flag value may be either a local filesystem path (local docker run, bind-mounted) or an s3:// href (MAAP, or any environment with S3 access) — the container fetches s3:// values to local scratch space itself before MATLAB runs, so the same image works unchanged in either environment (see Container Features below)
  • Optimized with shared memory configuration for MATLAB Runtime performance

Build and Run

⚠️ Important: The build process requires a valid MATLAB license server connection during compilation. If the license server is unreachable, the build may hang at the compilation step. Copy network.lic.example to mur/network.lic and point it at your license server before the first build.

# Build with build_module.sh from the mur/ directory
    cd mur
    ./build_module.sh landice

    # Run — every static input is an explicit flag, bind-mounted individually
    docker run --rm --shm-size=512M \
      -e OSISAF_FTP_ARCHIVE="https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc" \
      -e OSISAF_FTP_PROD="https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc" \
      -v /path/to/static-resources/grids/maskGLOBp01deg.gds:/input/landmask-p01.gds:ro \
      -v /path/to/static-resources/mat/p01/saf2north.mat:/input/gridindex-north-p01.mat:ro \
      -v /path/to/static-resources/mat/p01/saf2south.mat:/input/gridindex-south-p01.mat:ro \
      -v /path/to/static-resources/grids/maskGlob1km.gds:/input/landmask-p011.gds:ro \
      -v /path/to/static-resources/mat/p011/saf2north.mat:/input/gridindex-north-p011.mat:ro \
      -v /path/to/static-resources/mat/p011/saf2south.mat:/input/gridindex-south-p011.mat:ro \
      -v /path/to/output/p011:/output/p011 \
      -v /path/to/output/p01:/output/p01 \
      mur-landice:latest \
      --year 2024 --doy 100 \
      --landmask-p01-file /input/landmask-p01.gds \
      --gridindex-north-p01-file /input/gridindex-north-p01.mat \
      --gridindex-south-p01-file /input/gridindex-south-p01.mat \
      --landmask-p011-file /input/landmask-p011.gds \
      --gridindex-north-p011-file /input/gridindex-north-p011.mat \
      --gridindex-south-p011-file /input/gridindex-south-p011.mat

See Static input data for where the six static files (grids/maskGLOBp01deg.gds, mat/p01/saf2north.mat, etc.) come from and their full-tree layout.

Manual Build (Advanced)

build_module.sh is the supported path. Build by hand only for a custom tag, external CI, or Dockerfile debugging — the build context must be the parent mur directory so the Dockerfile can reach common/, and the mur-matlab-base:r2024b image must already exist (./build_matlab_base.sh from mur/):

cd mur/landice
    docker build --platform linux/amd64 -f Dockerfile -t mur-landice:latest ..

    # Or with Apple's container tools on macOS:
    # container build --arch amd64 -f Dockerfile -t mur-landice:latest ..

See Manual Builds (Advanced) and Building the image for the in-container mcc workflow.

Container Features

  • Explicit named inputs: every static file (landmask + grid-index files, both resolutions) is its own --flag; there is no directory mount to scan
  • Named args only: --year/--doy plus the six file flags — no positional-argument form is accepted
  • Local-path or S3-href inputs, uniformly: entrypoint.sh sources the shared common/bin/localize.sh helper and resolves each of the six file flags before MATLAB runs — a value starting with s3:// is fetched to local scratch space (via aws s3 cp) and the local path substituted; any other value is assumed to already be a local path (e.g. a bind mount) and passed through unchanged. AWS credentials come from the standard AWS credential chain — environment variables (AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY/AWS_SESSION_TOKEN) for local dev, an IAM role automatically when running on AWS/MAAP compute — no extra flags or config needed either way.
  • Automatic processing: Processes both P01 and P011 grids in a single run
  • Optimized performance: Requires --shm-size=512M for MATLAB Runtime (mandatory)
  • No license required: Self-contained with MATLAB Runtime R2024b
  • Memory optimization: Supports configurable memory limits and Java heap settings

Requirements

For Containerized Deployment

  • Docker
  • The six static input files (see Static input data)
  • Output directories (one per resolution: p011, p01)

For Development

  • MATLAB R2024b with MATLAB Compiler
  • Access to MATLAB license server

Usage Examples

Basic Usage

# Process day 150 of 2024
    docker run --rm --shm-size=512M \
      -v /data/static-resources/grids/maskGLOBp01deg.gds:/input/landmask-p01.gds:ro \
      -v /data/static-resources/mat/p01/saf2north.mat:/input/gridindex-north-p01.mat:ro \
      -v /data/static-resources/mat/p01/saf2south.mat:/input/gridindex-south-p01.mat:ro \
      -v /data/static-resources/grids/maskGlob1km.gds:/input/landmask-p011.gds:ro \
      -v /data/static-resources/mat/p011/saf2north.mat:/input/gridindex-north-p011.mat:ro \
      -v /data/static-resources/mat/p011/saf2south.mat:/input/gridindex-south-p011.mat:ro \
      -v /data/output/p011:/output/p011 \
      -v /data/output/p01:/output/p01 \
      mur-landice:latest \
      --year 2024 --doy 150 \
      --landmask-p01-file /input/landmask-p01.gds \
      --gridindex-north-p01-file /input/gridindex-north-p01.mat \
      --gridindex-south-p01-file /input/gridindex-south-p01.mat \
      --landmask-p011-file /input/landmask-p011.gds \
      --gridindex-north-p011-file /input/gridindex-north-p011.mat \
      --gridindex-south-p011-file /input/gridindex-south-p011.mat

With Custom OSI SAF Endpoints

docker run --rm --shm-size=512M \
      -e OSISAF_FTP_REPROCESSED="ftp://your-server/reprocessed" \
      -e OSISAF_FTP_ARCHIVE="ftp://your-server/archive" \
      -e OSISAF_FTP_PROD="ftp://your-server/prod" \
      -v /data/static-resources/grids/maskGLOBp01deg.gds:/input/landmask-p01.gds:ro \
      -v /data/static-resources/mat/p01/saf2north.mat:/input/gridindex-north-p01.mat:ro \
      -v /data/static-resources/mat/p01/saf2south.mat:/input/gridindex-south-p01.mat:ro \
      -v /data/static-resources/grids/maskGlob1km.gds:/input/landmask-p011.gds:ro \
      -v /data/static-resources/mat/p011/saf2north.mat:/input/gridindex-north-p011.mat:ro \
      -v /data/static-resources/mat/p011/saf2south.mat:/input/gridindex-south-p011.mat:ro \
      -v /data/output/p011:/output/p011 \
      -v /data/output/p01:/output/p01 \
      mur-landice:latest \
      --year 2024 --doy 150 \
      --landmask-p01-file /input/landmask-p01.gds \
      --gridindex-north-p01-file /input/gridindex-north-p01.mat \
      --gridindex-south-p01-file /input/gridindex-south-p01.mat \
      --landmask-p011-file /input/landmask-p011.gds \
      --gridindex-north-p011-file /input/gridindex-north-p011.mat \
      --gridindex-south-p011-file /input/gridindex-south-p011.mat

With Performance Optimization

docker run --rm \
      --memory=4g \
      --memory-reservation=2g \
      --memory-swap=6g \
      --shm-size=512M \
      --cpus="2.0" \
      -e MCR_CACHE_ROOT=/tmp/mcr_cache \
      -e MCR_CACHE_SIZE=1024M \
      -e _JAVA_OPTIONS="-Xmx2048m -Xms512m -XX:+UseG1GC" \
      -e OSISAF_FTP_ARCHIVE="https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc" \
      -e OSISAF_FTP_PROD="https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc" \
      -v /data/static-resources/grids/maskGLOBp01deg.gds:/input/landmask-p01.gds:ro \
      -v /data/static-resources/mat/p01/saf2north.mat:/input/gridindex-north-p01.mat:ro \
      -v /data/static-resources/mat/p01/saf2south.mat:/input/gridindex-south-p01.mat:ro \
      -v /data/static-resources/grids/maskGlob1km.gds:/input/landmask-p011.gds:ro \
      -v /data/static-resources/mat/p011/saf2north.mat:/input/gridindex-north-p011.mat:ro \
      -v /data/static-resources/mat/p011/saf2south.mat:/input/gridindex-south-p011.mat:ro \
      -v /data/output/p011:/output/p011 \
      -v /data/output/p01:/output/p01 \
      mur-landice:latest \
      --year 2024 --doy 150 \
      --landmask-p01-file /input/landmask-p01.gds \
      --gridindex-north-p01-file /input/gridindex-north-p01.mat \
      --gridindex-south-p01-file /input/gridindex-south-p01.mat \
      --landmask-p011-file /input/landmask-p011.gds \
      --gridindex-north-p011-file /input/gridindex-north-p011.mat \
      --gridindex-south-p011-file /input/gridindex-south-p011.mat

Batch Processing

For processing multiple days, create a simple bash script:

#!/bin/bash
    YEAR=2024
    START_DOY=1
    END_DOY=365
    STATIC=/data/static-resources

    for DOY in $(seq $START_DOY $END_DOY); do
        echo "Processing Year: $YEAR, DOY: $DOY"

        docker run --rm --shm-size=512M \
            -e OSISAF_FTP_ARCHIVE="https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc" \
            -e OSISAF_FTP_PROD="https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc" \
            -v "$STATIC/grids/maskGLOBp01deg.gds:/input/landmask-p01.gds:ro" \
            -v "$STATIC/mat/p01/saf2north.mat:/input/gridindex-north-p01.mat:ro" \
            -v "$STATIC/mat/p01/saf2south.mat:/input/gridindex-south-p01.mat:ro" \
            -v "$STATIC/grids/maskGlob1km.gds:/input/landmask-p011.gds:ro" \
            -v "$STATIC/mat/p011/saf2north.mat:/input/gridindex-north-p011.mat:ro" \
            -v "$STATIC/mat/p011/saf2south.mat:/input/gridindex-south-p011.mat:ro" \
            -v /data/output/p011:/output/p011 \
            -v /data/output/p01:/output/p01 \
            mur-landice:latest \
            --year "$YEAR" --doy "$DOY" \
            --landmask-p01-file /input/landmask-p01.gds \
            --gridindex-north-p01-file /input/gridindex-north-p01.mat \
            --gridindex-south-p01-file /input/gridindex-south-p01.mat \
            --landmask-p011-file /input/landmask-p011.gds \
            --gridindex-north-p011-file /input/gridindex-north-p011.mat \
            --gridindex-south-p011-file /input/gridindex-south-p011.mat

        if [ $? -ne 0 ]; then
            echo "Error processing Year: $YEAR, DOY: $DOY"
        fi
    done

In practice, run_mur_pipeline.py already does this per-day looping and flag construction for you — see the main the project README.

Environment Variables

The application uses these environment variables to locate OSI-SAF ice concentration data. OSI-SAF's anonymous FTP host is dead — connections to ftp://osisaf.met.no time out — so the defaults are HTTPS THREDDS endpoints. The names keep their _FTP_ spelling because they are part of the container's published interface; readosisafice.m dispatches on the URL scheme, so an ftp:// value still works if one is supplied.

  • OSISAF_FTP_ARCHIVE: recent days (default: https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc)
  • OSISAF_FTP_PROD: current day (default: same as archive)
  • OSISAF_FTP_REPROCESSED: pre-2009 dates — no default. The polstere-100 "reproc" product this once pointed at has no verified HTTPS equivalent on thredds.met.no, so historical reprocessing must supply it explicitly rather than inherit a dead URL.

Note on the amsr2_conc path: despite the name, that tree serves AMSR3 files from 2026-08-31 onward. readosisafice.m selects the amsr2/amsr3 filename token by date; the cutover is overridable with OSISAF_AMSR3_START=YYYYMMDD.

Directory Structure

landice/
    ├── src/
    │   ├── landice_wrapper.m      # Compiled entry point — accepts the six explicit static file paths
    │   ├── makeicefiles.m         # Ice file generation (core logic)
    │   ├── readosisafice.m        # OSI SAF data reader and downloader
    │   └── setup_environment.m    # Environment setup
    ├── bin/
    │   └── entrypoint.sh          # Named-args-only container entrypoint; localizes s3:// inputs before invoking MATLAB
    ├── tests/
    │   ├── test_entrypoint.sh     # Entrypoint argument-parsing + localization unit tests
    │   ├── in/, out/               # Test fixtures
    │   └── compare_outputs.py, validate_containerized.sh, test_execute_landice.py
    └── Dockerfile                 # Multi-stage build (MATLAB compiler → runtime-only image, includes AWS CLI)

    ../common/
    ├── bin/
    │   └── localize.sh            # Shared s3://-or-local-path resolver, sourced by entrypoint.sh (also used by other containers as they're converted)
    └── *.m                        # Shared MATLAB utilities (compiled into the MATLAB build stage)

Documentation

  • Building the image - Complete build instructions for development
  • Deployment - Production deployment guide with Docker Compose and Kubernetes examples
  • Static input data - Where the six static input files live and how they're laid out

Production Deployment

Docker Compose

Create docker-compose.yml:

version: '3.8'

    services:
      landice:
        image: mur-landice:latest
        environment:
          - OSISAF_FTP_REPROCESSED=${OSISAF_FTP_REPROCESSED}
          - OSISAF_FTP_ARCHIVE=${OSISAF_FTP_ARCHIVE}
          - OSISAF_FTP_PROD=${OSISAF_FTP_PROD}
        volumes:
          - ${STATIC_RESOURCES_DIR}/grids/maskGLOBp01deg.gds:/input/landmask-p01.gds:ro
          - ${STATIC_RESOURCES_DIR}/mat/p01/saf2north.mat:/input/gridindex-north-p01.mat:ro
          - ${STATIC_RESOURCES_DIR}/mat/p01/saf2south.mat:/input/gridindex-south-p01.mat:ro
          - ${STATIC_RESOURCES_DIR}/grids/maskGlob1km.gds:/input/landmask-p011.gds:ro
          - ${STATIC_RESOURCES_DIR}/mat/p011/saf2north.mat:/input/gridindex-north-p011.mat:ro
          - ${STATIC_RESOURCES_DIR}/mat/p011/saf2south.mat:/input/gridindex-south-p011.mat:ro
          - ${OUTPUT_DIR_P011}:/output/p011
          - ${OUTPUT_DIR_P01}:/output/p01
        command:
          - "--year"
          - "${YEAR}"
          - "--doy"
          - "${DOY}"
          - "--landmask-p01-file"
          - "/input/landmask-p01.gds"
          - "--gridindex-north-p01-file"
          - "/input/gridindex-north-p01.mat"
          - "--gridindex-south-p01-file"
          - "/input/gridindex-south-p01.mat"
          - "--landmask-p011-file"
          - "/input/landmask-p011.gds"
          - "--gridindex-north-p011-file"
          - "/input/gridindex-north-p011.mat"
          - "--gridindex-south-p011-file"
          - "/input/gridindex-south-p011.mat"
        shm_size: '512m'

Kubernetes

Deploy as a Job:

apiVersion: batch/v1
    kind: Job
    metadata:
      name: landice-processor
    spec:
      template:
        spec:
          containers:
          - name: landice
            image: mur-landice:latest
            args:
              - "--year"
              - "2024"
              - "--doy"
              - "100"
              - "--landmask-p01-file"
              - "/input/landmask-p01.gds"
              - "--gridindex-north-p01-file"
              - "/input/gridindex-north-p01.mat"
              - "--gridindex-south-p01-file"
              - "/input/gridindex-south-p01.mat"
              - "--landmask-p011-file"
              - "/input/landmask-p011.gds"
              - "--gridindex-north-p011-file"
              - "/input/gridindex-north-p011.mat"
              - "--gridindex-south-p011-file"
              - "/input/gridindex-south-p011.mat"
            env:
            - name: OSISAF_FTP_ARCHIVE
              value: "https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc"
            - name: OSISAF_FTP_PROD
              value: "https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc"
            volumeMounts:
            - name: static-resources
              mountPath: /input
              readOnly: true
            - name: output-p011
              mountPath: /output/p011
            - name: output-p01
              mountPath: /output/p01
            resources:
              requests:
                memory: "2Gi"
                cpu: "1"
              limits:
                memory: "4Gi"
                cpu: "2"
          restartPolicy: OnFailure

(The example mounts a directory at /input for brevity here — in practice each --*-file flag should point at an individually-mounted file, matching the docker run examples above, so nothing inside the container ever scans a directory to find its inputs.)

Troubleshooting

Build Issues

Build hangs during compilation:

  • Check MATLAB license server connectivity
  • Ensure valid MATLAB Compiler license is available
  • Verify network access to license servers (may require VPN)

Runtime Issues

Memory errors:

# Increase memory allocation (append to any of the examples above)
    docker run --rm \
      --shm-size=512M \
      --memory="8g" \
      --memory-swap="16g" \
      ... \
      mur-landice:latest \
      --year 2024 --doy 100 \
      --landmask-p01-file /input/landmask-p01.gds \
      --gridindex-north-p01-file /input/gridindex-north-p01.mat \
      --gridindex-south-p01-file /input/gridindex-south-p01.mat \
      --landmask-p011-file /input/landmask-p011.gds \
      --gridindex-north-p011-file /input/gridindex-north-p011.mat \
      --gridindex-south-p011-file /input/gridindex-south-p011.mat

Permission issues:

# Run with user permissions
    docker run --rm \
      --shm-size=512M \
      --user $(id -u):$(id -g) \
      ... \
      mur-landice:latest \
      --year 2024 --doy 100 \
      --landmask-p01-file /input/landmask-p01.gds \
      --gridindex-north-p01-file /input/gridindex-north-p01.mat \
      --gridindex-south-p01-file /input/gridindex-south-p01.mat \
      --landmask-p011-file /input/landmask-p011.gds \
      --gridindex-north-p011-file /input/gridindex-north-p011.mat \
      --gridindex-south-p011-file /input/gridindex-south-p011.mat

Missing/unknown-flag errors: all nine flags (--year, --doy, and the six --*-file flags) are required — positional arguments are no longer accepted. Run docker run --rm mur-landice:latest with no arguments to see the usage message.

Resolution Notes

The landice container generates two grid resolutions per run:

ResolutionGridOutput directoryFilename patternUsed by
p01 (0.01°)maskGLOBp01deg.gdsland/p01/YYYY/landiceP01_YYYY_DDD.gds.gzMUR v4 (current)
p011 (0.011°/~1km)maskGlob1km.gdsland/p011/YYYY/landice_YYYY_DDD.gds.gzMUR v3 (legacy, not used)

Only p01 output is used by the current MUR v4 pipeline. The p011 resolution is a legacy artifact from MUR v3 (0.011° product). The MRVA container mounts only the land/p01/ subdirectory and expects landiceP01_ prefixed filenames. The p011 output is generated but not consumed by any current processing stage.

In the original production system, both resolutions were written to the same flat directory (/nas2/landice/YYYY/), so the P01 filename prefix was necessary to distinguish them. In the containerized setup, they are separated into land/p01/ and land/p011/ subdirectories, making the prefix redundant — but it is retained for compatibility with the MRVA MATLAB code that expects it.

General Notes

  • Execution time is logged for monitoring purposes
  • Container uses MATLAB Runtime R2024b - no MATLAB license required for execution
  • Shared memory (--shm-size=512M) is mandatory for MATLAB Runtime
  • Every static input is passed as its own bind-mounted file and named flag — nothing is discovered by scanning a directory
  • Each of the two output directories (p011, p01) will contain the processed land ice files for that resolution

Building the image

Build with build_module.sh from the mur/ directory. It builds the mur-matlab-base:r2024b image if missing, checks network.lic, sets --platform linux/amd64, uses the parent mur/ directory as build context (so the Dockerfile can reach common/), and tags the image mur-landice:latest:

cd mur
    ./build_module.sh landice

Run it (see the usage examples above for the full named-flag invocation):

docker run --rm --shm-size=512M \
      -v /path/to/static-resources:/input:ro \
      -v /path/to/output:/output \
      mur-landice:latest \
      --year 2024 --doy 100 ...

For a raw docker build — custom tags, external CI, Dockerfile debugging — see Manual Builds (Advanced):

cd mur/landice
    docker build --platform linux/amd64 -f Dockerfile -t mur-landice:latest ..

Manual Compilation Process (Advanced)

The rest of this document covers compiling the MATLAB application by hand inside a development container — useful when debugging mcc dependency problems or toolbox requirements, not part of the normal build path. The single landice/Dockerfile does all of this automatically; the Dockerfile.matlab / Dockerfile.runtime split described below is historical, so adapt the commands to the current Dockerfile's builder stage.

Prerequisites

  • Docker installed on your system
  • Access to MATLAB license server
  • Network connectivity to download MATLAB toolboxes

Find the required toolboxes with


    [files, products] = matlab.codetools.requiredFilesAndProducts('myScript.m');

    % Display required toolboxes
    for i = 1:length(products)
        fprintf('Required: %s\n', products(i).Name);
    end

Building the MATLAB Development Container

1. Build the Docker Image

Note: Dockerfile.matlab no longer exists. The equivalent today is the builder stage of landice/Dockerfile, or the shared base image mur-matlab-base:r2024b built by ./build_matlab_base.sh. Either gives you MATLAB R2024b + Compiler with common/ available:

# Build the shared MATLAB base image (from mur/)
    cd mur
    ./build_matlab_base.sh

    # Or stop the landice build at its builder stage
    cd mur/landice
    docker build --platform linux/amd64 --target builder -t matlab-landice:r2024b -f Dockerfile ..

Both build with the parent mur directory as context, which is what lets the Dockerfile reach landice/ and common/. Substitute whichever image you built for matlab-landice:r2024b in the commands below.

2. Run Interactive Terminal Session

# Run container with volume mounting
    # Builds to /tmp inside container to avoid permission issues
    docker run -it --rm \
      -v $(pwd):/app/landice \
      matlab-landice:r2024b

Note:

  • Container runs as the matlab user (not root) to comply with license server restrictions
  • Build process uses /tmp to avoid volume mount permission issues

3. Build the LandIce Application Inside Container

Once inside the container terminal:

# Navigate to the landice directory
    cd /app/landice

    # Method 1: Build to temp location and copy back (recommended)
    # This avoids volume mount permission issues
    # -C flag creates separate CTF file for faster runtime extraction
    mkdir -p /tmp/landice_build
    mcc -m src/landice_wrapper.m \
        -a src/makeicefiles.m \
        -a src/readosisafice.m \
        -a src/setup_environment.m \
        -a src/julian.m \
        -o LandIce \
        -d /tmp/landice_build \
        -C \
        -R -nodisplay \
        -R -nosplash

    # Copy the compiled files back to the mounted volume
    cp -R /tmp/landice_build/LandIce ./build

Alternative methods:

# Method 2: Auto-detect dependencies (build to temp and copy)
    mkdir -p /tmp/landice_build
    mcc -m -v -d /tmp/landice_build -o LandIce src/landice_wrapper.m
    mkdir -p ./build && cp /tmp/landice_build/* ./build/

    # Method 3: Interactive MATLAB (if needed for debugging)
    matlab -nodisplay -nosplash
    # In MATLAB:
    # >> mcc -m src/landice_wrapper.m -a src/makeicefiles.m -a src/readosisafice.m -a src/setup_environment.m -a src/julian.m -o LandIce -d /tmp/landice_build
    # >> exit
    # Then copy files: cp /tmp/landice_build/* ./build/

Notes:

  • Building to /tmp/landice_build avoids volume mount permission issues completely
  • The -m flag creates a standalone executable binary (not an installer)
  • The -C flag creates a separate CTF archive for faster runtime extraction
  • The -n flag automatically treats numeric command line arguments as MATLAB doubles (solves string conversion issues)
  • The resulting executable only requires MATLAB Runtime R2024b, not full MATLAB
  • The -v flag provides verbose output for troubleshooting
  • MATLAB R2015b+ automatically finds dependencies, so -a for each file may not be necessary
  • Files are copied back to ./build/ which is accessible from the host system

Output Files: The compilation will produce:

  • LandIce - The standalone executable binary (no installer)
  • LandIce.ctf - Separate CTF (Component Technology File) archive for faster runtime extraction
  • run_LandIce.sh - Shell script wrapper that sets up environment variables
  • readme.txt - Runtime requirements and deployment instructions
  • requiredMCRProducts.txt - List of required MATLAB Runtime components

Running the Compiled Executable

Direct Execution (with MATLAB Runtime installed)

# Using the wrapper script (recommended - handles environment setup)
    # MCR path may vary by installation - check your system
    ./build/run_LandIce.sh /usr/local/MATLAB/MATLAB_Runtime/R2024b input_dir /path/to/input output_dir /path/to/output year 2024 doy 100

    # Alternative common MCR paths:
    ./build/run_LandIce.sh /opt/mcr/R2024b input_dir /path/to/input output_dir /path/to/output year 2024 doy 100

    # Or directly (requires LD_LIBRARY_PATH to be set)
    export MCR_ROOT=/usr/local/MATLAB/MATLAB_Runtime/R2024b
    export LD_LIBRARY_PATH=$MCR_ROOT/runtime/glnxa64:$MCR_ROOT/bin/glnxa64:$MCR_ROOT/sys/os/glnxa64:$LD_LIBRARY_PATH
    ./build/LandIce input_dir /path/to/input output_dir /path/to/output year 2024 doy 100

Note: When running the compiled executable directly (not in container), you need to specify input and output paths. The container entrypoint automatically handles this by passing input_dir /input output_dir /output to the application.

Important:

  • The executable is a binary file, NOT an installer
  • It only requires MATLAB Runtime R2024b to be installed
  • No MATLAB license is needed to run the compiled executable
  • The runtime can be freely distributed
  • MCR installation path varies by system - check your actual installation location

Building Runtime-Only Container

1. Create Runtime Dockerfile

Historical: the current landice/Dockerfile already produces the runtime image as its final stage, so this step is not needed for a normal build. Kept as a reference for standalone runtime images. Create Dockerfile.runtime:

FROM containers.mathworks.com/matlab-runtime:r2024b

    # Copy compiled application
    COPY --from=matlab-landice:r2024b /app/landice/LandIce/for_testing /opt/landice

    # Set environment variables
    ENV LD_LIBRARY_PATH="/opt/mcr/v913/runtime/glnxa64:/opt/mcr/v913/bin/glnxa64:/opt/mcr/v913/sys/os/glnxa64:${LD_LIBRARY_PATH}"

    # Create working directory
    WORKDIR /data

    # Set entrypoint to the application
    ENTRYPOINT ["/opt/landice/run_LandIce.sh", "/opt/mcr/v913"]

2. Build Runtime Container

docker build -f Dockerfile.runtime -t landice-runtime:latest .

3. Run the Application

docker run --rm \
      --shm-size=512M \
      -v /path/to/input:/input:ro \
      -v /path/to/output:/output \
      landice-runtime:latest \
      year 2024 \
      doy 100

Environment Variables Required

The application expects these environment variables to be set:

  • OSISAF_FTP_REPROCESSED: FTP path for reprocessed data
  • OSISAF_FTP_ARCHIVE: FTP path for archive data
  • OSISAF_FTP_PROD: FTP path for production data

Set these in the Docker run command:

docker run --rm \
      --shm-size=512M \
      -e OSISAF_FTP_REPROCESSED="ftp://example.com/reprocessed" \
      -e OSISAF_FTP_ARCHIVE="ftp://example.com/archive" \
      -e OSISAF_FTP_PROD="ftp://example.com/prod" \
      -v /path/to/input:/input:ro \
      -v /path/to/output:/output \
      landice-runtime:latest \
      year 2024 \
      doy 100

Troubleshooting

License Server Connection Issues

If you get license errors, ensure:

  1. The container can reach the license server
  2. The license server hostname is resolvable
  3. Port 7282 is accessible

Memory Issues

If MATLAB crashes or behaves unexpectedly, increase shared memory:

docker run -it --rm --shm-size=4gb matlab-landice:r2024b

Compilation Errors

If compilation fails, check:

  1. All source files are present in the container
  2. The project file paths are correct
  3. Required toolboxes are installed

Verify toolboxes:

docker run --rm matlab-landice:r2024b matlab -batch "ver"

Deployment

Overview

This guide explains how to deploy the compiled LandIce MATLAB application in a lightweight runtime container that only includes the MATLAB Runtime (MCR), not the full MATLAB installation.

The container uses fixed paths:

  • /input - Mount your input data here (read-only)
  • /output - Mount your output directory here
  • Only year and doy arguments are required as positional parameters - the entrypoint automatically maps /input and /output directories

Building Containers

Build with build_module.sh from the mur/ directory — it handles the base image, license check, platform flag, build context, and the mur-landice:latest tag the pipeline config expects:

cd mur
    ./build_module.sh landice

Manual Build (Advanced)

Only for custom tags, external CI, or Dockerfile debugging.

IMPORTANT: The Dockerfile references shared utilities from the ../common folder, so the build context must include the parent mur directory. Build from within the landice directory and set the context to the parent (..). The mur-matlab-base:r2024b image must already exist — run ./build_matlab_base.sh from mur/ first.

# Navigate to the landice directory
    cd mur/landice

    # Build with Docker specifying the platform
    docker build --platform linux/amd64 -f Dockerfile -t mur-landice:latest ..

    # Or, on macOS with Apple's container tools, build for AMD64
    container build --arch amd64 -f Dockerfile -t mur-landice:latest ..

Note: The .. at the end sets the build context to the parent mur directory, which allows the Dockerfile to access both landice/ and common/ folders. When building AMD64 containers on Apple Silicon Macs, the --arch amd64 flag ensures the container is built for x86_64 architecture, which may be required for compatibility with production environments.

Deployment Options

Build and deploy in a single Docker build process:

⚠️ Important License Warning: The build process requires a valid MATLAB license server connection during compilation. If the license server is unreachable or the license cannot be obtained, the build may hang indefinitely at the compilation step. Ensure:

  • Your network can reach the MATLAB license servers
  • You have valid MATLAB and MATLAB Compiler licenses
  • The network.lic file exists in the parent directory (copy from network.lic.example)
# Build the multi-stage container
    cd mur
    ./build_module.sh landice

    # Run the application
    docker run --rm \
      --memory=4g \
      --memory-reservation=2g \
      --memory-swap=6g \
      --shm-size=512M \
      --cpus="2.0" \
      -e MCR_CACHE_ROOT=/tmp/mcr_cache \
      -e MCR_CACHE_SIZE=1024M \
      -e _JAVA_OPTIONS="-Xmx2048m -Xms512m -XX:+UseG1GC -XX:+UseStringDeduplication" \
      -e OSISAF_FTP_ARCHIVE="https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc" \
      -e OSISAF_FTP_PROD="https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc" \
      -v $(pwd)/tests/in:/input:ro \
      -v $(pwd)/tests/out:/output \
      mur-landice:latest \
      2024 101

Performance Optimization

Shared Memory Configuration (Required)

MATLAB Runtime requires shared memory to be set to 512MB for optimal performance:

docker run --rm \
      --shm-size=512M \
      mur-landice:latest [arguments]

Memory Settings

For large datasets, adjust Docker memory limits:

docker run --rm \
      --shm-size=512M \
      --memory="4g" \
      --memory-swap="4g" \
      --cpus="2" \
      mur-landice:latest [arguments]

MCR Cache Settings

Optimize MATLAB Runtime cache:

docker run --rm \
      --shm-size=512M \
      -e MCR_CACHE_SIZE=2048M \
      -e MCR_CACHE_ROOT=/tmp/mcr_cache \
      -v /fast/storage/mcr_cache:/tmp/mcr_cache \
      mur-landice:latest [arguments]

Monitoring and Logging

Enable Verbose Logging

docker run --rm \
      --shm-size=512M \
      -e MCR_CACHE_VERBOSE=true \
      mur-landice:latest [arguments] 2>&1 | tee landice_$(date +%Y%m%d_%H%M%S).log

Health Check Script

#!/bin/bash
    # healthcheck.sh

    docker run --rm mur-landice:latest -? > /dev/null 2>&1
    if [ $? -eq 0 ]; then
        echo "LandIce container is healthy"
        exit 0
    else
        echo "LandIce container health check failed"
        exit 1
    fi

Troubleshooting

Common Issues and Solutions

  1. Build Hangs During Compilation If the Docker build appears to hang at the mcc compilation step: 
    # Check license server connectivity from your host
    telnet .....

    # Or test with a simpler MATLAB command first
    docker run --rm \
      -e MLM_LICENSE_FILE={PORT}@{SERVER} \
      mathworks/matlab:r2024b \
      -batch "disp('License check successful')"

    Common causes:

    • License server unreachable (network/firewall issues)
    • No available licenses
    • Incorrect license server configuration
    • VPN required for license server access
  2. Missing Libraries 
    # Check for missing libraries
    docker run --rm mur-landice:latest ldd /opt/landice/bin/LandIce
  3. Permission Issues 
    # Ensure output directory is writable
    docker run --rm \
      --shm-size=512M \
      --user $(id -u):$(id -g) \
      -v /path/to/output:/output \
      mur-landice:latest [arguments]
  4. Network Issues 
    # Test connectivity to the OSI-SAF THREDDS endpoint. Do NOT test
    # ftp://osisaf.met.no — that host is dead and the probe just hangs
    # until it times out.
    docker run --rm mur-landice:latest \
      bash -c "curl -sS -o /dev/null -w '%{http_code}\n' -I \
        https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/amsr2_conc/2026/09/ice_conc_nh_polstere-100_amsr3_202609131200.nc"
  5. Memory Issues Increase Docker memory allocation or use swap: 
    docker run --rm \
      --shm-size=512M \
      --memory="8g" \
      --memory-swap="16g" \
      mur-landice:latest [arguments]

Security Considerations

  1. Run as Non-Root User 
    # Add to Dockerfile.runtime
    RUN useradd -m -u 1000 landice
    USER landice
  2. Read-Only Root Filesystem 
    docker run --rm \
      --read-only \
      --tmpfs /tmp \
      mur-landice:latest [arguments]
  3. Network Isolation Use Docker networks to isolate containers: 
    docker network create landice-net
    docker run --rm --network=landice-net mur-landice:latest [arguments]

Reprocessing past dates

Summary

Different grid resolutions (p01 at 0.01° and p011 at 0.011°) have different reprocessing cutoff dates, which means they may pull ice concentration data from different FTP sources for the same calendar date.

Critical Dates

p011 Grid (0.011° / ~1km resolution)

  • Cutoff Date: 2006-12-31 (MJD = 54100)
  • Dates ≤ 2006-12-31: Use REPROCESSED FTP source (*_reproc_*.nc files)
  • Dates > 2006-12-31: Use ARCHIVE/PROD FTP source (*_multi_*.nc files)

p01 Grid (0.01° resolution)

  • Cutoff Date: 2008-12-31 (MJD = 54831)
  • Dates ≤ 2008-12-31: Use REPROCESSED FTP source (*_reproc_*.nc files)
  • Dates > 2008-12-31: Use ARCHIVE/PROD FTP source (*_multi_*.nc files)

Date Range Analysis

Range 1: Before 2006-12-31

  • p011: REPROCESSED source
  • p01: REPROCESSED source
  • Result: ✅ SAME files - optimization possible

Range 2: 2007-01-01 to 2008-12-31 ⚠️

  • p011: ARCHIVE/PROD source
  • p01: REPROCESSED source
  • Result: ❌ DIFFERENT files - must download separately!

Range 3: After 2008-12-31

  • p011: ARCHIVE/PROD source
  • p01: ARCHIVE/PROD source
  • Result: ✅ SAME files - optimization possible

Why This Matters

The attempted optimization in landice.m (downloading ice files once and reusing for both grids) fails for dates in 2007-2008 because:

  1. p011 needs ice_conc_*_polstere-100_multi_YYYYMMDD1200.nc (archive)
  2. p01 needs ice_conc_*_polstere-100_reproc_YYYYMMDD1200.nc (reprocessed)
  3. These are different files with potentially different data

Solution: Enhanced makeicefiles.m

We've reverted to using makeicefiles.m with enhanced logging:

Changes Made:

  1. Clear documentation in switch statement (makeicefiles.m:48-67)
    • Added comments explaining cutoff dates
    • Shows which file type is used for each date range
  2. Logging of reprocessing cutoff (makeicefiles.m:85-88)
    • Prints the cutoff date for the current resolution
    • Shows MJD value for debugging
  3. Enhanced FTP source logging (readosisafice.m:24-51)
    • Shows current date's MJD vs cutoff MJD
    • Explicitly states which FTP source is selected (REPROCESSED/ARCHIVE/PROD)
    • Logs hemisphere being processed

Example Log Output

makeicefiles: resolution - p01
    makeicefiles: OSI-SAF reprocessing cutoff for p01 grid: 2008-12-31 (MJD=54831)

    readOSISAF: Date 20070701 (nh, MJD=54282) <= reprocessing cutoff (MJD=54831)
    readOSISAF: Using REPROCESSED FTP source: https://thredds.met.no/thredds/fileServer/osisaf/met.no/reprocessed/ice/conc

vs

makeicefiles: resolution - p011
    makeicefiles: OSI-SAF reprocessing cutoff for p011 grid: 2006-12-31 (MJD=54100)

    readOSISAF: Date 20070701 (nh, MJD=54282) > reprocessing cutoff (MJD=54100), <= archive cutoff (MJD=Inf)
    readOSISAF: Using ARCHIVE FTP source: https://thredds.met.no/thredds/fileServer/osisaf/met.no/ice/conc

Notice how the same date (2007-07-01) triggers different FTP sources for p01 vs p011!

Files Modified

  1. src/makeicefiles.m - Added comments and logging for cutoff dates
  2. src/readosisafice.m - Enhanced FTP source selection logging

Recommendation

Continue using makeicefiles.m for each resolution separately.

While there is theoretical efficiency to be gained for ~90% of dates (outside 2007-2008 range), the added complexity of a "smart" hybrid system is not worth the risk of incorrect results. The current approach is:

  • ✅ Always correct
  • ✅ Clear and maintainable
  • ✅ Self-documenting via enhanced logs
  • ✅ Easy to debug when issues arise