L2P container

Converts one sensor-day of PO.DAAC L2P granules into the compact binary bundle the analysis reads.

Overview

The L2P (Level 2 Preprocessed) module processes satellite-derived sea surface temperature observations from multiple sensors for assimilation into the Multi-scale Ultra-high Resolution (MUR) SST analysis system.

Containerized Deployment: This application is containerized using Docker with a multi-stage build process following the same pattern as the IQUAM module - MATLAB compilation in stage 1, MATLAB Runtime-only execution in stage 2.

What This Module Produces

Primary Output: Daily binary files (.bic.gz format) containing satellite SST observations with bias and error metadata

Processing Steps:

1. Read L2P/L3U format NetCDF files with SST, bias, and quality metadata
2. Filter by confidence/proximity thresholds (sensor-specific)
3. Time Translation from reference time to hours relative to analysis day
4. Format Conversion to compact BIC (Binary Input with Confidence) format
5. Compress output files with gzip

Note: Downloads are handled separately in production (cron jobs using podaac-data-subscriber), not by this container.

Output Characteristics:

• Format: Fortran-compatible binary (.bic.gz)
• Size: ~50-500 MB per sensor per day (compressed)
• Content: SST, lon/lat, hour, bias, RMS error, quality flag
• Coverage: Sensor-dependent (swath data for polar orbiters)

Supported Sensors

Microwave sensors (AMSR2R) provide all-weather capability but at coarser resolution. Infrared sensors (MODIS, AVHRR) provide high resolution but are affected by clouds.

Architecture

Production-Matching Interface

This containerized module exactly matches the production nrtMRVA.py calling pattern, using a pure MATLAB wrapper compiled into a standalone executable. The container exposes the same 7-argument l2p2bic() signature used in production:

l2p2bic(sensor, region, indir, bicdir, year, day, rewrite)

Benefits:

• Production parity - Same interface as nrtMRVA.py (lines 350-363)
• No Python - Pure MATLAB Runtime (no integration issues)
• Smaller image - ~2.5 GB (vs ~8 GB with Python)
• Flexible paths - No hardcoded directories
• Separation of concerns - Downloads handled separately (like production cron jobs)

File Structure

l2p/
    ├── Dockerfile              # Multi-stage build: compile → runtime
    ├── requirements.txt       # REMOVED - No longer needed
    └── src/
        ├── l2p_wrapper.m      # NEW: Main wrapper (compiled entry point)
        ├── l2p2bic.m          # Core L2P → BIC conversion
        ├── SensorTable.m      # Sensor-specific configuration
        ├── readL2Pboth.m      # L2P swath reader
        ├── readL3UasL2P.m     # L3U gridded reader
        ├── readL3UasL2Pviirso.m # VIIRS-specific reader
        └── writebic.m         # BIC format writer

    Common dependencies (copied during build):
        common/julian.m        # Date conversion utilities
        common/fortwrite.m     # Fortran binary I/O

Obsolete files (can be removed):

• execute_l2p.py - Replaced by pure MATLAB l2p_wrapper.m
• l2p_template.m - No longer needed (direct function calls)
• requirements.txt - No Python dependencies
• config.json - Not used in production (config embedded in wrapper)

New files added:

• l2p_wrapper.m - Production-matching wrapper with 7-argument interface

Docker Build and Run Instructions

Build the Image

Use build_module.sh from the mur/ directory:

cd /path/to/mur
    ./build_module.sh l2p

It builds the mur-matlab-base:r2024b image first if missing, checks that network.lic exists, builds --platform linux/amd64 with the parent mur/ directory as context, and tags the image mur-l2p:latest — the name the pipeline config expects.

Build Process:

1. Stage 1 (builder): Starts from mur-matlab-base:r2024b (MATLAB R2024b + Compiler), compiles wrapper and dependencies
2. Stage 2 (runtime): Copies compiled executable to minimal MATLAB Runtime container

Build time: ~10-20 minutes (first build), ~2-3 minutes (cached); add ~15-20 minutes the first time the base image is built.

Manual Build (Advanced)

Only when build_module.sh doesn't fit — custom tags, external CI, or debugging the Dockerfile. Build from the l2p directory with parent context:

cd /path/to/mur/l2p
    docker build --platform linux/amd64 -t mur-l2p:latest -f Dockerfile ..

Why parent context? The .. allows access to common/ utilities (julian.m, fortwrite.m) while the local .dockerignore file controls exactly what gets included from the parent directory.

The base image must already exist; build it with ./build_matlab_base.sh from mur/ first. See Manual Builds (Advanced).

Run the Container

Named-args interface — every input is an explicit flag; --granules-manifest replaces the old bind-mounted indir with a JSON manifest listing exactly which granule files this run needs (schema per the input contract, §3: {"files": [{"path": "..."}]}). Each entry's path may be a local filesystem path or an s3:// href — common/bin/localize.sh (sourced by entrypoint.sh) materializes them into a scratch directory before MATLAB runs, so l2p2bic.m's own file-selection logic (which of the sensor's file-pattern candidates to use) runs completely unchanged against that directory:

docker run --rm \
      --shm-size=512M \
      -v /path/to/manifest.json:/data/manifest.json:ro \
      -v /path/to/l2p_files:/path/to/l2p_files:ro \
      -v /path/to/bic_output:/data/output \
      mur-l2p:latest \
      --sensor AMSR2R --region Global --year 2025 --doy 220 --rewrite 0 \
      --granules-manifest /data/manifest.json

The manifest's path entries must point at files reachable from inside the container — bind-mount the same host directory tree the manifest references, or (on MAAP) use s3:// hrefs directly, no mount needed. run_mur_pipeline.py/run_mur_maap.py build this manifest automatically from whatever they've already tracked downloading; you don't need to hand-write one for normal pipeline runs.

Arguments (all required, named flags):

--sensor            Sensor name (AMSR2R, AVMTBG, MODISA, MODIST, or AVMTAG)
    --region             Region name ('Global' - typically always Global)
    --year               4-digit year (e.g., 2025)
    --doy                Day of year (1-366)
    --rewrite            0=skip existing files, 1=overwrite existing files
    --granules-manifest  Path or s3:// href to the granules manifest JSON

Note: Output directory (bicdir) stays a fixed container-internal bind mount (/data/output), unchanged from before — only the input side became explicit.

Expected Output Directory Structure

The module produces compressed BIC files in the specified bicdir:

{bicdir}/
    └── {region}_{sensor}_{year}_{day}.bic.gz

Example with /data/output as bicdir:

/data/output/
    ├── Global_AMSR2R_2025_220.bic.gz
    ├── Global_MODISA_2025_220.bic.gz
    └── Global_MODIST_2025_220.bic.gz

Data Download

IMPORTANT: This container does NOT handle downloads. Like production, downloads are handled separately.

Production Architecture

In the operational MUR system:

1. Cron jobs download L2P files hourly using podaac-data-subscriber (Python tool)
2. nrtMRVA.py calls l2p2bic() to process already-downloaded files
3. Container matches step #2 - processing only

Downloading L2P Files

Use podaac-data-subscriber on your host system or via a separate container:

# Install (requires Python on host)
    pip install podaac-data-subscriber

    # Download data for a specific date
    podaac-data-subscriber \
      -c AMSR2-REMSS-L2P-v8.2 \
      -d /path/to/l2p_files \
      -sd 2025-08-08T00:00:00Z \
      -ed 2025-08-08T23:59:59Z

    # Then build a manifest listing the downloaded files and process with container
    # (run_mur_pipeline.py does this automatically; shown here for manual/debug use)
    docker run --rm \
      -v /path/to/manifest.json:/data/manifest.json:ro \
      -v /path/to/l2p_files:/path/to/l2p_files:ro \
      -v /path/to/bic_output:/data/output \
      mur-l2p:latest \
      --sensor AMSR2R --region Global --year 2025 --doy 220 --rewrite 0 \
      --granules-manifest /data/manifest.json

PO.DAAC Collections by Sensor

NASA Earthdata Authentication

Create a .netrc file for podaac-data-subscriber:

touch ~/.netrc
    chmod 600 ~/.netrc

    cat > ~/.netrc << 'EOF'
    machine urs.earthdata.nasa.gov
        login your-username
        password your-password
    EOF

Get credentials:

• Register: https://urs.earthdata.nasa.gov/
• Approve PO.DAAC: https://urs.earthdata.nasa.gov/approve_app?client_id=BO_n7nTIlMljdvU6kRRB3g

Processing Modes

Rewrite Flag

The --rewrite flag controls whether existing output files are overwritten:

• rewrite=0: Skip processing if output BIC file already exists (default for stable data)
• rewrite=1: Always process, overwrite existing BIC file (use for NRT reprocessing)

Usage in production:

• NRT processing uses rewrite=1 within stability window (2-3 days depending on sensor)
• Historical processing uses rewrite=0 to skip already-processed days

Integration with MUR Workflow

In the MUR processing pipeline:

1. L2P Processing (this module): Download and convert satellite data to BIC format
2. IQUAM Processing: Process in-situ buoy observations
3. Land/Ice Processing: Generate land/ice masks
4. Input Generation: Combine all sources into unified BIQ format
5. MRVA Analysis: Multi-scale variational analysis
6. NetCDF Output: Generate final MUR product

Volume Mounts

Performance Considerations

Resource Requirements

• Memory: 2-4 GB (depends on file count and size)
• CPU: Single-threaded (MATLAB compiled code)
• Disk I/O: Significant (decompression, NetCDF reading, binary writing)
• Network: Only if downloading (10-500 MB/sensor/day)

Optimization Tips

1. Pre-stage data - Download L2P files outside container for better control
2. Use SSD - NetCDF reading is I/O intensive
3. Increase shared memory - Add --shm-size=512M to docker run
4. Parallel processing - Run multiple sensors in parallel (different containers)
5. Batch by sensor - Process all days for one sensor before switching

Typical Runtime

• AMSR2R (microwave): 2-5 minutes (few large swaths)
• MODISA/MODIST (infrared): 10-30 minutes (many granules)
• AVMTBG (AVHRR): 5-15 minutes (moderate granule count)

Sensor-Specific Notes

AMSR2R (Microwave)

• Two collections: Standard (reprocessed) + RT (real-time)
• Wrapper checks both, prioritizes standard
• All-weather capability (cloud-transparent)
• Coarser spatial resolution (~25 km)
• Fewer files per day (~30-50 granules)

MODIS Aqua/Terra (Infrared)

• High spatial resolution (~1 km)
• Cloud contamination (gaps in coverage)
• Many granules per day (200-300 files)
• Longer processing time due to file count
• Terra (MODIST) has 3-day stability vs 2-day for Aqua

AVHRR MetOp-B (Infrared)

• Moderate resolution (~1-4 km)
• Heritage sensor (long data record)
• Similar characteristics to MODIS but fewer granules

Configuration

Sensor Parameters (in SensorTable.m)

The SensorTable.m function provides sensor-specific parameters for each supported sensor:

• File name patterns - For locating L2P files in input directory
• Confidence thresholds - Quality filtering criteria
• Decompression commands - bzip2/gzip handling
• Subdirectory paths - Data organization structure

l2p_wrapper.m Validation

The wrapper validates all inputs before calling l2p2bic():

• Sensor name - Must be one of: AMSR2R, AVMTBG, MODISA, MODIST, AVMTAG
• Year - Must be between 1900 and 2100
• Day - Must be between 1 and 366
• Rewrite flag - Must be 0 or 1
• Directories - Creates output directory if it doesn't exist

Troubleshooting

Build Issues

Problem: Compilation timeout after 10 minutes Solution: Check MATLAB license server connectivity, increase timeout in Dockerfile

Problem: Missing MATLAB toolbox error during build Solution: Verify MATLAB_Compiler is installed (stage 1 of Dockerfile)

Runtime Issues

Problem: "No .nc files found" warning Solution: Check input directory structure matches expected layout (SENSOR/DOY/*.nc)

Problem: "Expected output file not found" Solution: Check logs in /data/logs, verify l2p2bic completed successfully

Problem: Out of memory error Solution: Reduce file count (process fewer days), increase container memory limit

Problem: Slow performance Solution: Use SSD storage, increase --shm-size, check for excessive decompression

Comparison with Legacy Python Driver

Integration with Other MUR Modules

All MUR processing modules follow a consistent containerization pattern:

Future Enhancements

1. Parallel granule processing - Use parfor for multiple NetCDF files
2. Quality metrics - Track observation counts, coverage statistics
3. Automated retry - Handle transient processing failures
4. Multi-day processing - Process date ranges in single container run

References

External Documentation

PO.DAAC Data Access:

• Portal: https://podaac.jpl.nasa.gov/
• Data Subscriber: https://github.com/podaac/data-subscriber
• Earthdata Login: https://urs.earthdata.nasa.gov/

GHRSST L2P Specification:

• Format Guide: https://www.ghrsst.org/ghrsst-data-services/products/
• Data Specification v2.0

Sensor Information:

• AMSR2: https://suzaku.eorc.jaxa.jp/GCOM_W/
• MODIS: https://modis.gsfc.nasa.gov/
• AVHRR: https://www.star.nesdis.noaa.gov/smcd/emb/avhrr/

Citation

If using MUR SST data or this processing system in publications, please cite:

License

This software is part of the MUR SST processing system developed at NASA Jet Propulsion Laboratory.

Architecture decisions

Production vs Container Usage

Production Architecture (nrtMRVA.py)

┌─────────────────┐
    │ Cron Jobs       │
    │ (hourly)        │
    │ - amsr2r.sh     │  Downloads L2P files
    │ - modis_a.sh    │  to /measures_mur/.../
    │ - modis_t.sh    │  using podaac-data-subscriber
    └────────┬────────┘
             │ Downloaded files
             ↓
    ┌─────────────────┐
    │ nrtMRVA.py      │
    │                 │  Creates makebiccmd_*.m script
    │                 │  Calls: l2p2bic(sensor,region,
    │                 │         indir,bicdir,year,day,rewrite)
    └────────┬────────┘
             │
             ↓
    ┌─────────────────┐
    │ l2p2bic.m       │  Reads NetCDF from indir
    │                 │  Writes BIC to bicdir
    │                 │  Uses SensorTable() for config
    └─────────────────┘

Container Options

Option A: Minimal (Production-Like)

# Container only does l2p2bic processing
    # User must download separately

    # 1. Download externally (host or separate container)
    podaac-data-subscriber -c AMSR2-REMSS-L2P-v8.2 ...

    # 2. Process with container
    docker run --rm \
      -v /downloads:/data/input \
      -v /output:/data/output \
      mur-l2p:latest \
      AMSR2R Global /data/input /data/output 2025 220 0

Arguments: sensor region indir bicdir year day rewrite

Option B: Integrated (Current)

# Container does download + processing

    docker run --rm \
      -v ~/.netrc:/root/.netrc:ro \
      -v /data/input:/data/input \
      -v /data/output:/data/output \
      mur-l2p:latest \
      AMSR2R 2025 220 1

Arguments: sensor year doy download_flag Paths: Hardcoded to /data/input and /data/output

Option C: Dual-Mode (Recommended)

# Mode 1: Like production - pass all paths explicitly
    docker run --rm \
      -v /downloads:/mnt/l2p \
      -v /output:/mnt/bic \
      mur-l2p:latest \
      --mode production \
      --sensor AMSR2R \
      --region Global \
      --indir /mnt/l2p/AMSR2R \
      --outdir /mnt/bic/AMSR2R \
      --year 2025 \
      --day 220 \
      --rewrite 0

    # Mode 2: Integrated - download + process
    docker run --rm \
      -v ~/.netrc:/root/.netrc:ro \
      mur-l2p:latest \
      --mode integrated \
      --sensor AMSR2R \
      --year 2025 \
      --day 220 \
      --download

Decision: Go with Simplified Production-Like Interface

Rationale:

1. Production uses l2p2bic(sensor,region,indir,bicdir,year,day,rewrite) - 7 args
2. Downloads handled separately in production (cron jobs)
3. Container should match production calling pattern
4. Downloads can be separate concern (user's choice)

Recommended Container Interface:

docker run --rm \
      -v /downloads:/data/input \
      -v /output:/data/output \
      mur-l2p:latest \
      AMSR2R Global /data/input/AMSR2R /data/output/AMSR2R 2025 220 0

Arguments (match l2p2bic exactly):

1. sensor - AMSR2R, MODISA, MODIST, AVMTBG
2. region - Global
3. indir - Input directory with L2P NetCDF files
4. bicdir - Output directory for BIC files
5. year - 2025
6. day - 220
7. rewrite - 0 or 1

For downloads: User can:

• Use host with podaac-data-subscriber
• Use our provided download function as optional preprocessing step
• Create separate download container
• Use cron jobs (like production)

What About config.json?

Analysis of config.json usage:

{
        "AMSR2R": {
            "collection_name": ["AMSR2-REMSS-L2P-v8.2", "AMSR2-REMSS-L2P_RT-v8.2"],
            "region": "Global",
            "La": 2,
            "Lb": 8,
            "day_range": [12, 1],
            "stable": 2
        }
    }

Fields:

• collection_name - Used for downloads only
• region - Passed as argument to l2p2bic
• La, Lb - Used in MRVA (not l2p2bic)
• day_range - Used in MRVA (not l2p2bic)
• stable - Used for rewrite logic

Conclusion: config.json was only used by execute_l2p.py test wrapper, not by production!

Recommendation: Remove config.json dependency. If downloads needed, embed collection mappings in download function.

Final Recommendation

Primary interface - matches production:

docker run mur-l2p:latest SENSOR REGION INDIR OUTDIR YEAR DAY REWRITE

Optional convenience wrapper for downloads:

# Separate download script/container if needed
    docker run mur-l2p-download:latest SENSOR YEAR DAY /output

Result:

• Clean separation of concerns
• Matches production exactly
• Downloads are optional/separate
• No config.json needed

Performance notes

Executive Summary

Analysis of the L2P MATLAB code reveals several significant performance optimization opportunities:

1. Memory type conversion issues - NetCDF data is read as doubles then converted to single, wasting memory
2. Inefficient loop-based indexing - Using loops instead of vectorized mask operations
3. Redundant find() operations - Multiple calls on the same data
4. Unnecessary memory allocations - Creating temporary arrays with ones(size(inx))

Estimated Performance Improvement: 30-50% reduction in processing time and memory usage

Critical Issues Found

1. NetCDF Data Type Conversion in readL2Pboth.m

Current Code (Lines 56-63):

varid = netcdf.inqVarID(ncid,'sea_surface_temperature');
    sst = netcdf.getVar(ncid,varid);        % Returns double by default
    badpix = netcdf.getAtt(ncid,varid,'_FillValue');
    const = netcdf.getAtt(ncid,varid,'add_offset');
    scale = netcdf.getAtt(ncid,varid,'scale_factor');
    sst=single(sst);                        % Convert entire array double→single
    inx=find(sst(:)==badpix);               % Linear indexing with find()
    sst=sst*single(scale)+single(const);
    if length(inx), sst(inx)=vfv*ones(size(inx)); end;

Problems:

• netcdf.getVar() returns double by default, then converts to single → 2x memory usage temporarily
• find() creates index array instead of using logical mask
• vfv*ones(size(inx)) creates unnecessary temporary array

Optimized Code:

varid = netcdf.inqVarID(ncid,'sea_surface_temperature');
    sst = netcdf.getVar(ncid,varid,'single');  % Use output_type parameter!
    badpix = single(netcdf.getAtt(ncid,varid,'_FillValue'));
    const = single(netcdf.getAtt(ncid,varid,'add_offset'));
    scale = single(netcdf.getAtt(ncid,varid,'scale_factor'));
    mask = (sst == badpix);                    % Logical mask (no find needed)
    sst = sst * scale + const;                 % Vectorized operation
    if any(mask(:)), sst(mask) = single(vfv); end;  % Direct assignment, no ones()

Benefits:

• 100% elimination of double→single conversion using output_type parameter
• netcdf.getVar() supports optional output_type argument: 'single', 'double', 'int16', etc.
• Eliminates find() overhead (index array creation)
• Eliminates ones() temporary array allocation
• More readable and maintainable

Note: MATLAB's netcdf.getVar() syntax is:

data = netcdf.getVar(ncid, varid, output_type)
    data = netcdf.getVar(ncid, varid, start, count, output_type)
    data = netcdf.getVar(ncid, varid, start, count, stride, output_type)

Same pattern appears in:

• Lines 86-91 (sst_dtime)
• Lines 96-104 (bias)
• Lines 114-122 (sigma)
• readL3UasL2P.m lines 16-23, 46-50, 56-63, 69-76
• readL3UasL2Pviirso.m lines 18-34, 77-80, 86-94, 100-108

2. Loop-Based Time Adjustment in l2p2bic.m

Current Code (Lines 101-107):

if n~=length(tt), error('l2p2bic: # time stamps mismatches dim(dt)'); end;
    dt=reshape(dt,m,n);
    for j=1:length(tt),
      dt(:,j)=dt(:,j)+tt(j);  % Loop over columns
    end;

Problem: Uses explicit loop instead of MATLAB's broadcast/bsxfun capabilities

Optimized Code:

if n~=length(tt), error('l2p2bic: # time stamps mismatches dim(dt)'); end;
    dt = reshape(dt, m, n);
    dt = dt + tt(:)';  % Broadcasting (MATLAB R2016b+)
    % OR for older MATLAB:
    % dt = bsxfun(@plus, dt, reshape(tt, 1, []));

Benefits:

• 5-10x faster for large arrays (MATLAB-optimized BLAS operations)
• More readable

3. Inefficient Index Trimming in l2p2bic.m

Current Code (Lines 119-121):

% trim by confidence:
    inx=find(prox(:)>=minConfValue);
    x=x(inx); y=y(inx); dt=dt(inx);
    tmp=tmp(inx); b=b(inx); sigma=sigma(inx); prox=prox(inx);

Problem: find() creates index array, then uses it 7 times for indexing

Optimized Code:

% trim by confidence using logical mask:
    mask = prox(:) >= minConfValue;
    x=x(mask); y=y(mask); dt=dt(mask);
    tmp=tmp(mask); b=b(mask); sigma=sigma(mask); prox=prox(mask);

Benefits:

• Logical indexing is typically faster than linear indexing
• Eliminates memory allocation for index array
• More memory-efficient for large datasets

4. Data Accumulation Pattern in l2p2bic.m

Current Code (Lines 124-126):

% collect:
    lon=[lon;x(:)]; lat=[lat;y(:)]; hour=[hour;dt(:)];
    sst=[sst;tmp(:)]; bias=[bias;b(:)]; rms=[rms;sigma(:)]; flag=[flag;prox(:)];

Problem: Growing arrays in a loop causes repeated memory reallocation

Optimized Code:

Option A: Pre-allocate if size is known

% Before loop (if you can estimate total size):
    estimated_size = length(names) * expected_points_per_file;
    lon = zeros(estimated_size, 1, 'single');
    lat = zeros(estimated_size, 1, 'single');
    % ... etc for all arrays
    current_idx = 1;

    % In loop:
    n_points = length(x);
    idx_range = current_idx:(current_idx + n_points - 1);
    lon(idx_range) = x(:);
    lat(idx_range) = y(:);
    % ... etc
    current_idx = current_idx + n_points;

    % After loop:
    lon = lon(1:current_idx-1);  % Trim to actual size

Option B: Use cell arrays then concatenate once

% Initialize cell arrays before loop:
    lon_cell = cell(length(names), 1);
    lat_cell = cell(length(names), 1);
    % ... etc

    % In loop:
    lon_cell{k} = x(:);
    lat_cell{k} = y(:);
    % ... etc

    % After loop - single concatenation:
    lon = vertcat(lon_cell{:});
    lat = vertcat(lat_cell{:});
    % ... etc

Benefits:

• Eliminates O(n²) behavior from repeated array growth
• Can be 10-100x faster for large loops
• Option B is safer if total size is unknown

5. fortwrite.m Data Type Handling

Current Code (Lines 85-89):

fwrite(file,n,'int32');
    for k=1:length(inxvar),
      fwrite(file,varargin{inxvar(k)},otype{k});
    end;
    fwrite(file,n,'int32');

Issue: fwrite(file, data, 'int16') reads as int16 but writes as double internally, then converts.

Analysis: Actually, checking MATLAB documentation, fwrite does NOT have this issue - it writes in the specified precision directly. However, if you were using fread, the issue you mentioned would apply.

For fread (not currently in code, but for reference):

% Bad - reads as int16, returns double:
    data = fread(f, N, 'int16');

    % Good - reads as int16, returns int16:
    data = fread(f, N, 'int16=>int16');

The fortwrite.m function is already correctly implemented.

Performance Optimization Summary Table

Implementation Priority

High Priority (Implement First)

1. Fix data type conversions in all read functions - Biggest memory impact
2. Fix array growth in l2p2bic.m loop - Can cause severe slowdown with many files

Medium Priority

1. Replace find() with logical masks - Consistent moderate improvement
2. Vectorize time adjustment loop - Simple change, good improvement

Low Priority

1. Code cleanup and standardization

Testing Recommendations

1. Validate output files match exactly (bit-for-bit comparison of .bic.gz files)
2. Profile before/after using MATLAB Profiler:

   profile on
       l2p2bic('MODISA', 'Global', indir, bicdir, 2025, 306, 0)
       profile viewer

3. Memory monitoring:

   clear all
       memory  % Before
       l2p2bic(...)
       memory  % After - check peak usage

Additional Notes

• The readL3UasL2Pviirso.m file already shows optimization awareness (specialized for speed)
• Lines 24-31 use selective reading with o0 and oN parameters - good practice
• Consider applying similar selective reading to other sensors if applicable

Example: Complete Optimized Function for SST Reading

% BEFORE (readL2Pboth.m lines 54-64):
    if nargout>=1,
      varid = netcdf.inqVarID(ncid,'sea_surface_temperature');
      sst = netcdf.getVar(ncid,varid);
      badpix = netcdf.getAtt(ncid,varid,'_FillValue');
      const = netcdf.getAtt(ncid,varid,'add_offset');
      scale = netcdf.getAtt(ncid,varid,'scale_factor');
      sst=single(sst);
      inx=find(sst(:)==badpix);
      sst=sst*single(scale)+single(const);
      if length(inx), sst(inx)=vfv*ones(size(inx)); end;
    end;

    % AFTER (optimized):
    if nargout>=1,
      varid = netcdf.inqVarID(ncid,'sea_surface_temperature');
      sst = single(netcdf.getVar(ncid,varid));
      badpix = single(netcdf.getAtt(ncid,varid,'_FillValue'));
      const = single(netcdf.getAtt(ncid,varid,'add_offset'));
      scale = single(netcdf.getAtt(ncid,varid,'scale_factor'));
      mask = (sst == badpix);
      sst = sst * scale + const;
      if any(mask(:)), sst(mask) = vfv; end;
    end;

Key changes:

• Cast to single immediately on read
• Use logical mask instead of find()
• Use any(mask(:)) instead of length(inx) for cleaner logic
• Direct assignment instead of ones() multiplication