From the PO.DAAC Cookbook, to access the GitHub version of the notebook, follow this link.

SWOT Quality Flag Demonstration

Summary:

SWOT standard data products such as L2_LR_SSH include quality flags that communicate detailed information about the quality of the observation reported. SWOT quality flags are paired one-to-one with the core science data variables in each netcdf file. They’re identified by the qual suffix, e.g.

  • Science data variable: <variable name>

  • Quality flag variable: <variable name>_qual

Depending on the data product though, the quality flag may not end with qual, you may need to alter the code for your particular dataset. See the summary chart below.

Quality flags have different syntax depending on which data product is being used. If a measurement has a quality flag, it can also have a bit flag that provides the detail of why the quality flags are set as they are (see Product Description Documents (PDDs) for specific value meanings). In addition to the ’_qual’ or ’_q’ indications, ’_flag’ or ’_f’ (e.g., ‘ice_flag’) may be used in each data product to raise different flags with unique values and meanings. See specific PDDs for more information.

SWOT Product Quality Flag Identifier Values and Meanings
L2_HR_RiverSP
L2_HR_RiverAvg		 Var + ’_q’

Overall Quality Variables:
‘reach_q’ or ‘node_q’

Bitwise:
Var + ’_q_b’		 0 = good
1 = suspect - may have large errors
2 = degraded - likely to have large errors
3 = bad - may be nonsensical and should be ignored

For discharge parameters: (e.g., ‘dschg_c_q’)
0 = valid
1 = questionable
2 = invalid
L2_HR_LakeSP
L2_HR_LakeAvg		 Overall quality Variable: ‘quality_f’ 0 = good
1 = bad
L2_HR_Raster Var + ’_qual’

Ex: ‘wse_qual’

Bitwise:
Var + ’_qual_bitwise’		 0 = good
1 = suspect - may have large errors
2 = degraded - likely to have large errors
3 = bad - may be nonsensical and should be ignored
L2_NALT_GDR
L2_NALT_IGDR
L2_NALT_OGDR
L2_RAD_GDR
L2_RAD_IGDR
L2_RAD_OGDR
L2_FPDEM		 Var + ’_qual’
Ex: ‘rad_water_vapor_qual’		 0 = good
1 = bad
L2_LR_SSH
L2_HR_PIXC
L1B_HR_SLC
L1B_LR_INTF		 Var + ’_qual’ Varies, see PDDs

Quality flags are stored as arrays of integers. Each integer decomposes into a series of bits that individually describe data quality according to various criteria specific to each science data product from SWOT. This example will demonstrate how to interrogate the quality information about the ssha_karin_2 variable in one L2 LR SSH product/file.

Requirement:

This tutorial can be run on a local machine or the AWS cloud, if AWS cloud, it must be an instance running in us-west-2: NASA Earthdata Cloud data in https or S3 can be directly accessed via earthaccess python library.

Learning Objectives:

  • Demonstrate how to interrogate the quality information for the ssha_karin_2 variable in one L2 LR SSH product/files, though this process can be used for other SWOT data with associated 8-bit quality flags.
  • Filter data via quality flags and visualize

Notebook Author: Jack McNelis, NASA PO.DAAC (April 2024) || Other Contributors: Cassie Nickles (NASA PO.DAAC)

Libraries Needed:

import numpy as np
import pandas as pd
import xarray as xr
import earthaccess
import matplotlib.pyplot as plt

Earthdata Login

An Earthdata Login account is required to access data, as well as discover restricted data, from the NASA Earthdata system. Thus, to access NASA data, you need Earthdata Login. Please visit https://urs.earthdata.nasa.gov to register and manage your Earthdata Login account. This account is free to create and only takes a moment to set up. We use earthaccess to authenticate your login credentials below.

auth = earthaccess.login()

Use earthaccess to search SWOT data

Here we use the SSH shortname, SWOT_L2_LR_SSH_Expert_2.0 from Version C of the data.

ssh_results = earthaccess.search_data(short_name = 'SWOT_L2_LR_SSH_Expert_2.0', 
                                      temporal = ("2024-03-22 00:00:00", "2024-03-22 23:59:59"))#,
                                     #granule_name = '*_Expert_012_455*') #if you know what cycle and pass you want, you can be more specific
Granules found: 29

If accessing via Local Machine

# If you are on a local machine, download the first file to your local computer
earthaccess.download(ssh_results[0], "./data_downloads/SWOT_files")
 Getting 1 granules, approx download size: 0.03 GB
['data_downloads\\SWOT_files\\SWOT_L2_LR_SSH_Expert_012_428_20240321T235733_20240322T004901_PIC0_01.nc']
# Open via xarray
ds = xr.open_mfdataset("./data_downloads/SWOT_files/SWOT_L2_LR_SSH_Expert*.nc")#, mask_and_scale=False)
ds
<xarray.Dataset>
Dimensions:                                (num_lines: 9866, num_pixels: 69,
                                            num_sides: 2)
Coordinates:
    latitude                               (num_lines, num_pixels) float64 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    longitude                              (num_lines, num_pixels) float64 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    latitude_nadir                         (num_lines) float64 dask.array<chunksize=(9866,), meta=np.ndarray>
    longitude_nadir                        (num_lines) float64 dask.array<chunksize=(9866,), meta=np.ndarray>
Dimensions without coordinates: num_lines, num_pixels, num_sides
Data variables: (12/98)
    time                                   (num_lines) datetime64[ns] dask.array<chunksize=(9866,), meta=np.ndarray>
    time_tai                               (num_lines) datetime64[ns] dask.array<chunksize=(9866,), meta=np.ndarray>
    ssh_karin                              (num_lines, num_pixels) float64 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    ssh_karin_qual                         (num_lines, num_pixels) float64 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    ssh_karin_uncert                       (num_lines, num_pixels) float32 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    ssha_karin                             (num_lines, num_pixels) float64 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    ...                                     ...
    swh_ssb_cor_source                     (num_lines, num_pixels) float32 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    swh_ssb_cor_source_2                   (num_lines, num_pixels) float32 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    wind_speed_ssb_cor_source              (num_lines, num_pixels) float32 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    wind_speed_ssb_cor_source_2            (num_lines, num_pixels) float32 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    volumetric_correlation                 (num_lines, num_pixels) float32 dask.array<chunksize=(9866, 69), meta=np.ndarray>
    volumetric_correlation_uncert          (num_lines, num_pixels) float32 dask.array<chunksize=(9866, 69), meta=np.ndarray>
Attributes: (12/62)
    Conventions:                                   CF-1.7
    title:                                         Level 2 Low Rate Sea Surfa...
    institution:                                   CNES
    source:                                        Ka-band radar interferometer
    history:                                       2024-03-24T22:39:40Z : Cre...
    platform:                                      SWOT
    ...                                            ...
    ellipsoid_semi_major_axis:                     6378137.0
    ellipsoid_flattening:                          0.0033528106647474805
    good_ocean_data_percent:                       67.62656103685167
    ssha_variance:                                 0.7787372594877449
    references:                                    V1.2.1
    equator_longitude:                             52.58

If accessing via AWS Cloud

#If you are on the cloud, skip the download step above and access the file via xarray directly
ds = xr.open_mfdataset(earthaccess.open([ssh_results[0]]))#, mask_and_scale=False)
ds

List all L2 LR SSH quality flag variables in a new python dictionary:

quality_variables = {}

for name, variable in ds.variables.items():
    if name.endswith("qual"):
        quality_variables[name] = None

sorted(list(quality_variables))
['height_cor_xover_qual',
 'orbit_qual',
 'sig0_karin_2_qual',
 'sig0_karin_qual',
 'ssh_karin_2_qual',
 'ssh_karin_qual',
 'ssha_karin_2_qual',
 'ssha_karin_qual',
 'swh_karin_qual',
 'wind_speed_karin_2_qual',
 'wind_speed_karin_qual']

Depending on the data product, the quality flag may not end with qual, you may need to alter the above code for your particular dataset. See the summary chart in the introduction.

For this example, we will use the ssha_karin_2 variable from the L2_LR_SSH dataset.

variable_name = "ssha_karin_2"

quality_variable_name = f"{variable_name}_qual"

print(variable_name, quality_variable_name)
ssha_karin_2 ssha_karin_2_qual

Select the ssha_karin_2_qual variable and print its header information:

qual = ds.variables[quality_variable_name].copy() 

print(qual)
<xarray.Variable (num_lines: 9866, num_pixels: 69)>
dask.array<copy, shape=(9866, 69), dtype=float64, chunksize=(9866, 69), chunktype=numpy.ndarray>
Attributes:
    long_name:      sea surface height anomaly quality flag
    standard_name:  status_flag
    flag_meanings:  suspect_large_ssh_delta suspect_large_ssh_std suspect_lar...
    flag_masks:     [         1          2          4          8         16  ...
    valid_min:      0
    valid_max:      3876569055
    comment:        Quality flag for the SSHA from KaRIn in the ssha_karin_2 ...

The flag_meanings and flag_masks attributes define the quality fields/values assigned to each bit (and/or sequence of bits) comprising the integers in the quality variable array. (Refer to the CF Conventions documentation and SWOT PDDs for additional info about standard attributes stored in each netcdf file from SWOT.) The flag meanings/masks attributes are typically space-delimited strings.

Make a simple lookup table for the ssha_karin_2_qual variable:

atts = dict(qual.attrs.copy())

flags = pd.DataFrame({'flag_meanings': atts.get("flag_meanings").split(" "), 
                      'flag_masks': atts.get("flag_masks"), }) \
          .set_index("flag_meanings")

display(flags)
flag_masks
flag_meanings
suspect_large_ssh_delta 1
suspect_large_ssh_std 2
suspect_large_ssh_window_std 4
suspect_beam_used 8
suspect_less_than_nine_beams 16
suspect_ssb_out_of_range 64
suspect_pixel_used 128
suspect_num_pt_avg 256
suspect_karin_telem 512
suspect_orbit_control 1024
suspect_sc_event_flag 2048
suspect_tvp_qual 4096
suspect_volumetric_corr 8192
degraded_ssb_not_computable 32768
degraded_media_delays_missing 65536
degraded_beam_used 131072
degraded_large_attitude 262144
degraded_karin_ifft_overflow 524288
bad_karin_telem 16777216
bad_very_large_attitude 33554432
bad_tide_corrections_missing 67108864
bad_outside_of_range 536870912
degraded 1073741824
bad_not_usable 2147483648

Now print the list of unique integer values in the ssha_karin_2_qual variable array. Each value is a sum of an assortment of the numbers above we made in our lookup table. We will decode the quality bits for each one in a new column of our lookup table.

unq_ints_qual = sorted(list(set(qual.load().data.astype(np.int64).flatten().tolist())))

print(f"\n# Counted {len(unq_ints_qual)} unique integers in '{quality_variable_name}' variable array:\n\n{unq_ints_qual}\n")

# Counted 385 unique integers in 'ssha_karin_2_qual' variable array:

[0, 1, 4, 5, 128, 129, 130, 132, 133, 134, 135, 144, 145, 146, 148, 149, 150, 256, 257, 258, 261, 384, 385, 386, 388, 389, 390, 391, 400, 401, 402, 404, 405, 406, 407, 2184, 2185, 2186, 2188, 2189, 2190, 2191, 2200, 2201, 2204, 2205, 2206, 2207, 2440, 2441, 2442, 2444, 2445, 2446, 2447, 2456, 2457, 2458, 2460, 2461, 2462, 2463, 8328, 8329, 8332, 8333, 8344, 8345, 8346, 8348, 8349, 8350, 8584, 8585, 8586, 8588, 8589, 8590, 8591, 8600, 8601, 8602, 8604, 8605, 8606, 8607, 10376, 10380, 10381, 10392, 10393, 10396, 10397, 10398, 10632, 10633, 10634, 10636, 10637, 10638, 10639, 10648, 10649, 10650, 10652, 10653, 10654, 10655, 1073774592, 1073774593, 1073774594, 1073774596, 1073774597, 1073774608, 1073774609, 1073774613, 1073774720, 1073774721, 1073774724, 1073774725, 1073774736, 1073774737, 1073774741, 1073774848, 1073774849, 1073774850, 1073774852, 1073774853, 1073774854, 1073774855, 1073774864, 1073774865, 1073774866, 1073774868, 1073774869, 1073774870, 1073774976, 1073774977, 1073774978, 1073774980, 1073774981, 1073774992, 1073774993, 1073774994, 1073774996, 1073774997, 1073774998, 1073782792, 1073782793, 1073782920, 1073782921, 1073782936, 1073782937, 1073782941, 1073783053, 1073783064, 1073783176, 1073783177, 1073783178, 1073783192, 1073783193, 1073783194, 1073783196, 1073783197, 1073783198, 1073783199, 1074397184, 1074397185, 1074397188, 1074397189, 1074397200, 1074397201, 1074397204, 1074397205, 1074397312, 1074397313, 1074397316, 1074397317, 1074397328, 1074397329, 1074397332, 1074397333, 1074397440, 1074397441, 1074397442, 1074397444, 1074397445, 1074397446, 1074397447, 1074397456, 1074397457, 1074397458, 1074397460, 1074397461, 1074397462, 1074397463, 1074397568, 1074397569, 1074397570, 1074397572, 1074397573, 1074397574, 1074397575, 1074397584, 1074397585, 1074397586, 1074397588, 1074397589, 1074397590, 1074397591, 1074399232, 1074399233, 1074399236, 1074399237, 1074399240, 1074399241, 1074399244, 1074399245, 1074399248, 1074399249, 1074399252, 1074399253, 1074399256, 1074399257, 1074399260, 1074399261, 1074399369, 1074399372, 1074399373, 1074399385, 1074399388, 1074399389, 1074399488, 1074399489, 1074399492, 1074399493, 1074399494, 1074399496, 1074399497, 1074399500, 1074399501, 1074399502, 1074399503, 1074399504, 1074399505, 1074399506, 1074399508, 1074399509, 1074399510, 1074399512, 1074399513, 1074399514, 1074399516, 1074399517, 1074399518, 1074399519, 1074399624, 1074399625, 1074399626, 1074399628, 1074399629, 1074399630, 1074399631, 1074399640, 1074399641, 1074399642, 1074399644, 1074399645, 1074399646, 1074399647, 1074405376, 1074405377, 1074405381, 1074405384, 1074405385, 1074405392, 1074405393, 1074405396, 1074405397, 1074405400, 1074405401, 1074405405, 1074405521, 1074405525, 1074405528, 1074405529, 1074405532, 1074405533, 1074405632, 1074405633, 1074405637, 1074405640, 1074405641, 1074405648, 1074405649, 1074405650, 1074405652, 1074405653, 1074405656, 1074405657, 1074405658, 1074405660, 1074405661, 1074405662, 1074405769, 1074405773, 1074405776, 1074405777, 1074405780, 1074405781, 1074405784, 1074405785, 1074405786, 1074405788, 1074405789, 1074405790, 1074407436, 1074407437, 1074407440, 1074407441, 1074407444, 1074407445, 1074407448, 1074407449, 1074407452, 1074407453, 1074407564, 1074407576, 1074407577, 1074407581, 1074407688, 1074407692, 1074407693, 1074407696, 1074407697, 1074407700, 1074407701, 1074407704, 1074407705, 1074407708, 1074407709, 1074407816, 1074407817, 1074407818, 1074407820, 1074407821, 1074407832, 1074407833, 1074407834, 1074407836, 1074407837, 1074407838, 1074407839, 1074429952, 1074429953, 1074429957, 1074429968, 1074429969, 1074429972, 1074430097, 1074430208, 1074430210, 1074430212, 1074430213, 1074430224, 1074430225, 1074430228, 1074430352, 1074430353, 1074430356, 1074438145, 1074438148, 1074438161, 1074438164, 1074438420, 1074438421, 1074438548, 1074438549, 1074438552, 1074438553, 1074438556, 1074438557, 2684354816, 2684354820, 2751463684]

Create a bitwise True/False mapping for each unique integer in the ssha_karin_2_qual array.

It’s often advantageous to construct a data mask based on the pixels/cells that you’d like to include in your analysis (rather than exclude from it; masking in vs. masking out).

def qual_bits_iter(n):
    while n:
        b = n & (~n+1)
        yield b
        n ^= b

decomposed_qual = {}
for i in sorted(unq_ints_qual):
    if i not in decomposed_qual:
        decomposed_qual[i] = qual_bits_iter(i)

type(decomposed_qual), len(decomposed_qual)
(dict, 385)

Add one new column for each unique integer in the ssha_karin_qual variable. Highlight table cells that contain True, which indicates the quality criteria on the left/index applies to the pixels/cells in ssha_karin wherever the ssha_karin_qual array contains one of the corresponding integers.

new_columns_qual = {}
for i, j in enumerate(list(set(decomposed_qual))):
    new_columns_qual[j] = flags['flag_masks'] \
        .apply(lambda x: x in [b for b in qual_bits_iter(j)])

lookup = flags.join(pd.DataFrame(new_columns_qual).sort_index(axis=1)).copy()

display(lookup.style.apply(lambda x: ['background-color:yellow' if all([i,type(i)==bool]) else '' for i in x]))
  flag_masks 0 1 4 5 128 129 130 132 133 134 135 144 145 146 148 149 150 256 257 258 261 384 385 386 388 389 390 391 400 401 402 404 405 406 407 2184 2185 2186 2188 2189 2190 2191 2200 2201 2204 2205 2206 2207 2440 2441 2442 2444 2445 2446 2447 2456 2457 2458 2460 2461 2462 2463 8328 8329 8332 8333 8344 8345 8346 8348 8349 8350 8584 8585 8586 8588 8589 8590 8591 8600 8601 8602 8604 8605 8606 8607 10376 10380 10381 10392 10393 10396 10397 10398 10632 10633 10634 10636 10637 10638 10639 10648 10649 10650 10652 10653 10654 10655 1073774592 1073774593 1073774594 1073774596 1073774597 1073774608 1073774609 1073774613 1073774720 1073774721 1073774724 1073774725 1073774736 1073774737 1073774741 1073774848 1073774849 1073774850 1073774852 1073774853 1073774854 1073774855 1073774864 1073774865 1073774866 1073774868 1073774869 1073774870 1073774976 1073774977 1073774978 1073774980 1073774981 1073774992 1073774993 1073774994 1073774996 1073774997 1073774998 1073782792 1073782793 1073782920 1073782921 1073782936 1073782937 1073782941 1073783053 1073783064 1073783176 1073783177 1073783178 1073783192 1073783193 1073783194 1073783196 1073783197 1073783198 1073783199 1074397184 1074397185 1074397188 1074397189 1074397200 1074397201 1074397204 1074397205 1074397312 1074397313 1074397316 1074397317 1074397328 1074397329 1074397332 1074397333 1074397440 1074397441 1074397442 1074397444 1074397445 1074397446 1074397447 1074397456 1074397457 1074397458 1074397460 1074397461 1074397462 1074397463 1074397568 1074397569 1074397570 1074397572 1074397573 1074397574 1074397575 1074397584 1074397585 1074397586 1074397588 1074397589 1074397590 1074397591 1074399232 1074399233 1074399236 1074399237 1074399240 1074399241 1074399244 1074399245 1074399248 1074399249 1074399252 1074399253 1074399256 1074399257 1074399260 1074399261 1074399369 1074399372 1074399373 1074399385 1074399388 1074399389 1074399488 1074399489 1074399492 1074399493 1074399494 1074399496 1074399497 1074399500 1074399501 1074399502 1074399503 1074399504 1074399505 1074399506 1074399508 1074399509 1074399510 1074399512 1074399513 1074399514 1074399516 1074399517 1074399518 1074399519 1074399624 1074399625 1074399626 1074399628 1074399629 1074399630 1074399631 1074399640 1074399641 1074399642 1074399644 1074399645 1074399646 1074399647 1074405376 1074405377 1074405381 1074405384 1074405385 1074405392 1074405393 1074405396 1074405397 1074405400 1074405401 1074405405 1074405521 1074405525 1074405528 1074405529 1074405532 1074405533 1074405632 1074405633 1074405637 1074405640 1074405641 1074405648 1074405649 1074405650 1074405652 1074405653 1074405656 1074405657 1074405658 1074405660 1074405661 1074405662 1074405769 1074405773 1074405776 1074405777 1074405780 1074405781 1074405784 1074405785 1074405786 1074405788 1074405789 1074405790 1074407436 1074407437 1074407440 1074407441 1074407444 1074407445 1074407448 1074407449 1074407452 1074407453 1074407564 1074407576 1074407577 1074407581 1074407688 1074407692 1074407693 1074407696 1074407697 1074407700 1074407701 1074407704 1074407705 1074407708 1074407709 1074407816 1074407817 1074407818 1074407820 1074407821 1074407832 1074407833 1074407834 1074407836 1074407837 1074407838 1074407839 1074429952 1074429953 1074429957 1074429968 1074429969 1074429972 1074430097 1074430208 1074430210 1074430212 1074430213 1074430224 1074430225 1074430228 1074430352 1074430353 1074430356 1074438145 1074438148 1074438161 1074438164 1074438420 1074438421 1074438548 1074438549 1074438552 1074438553 1074438556 1074438557 2684354816 2684354820 2751463684
flag_meanings                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    
suspect_large_ssh_delta 1 False True False True False True False False True False True False True False False True False False True False True False True False False True False True False True False False True False True False True False False True False True False True False True False True False True False False True False True False True False False True False True False True False True False True False False True False False True False False True False True False True False False True False True False False True False True False True False False True False False True False True False True False False True False True False True False False True False True True False True False True False True True False True False False True False True False True False False True False False True False False True False True False False True False False True False True False True True True False False True False False True False False True False True False True False True False True False True False True False True False True False True False True False False True False True False True False False True False True False True False False True False True False True False False True False True False True False True False True False True False True False True False True False True True False True True False True False True False True False False True False True False True False True False False True False False True False False True False True False True False False True False True False True False False True False True False True True False True False True False True False True True True True False True False True False True True False True False True False False True False True False False True False True True False True False True False True False False True False False True False True False True False True False True False False True True False False True False True False True False True False True False True False False True False True False False True False True False True True False True False True False False False True False True False False True False True False True False False True False True False True False True False False False
suspect_large_ssh_std 2 False False False False False False True False False True True False False True False False True False False True False False False True False False True True False False True False False True True False False True False False True True False False False False True True False False True False False True True False False True False False True True False False False False False False True False False True False False True False False True True False False True False False True True False False False False False False False True False False True False False True True False False True False False True True False False True False False False False False False False False False False False False False False True False False True True False False True False False True False False True False False False False True False False True False False False False False False False False False False False True False False True False False True True False False False False False False False False False False False False False False False False False False True False False True True False False True False False True True False False True False False True True False False True False False True True False False False False False False False False False False False False False False False False False False False False False False False False False False True False False False False True True False False True False False True False False True False False True True False False True False False True True False False True False False True True False False False False False False False False False False False False False False False False False False False False False False False False False True False False False False True False False True False False False False False False False False True False False True False False False False False False False False False False False False False False False False False False False False False False False False False False False True False False False False True False False True True False False False False False False False False True False False False False False False False False False False False False False False False False False False False False False False False
suspect_large_ssh_window_std 4 False False True True False False False True True True True False False False True True True False False False True False False False True True True True False False False True True True True False False False True True True True False False True True True True False False False True True True True False False False True True True True False False True True False False False True True True False False False True True True True False False False True True True True False True True False False True True True False False False True True True True False False False True True True True False False False True True False False True False False True True False False True False False False True True True True False False False True True True False False False True True False False False True True True False False False False False False True True False False False False False False False True True True True False False True True False False True True False False True True False False True True False False False True True True True False False False True True True True False False False True True True True False False False True True True True False False True True False False True True False False True True False False True True False True True False True True False False True True True False False True True True True False False False True True True False False False True True True True False False False True True True True False False False True True True True False False True False False False False True True False False True False True False False True True False False True False False False False False True True False False False True True True False True False False True True False False False True True True True True False False True True False False True True True False False True False True True False False True True False False True True False False False True True False False False True True True True False False True False False True False False False True True False False True False False True False True False True True True True True False False True True False True True
suspect_beam_used 8 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True False False False False True True True True True True True True True True False False False False False True True True True True True False False False False False False True True True True True True True True True True True True True True True True True True True True True False False False True True False False False False True True True False False True True True True False False False True True False False False False False True True True True True True True True False False False False True True True True True True True True False False False False True True True True True True True True True True True False False False False True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False False True True True True False False False
suspect_less_than_nine_beams 16 False False False False False False False False False False False True True True True True True False False False False False False False False False False False True True True True True True True False False False False False False False True True True True True True False False False False False False False True True True True True True True False False False False True True True True True True False False False False False False False True True True True True True True False False False True True True True True False False False False False False False True True True True True True True False False False False False True True True False False False False True True True False False False False False False False True True True True True True False False False False False True True True True True True False False False False True True True False True False False False True True True True True True True False False False False True True True True False False False False True True True True False False False False False False False True True True True True True True False False False False False False False True True True True True True True False False False False False False False False True True True True True True True True False False False True True True False False False False False False False False False False False True True True True True True True True True True True True True False False False False False False False True True True True True True True False False False False False True True True True True True True True True True True True True False False False False False True True True True True True True True True True True False False True True True True True True True True True True False False True True True True True True True True False True True True False False False True True True True True True True True False False False False False True True True True True True True False False False True True True True False False False False True True True True True True False False True True True True True True True True True True False False False
suspect_ssb_out_of_range 64 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False
suspect_pixel_used 128 False False False False True True True True True True True True True True True True True False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False True True True True True True True False False False False False False False False False False False False False True True True True True True True True True True True False False True True True True True False False True True True True True True True True True True False False False False False False False False True True True True True True True True False False False False False False False False False False False False False False True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True False False False False False False False False False False False False True True True True True True False False False False False False False False False False False False False False False False True True True True True True True True True True True True False False False False False False False False False False True True True True False False False False False False False False False False False True True True True True True True True True True True True False False False False False False True False False False False False False False True True True False False False False False False True True True True True True False False False
suspect_num_pt_avg 256 False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False True True True True True True True True True True True True True True False False False False False False False False False False True True True True True True True True True True True True True True False False False False False False False False True True True True True True True True True True True True True True False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False True True True True True True True True True True True True False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False True True True True True True True True True True False False False False True True True True True True True True True True True
suspect_karin_telem 512 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False
suspect_orbit_control 1024 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False
suspect_sc_event_flag 2048 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False
suspect_tvp_qual 4096 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False
suspect_volumetric_corr 8192 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False True True True True True True True True True True True True False False False
degraded_ssb_not_computable 32768 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False
degraded_media_delays_missing 65536 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False
degraded_beam_used 131072 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False
degraded_large_attitude 262144 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False
degraded_karin_ifft_overflow 524288 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False
bad_karin_telem 16777216 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False
bad_very_large_attitude 33554432 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False
bad_tide_corrections_missing 67108864 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True
bad_outside_of_range 536870912 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True
degraded 1073741824 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True True False False False
bad_not_usable 2147483648 False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False False True True True

Note the distribution of the highlighted cells in the table above. (And see my earlier point about integer size and the number significant bits.)

Finally, create a mask based on some criteria selected from the table above.

For a simple example, let’s create a mask that excludes pixels that have quality flags prefixed with “bad” or “degraded”, which includes all the flag meanings in this list:

flags_to_mask = sorted([i for i in lookup.index if i.startswith(('bad', 'degraded', 'suspect', ))])

display(flags_to_mask)
['bad_karin_telem',
 'bad_not_usable',
 'bad_outside_of_range',
 'bad_tide_corrections_missing',
 'bad_very_large_attitude',
 'degraded',
 'degraded_beam_used',
 'degraded_karin_ifft_overflow',
 'degraded_large_attitude',
 'degraded_media_delays_missing',
 'degraded_ssb_not_computable',
 'suspect_beam_used',
 'suspect_karin_telem',
 'suspect_large_ssh_delta',
 'suspect_large_ssh_std',
 'suspect_large_ssh_window_std',
 'suspect_less_than_nine_beams',
 'suspect_num_pt_avg',
 'suspect_orbit_control',
 'suspect_pixel_used',
 'suspect_sc_event_flag',
 'suspect_ssb_out_of_range',
 'suspect_tvp_qual',
 'suspect_volumetric_corr']
Now select the columns that contain True in any of the rows indexed by the list of flags_to_mask:
tmp = lookup.copy()  # Make a copy of the table.

del tmp['flag_masks']  # Remove the 'flag_masks' column.

indices = tmp.loc[flags_to_mask].apply(lambda x: x.any()).copy()  # Get a boolean array of integers that comprise the mask.

integers = sorted(tmp.columns[indices].astype(np.int64).tolist())  # Get the columns/integers with one or more flag set to True.

print(len(integers))
384
Now get the 2d boolean index where ssha_karin_2_qual array contains any of the integers in the list.
ssha_karin_2_mask = np.isin(qual.load().data, integers)

print(f"Masked: {ssha_karin_2_mask.sum()} / {ssha_karin_2_mask.size}\n")

display(ssha_karin_2_mask)
Masked: 428494 / 680754

array([[ True,  True,  True, ...,  True,  True,  True],
       [ True,  True,  True, ...,  True,  True,  True],
       [ True,  True,  True, ...,  True,  True,  True],
       ...,
       [ True,  True,  True, ...,  True,  True,  True],
       [ True,  True,  True, ...,  True,  True,  True],
       [ True,  True,  True, ...,  True,  True,  True]])
Convert the boolean mask array to the equivalent 0,1 integer array and add it to the xarray Dataset as a new variable:
out = ds.copy()

out['mask'] = out[variable_name].copy()
out['mask'].data = np.where(out.ancillary_surface_classification_flag==0, ssha_karin_2_mask.astype(np.int64), 0)
out['mask'].attrs = {}

#correct for cross over calibration for the ssha variable so it plots correctly
out['ssha_karin_2_corrected'] = out.ssha_karin_2 + out.height_cor_xover

Plot

#Plot
SLICE = slice(5000, 5500)

fig, axes = plt.subplots(1, 3, figsize=(14, 6), sharey=True) #sharex=True, 

LVLS = sorted(set(list(qual.load().isel(num_lines=SLICE).data.flatten())))

out.ssha_karin_2_qual.isel(num_lines=SLICE).astype(int) \
    .plot(ax=axes[0], levels=LVLS, add_colorbar=True, cbar_kwargs={'orientation': 'horizontal', 'spacing': 'uniform', 'ticks': LVLS})

out.mask.isel(num_lines=SLICE) \
    .plot(ax=axes[1], colors=['black'], add_colorbar=True, cbar_kwargs={'orientation': 'horizontal', 'ticks': [1], }, levels=[0, 1] )

ssha_karin_2_masked = out.ssha_karin_2_corrected \
    .where(out.mask==0, np.nan).copy()

ssha_karin_2_masked.isel(num_lines=SLICE) \
    .plot(ax=axes[2], add_colorbar=True, cbar_kwargs={'orientation': 'horizontal'}, )

We’ve applied the quality mask to the ssha_karin_2 variable and visualized the result!