Skip to content

Instantly share code, notes, and snippets.

@mdsumner
Last active July 14, 2026 22:02
Show Gist options
  • Select an option

  • Save mdsumner/60b5b1affed5400e24da4d59b48d1b7a to your computer and use it in GitHub Desktop.

Select an option

Save mdsumner/60b5b1affed5400e24da4d59b48d1b7a to your computer and use it in GitHub Desktop.

small example using a docker image from https://github.com/hypertidy/gdal-r-ci, I'm using dev because the mdim print output is human readable (by default, json by option)

docker pull ghcr.io/hypertidy/gdal-system:dev
docker run --rm -ti ghcr.io/hypertidy/gdal-system:dev

#── ghcr.io/hypertidy/gdal-system ──
#GDAL 3.14.0dev   PROJ 9.8.1   GEOS 3.15.0beta1
#Py   3.12.3   numpy 2.5.1

#Docs: https://github.com/hypertidy/gdal-r-ci#gdal-system

I have set AADC_KEY and AADC_SEC based on email from AADC.

export AWS_S3_ENDPOINT=transfer.data.aad.gov.au

export BUCKET=aadc-datasets
export PREPATH=voyage_202526VT1
export AWS_VIRTUAL_HOSTING=FALSE

## this lists the bucket at PREPATH
gdal vsi ls /vsis3/$BUCKET/$PREPATH --config AWS_ACCESS_KEY_ID=$AADC_KEY --config AWS_SECRET_ACCESS_KEY=$AADC_SEC


## this lists entire bucket to a local file 
gdal vsi ls -R  /vsis3/$BUCKET/$PREPATH --config AWS_ACCESS_KEY_ID=$AADC_KEY --config AWS_SECRET_ACCESS_KEY=$AADC_SEC > bucklist

## so let's grep for some format we understand

grep nc$ bucklist | tail -n 10
# ---------- 1 unknown unknown    505605145 2025-10-07 02:30 micro_rain_radar/raw/202509/20250915/20250915_190000.nc
# ---------- 1 unknown unknown    505605145 2025-10-07 02:30 micro_rain_radar/raw/202509/20250915/20250915_200000.nc
# ---------- 1 unknown unknown    505605145 2025-10-07 02:31 micro_rain_radar/raw/202509/20250915/20250915_210000.nc
# ---------- 1 unknown unknown    505605145 2025-10-07 02:31 micro_rain_radar/raw/202509/20250915/20250915_220000.nc
# ---------- 1 unknown unknown    505605145 2025-10-07 02:32 micro_rain_radar/raw/202509/20250915/20250915_230000.nc
# ---------- 1 unknown unknown    505605145 2025-10-07 02:32 micro_rain_radar/raw/202509/20250916/20250916_000000.nc
# ---------- 1 unknown unknown    505605145 2025-10-07 02:32 micro_rain_radar/raw/202509/20250916/20250916_010000.nc
# ---------- 1 unknown unknown    505605145 2025-10-07 02:33 micro_rain_radar/raw/202509/20250916/20250916_020000.nc
# ---------- 1 unknown unknown    505605145 2025-10-07 02:33 micro_rain_radar/raw/202509/20250916/20250916_030000.nc
# ---------- 1 unknown unknown     24020728 2025-10-07 07:24 underwaymerger/underway_merger.nc

## finally, open a file and investigate

gdal mdim info /vsis3/$BUCKET/$PREPATH/micro_rain_radar/raw/202509/20250916/20250916_030000.nc --config AWS_ACCESS_KEY_ID=$AADC_KEY --config AWS_SECRET_ACCESS_KEY=$AADC_SEC
# 
# 
# Driver: netCDF
# 
# Structural metadata:
#   NC_FORMAT  NETCDF4
# 
# Dimensions:
#   Name (path)      Size    Type    Direction
# -------------------  ----  --------  ---------
#   /time                3600  TEMPORAL
# /range                128  VERTICAL
# /sweep                  1
# /string_length        128
# /n_spectra            128
# /spectrum_n_samples    64  VERTICAL
# 
# Coordinates (indexing variables):
#   Name (path)  Dimension   Type                   Unit
# -----------  ---------  -------  ----------------------------------
#   /range       (range)    Float32  meters
# /time        (time)     Float64  seconds since 1970-01-01T00:00:00Z
# /azimuth     (time)     Float32  degrees
# 
# Data variables:
#   Name (path)         Type     Unit         Shape        Chunk size
# ----------------------  -------  -------  ---------------  ------------
# ...  

Now we could concat multiple files to MDIM VRT, or extract data for particular times, open in xarray, etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment