umwm

University of Miami Wave Model (UMWM)

A tiny, ultrafast, parallel spectral ocean wave model for CPUs and GPUs.

This is the reference implementation of UMWM, described by Donelan et al. (2012), with later improvements and bug fixes. UMWM solves the wave energy balance equation on a curvilinear grid. It has been used to simulate:

UMWM was initially designed with a primary goal of accurate and conservative momentum coupling with atmosphere and ocean circulation models. That design goal remains a priority. Further, UMWM takes a highly simplified approach to nonlinear downshifting of wave energy; this allows it to run signinficantly faster than other spectral wave models.

Getting started

Get the code

git clone https://github.com/umwm/umwm

System dependencies

Building UMWM

Type make to build the serial model and auxiliary tools:

make

The default serial compiler is gfortran. Select another compiler by setting FC:

make FC=ifx
make FC=flang

The default FCFLAGS are aggressive GNU-style optimization flags: -Ofast -march=native -ffast-math -funroll-loops -Wall, or -O3 -fast when FC is nvfortran or an MPI wrapper of it. Override FCFLAGS when using another compiler that needs different optimization options:

make FC=ifx FCFLAGS="-O3 -xHost -ipo -fp-model fast=2"

Build with MPI by setting MPI=yes. This defines -DMPI and defaults to the mpif90 wrapper unless FC is set explicitly:

make MPI=yes
make MPI=yes FC=mpiifx

NetCDF Fortran flags are discovered with nf-config by default. If you have multiple NetCDF installs, point to the matching nf-config for your compiler:

make NF_CONFIG=/path/to/nf-config

As a fallback, set NETCDF to an install prefix with include and lib subdirectories, or provide the flags manually:

make NETCDF=/path/to/netcdf
make NETCDF_FFLAGS="-I/path/include" NETCDF_FLIBS="-L/path/lib -lnetcdff -lnetcdf"

Use make print-config to see the resolved compiler and NetCDF settings. make shared builds the shared library src/shared/libumwm.so with the C API used by the Python package. Executable umwm will be built in the top-level directory and auxiliary tool executables will be built in tools/. PDF documentation can be built separately with make docs.

Building for NVIDIA GPUs

Build for NVIDIA GPUs by setting GPU=yes. This needs the NVIDIA HPC SDK, for nvfortran with OpenACC and its CUDA-aware HPC-X Open MPI, and a netCDF-Fortran built with nvfortran:

module load nvhpc-hpcx-cuda13
unset FC
make GPU=yes MPI=yes NF_CONFIG=/path/to/nvfortran/netcdf-fortran/bin/nf-config

Load the HPC-X module of the SDK rather than nvhpc, so that MPI uses the UCX of the SDK, which transfers GPU memory, and unset the FC that the module exports, which would take precedence over mpif90. GPU=yes selects nvfortran (or mpif90 with MPI=yes) with the default FCFLAGS -O3 -fast, and adds the OpenACC flags -acc=gpu -gpu=$(GPU_ARCH),mem:unified also when FCFLAGS is set. GPU_ARCH is the compute capability of the GPU, cc120 (Blackwell RTX PRO) by default. The kernels of the time step, in src/umwm_domain_acc.f90, are compiled with ACC_KERNEL_FLAGS, which inline the functions of the spectral bins and limit the registers of a thread.

In the GPU build, the source time step, the forcing interpolation, the diagnostics, the ocean stress, the Stokes drift, and most of the initialization run on the GPU, and the input and output on the CPU. The fields are in unified memory, which the CPU and the GPU share, as the code has no OpenACC data directives; this needs a system where the GPU can address the memory of the CPU, as with heterogeneous memory management (HMM), for which nvidia-smi -q shows the addressing mode HMM. It supports at most 128 frequencies and 128 directions. Its results differ from those of the CPU build at the level of single-precision rounding, because the GPU raises to the dissipation power in single precision, where the CPU uses double precision, and sums over the spectrum in another order. Run it on one GPU as in serial, selecting the GPU with CUDA_VISIBLE_DEVICES:

CUDA_VISIBLE_DEVICES=0 ./umwm

The GPU build is fast for large grids, of the order of 100,000 sea cells and more; small grids, such as most example cases, use a small part of the GPU.

With MPI, the GPU build runs on several GPUs, with the same results as on one. The processes of a node share its visible GPUs, one process per GPU, or blocks of consecutive ranks if there are more processes than GPUs, and the halo is exchanged while the cells inside the tiles are integrated. Map the processes to the NUMA domains in rank order, so that each runs next to its GPU on nodes where the GPUs are numbered by NUMA domain; for example, on a node with four GPUs, two on each of two NUMA domains:

mpirun -np 4 --map-by ppr:2:numa --bind-to numa ./umwm

The MPI library should be CUDA-aware, as the HPC-X of the SDK, which copies the halo from and to the GPU; through other libraries, the halo pages migrate to the CPU and back at each time step. On two GPUs, a time step of a grid of 360,000 sea cells or more takes half the time of one GPU.

Running tests

Run the test suite from the top-level directory:

make test

The tests generate their forcing with Python (requirements.txt); set PYTHON to use a particular interpreter, for example make test PYTHON="uv run python" with the environment of the Python package.

Building with fpm

Fortran Package Manager (fpm) 0.13 or newer can build the MPI-enabled model and run its test suite. The active compiler must have matching MPI and netCDF Fortran installations discoverable through their compiler wrappers and pkg-config files.

With GNU Fortran, OpenMPI, and a system netCDF installation:

fpm build --compiler gfortran
fpm test --compiler gfortran --runner ./tests/fpm-mpi-test-runner.sh

With Intel Fortran, Intel MPI, and a netCDF-Fortran installation built with ifx, first initialize oneAPI and expose that netCDF installation:

source ~/intel/oneapi/setvars.sh
export PKG_CONFIG_PATH=/path/to/intel-netcdf/lib/pkgconfig:$PKG_CONFIG_PATH
export LD_LIBRARY_PATH=/path/to/intel-netcdf/lib:$LD_LIBRARY_PATH
fpm build --compiler ifx --build-dir build-ifx
fpm test --compiler ifx --build-dir build-ifx \
  --runner ./tests/fpm-mpi-test-runner.sh

The MPI test runner launches one process by default; set FPM_TEST_PROCESSES to override that count. It prepares the generated Rankine-vortex forcing fixture and runs tests from the tests directory, preserving the paths used by the Make suite.

Running UMWM

Running in serial mode:

./umwm

Running in parallel, for example on 16 cores:

mpiexec -n 16 ./umwm

The model runs in the current directory, reading namelists/main.nml (or the namelist given as the argument, ./umwm path/to/main.nml) and writing its output to output/. ./umwm --help lists the options.

You can read the full technical reference doc here.

Example cases

The examples directory holds a gallery of idealized cases with constant wind: fetch-limited and duration-limited growth, the fetches of JONSWAP, a laboratory wave tank, lakes on Titan and Mars with their planetary parameters, and waves over a seamount and through a field of ocean eddies, whose bathymetry and currents come with the cases. Select one with --case:

./umwm --list-cases
./umwm --case jonswap
./umwm --case titan-ontario-lacus

or with the Python command-line tool, which runs a case in a run directory named after it:

umwm cases
umwm run --case mars-jezero-lake

Running from ERA5

UMWM starts from, and is forced by, the ERA5 reanalysis on a regional or global longitude-latitude grid, with no other program to run: the namelist names the grid and the data source, and the model prepares its missing input files from ERA5 before it runs, and again those it prepared for another grid, such as for another case in the same directory.

&GRID
  gridFromFile = .F.
  topoFromFile = .T.               ! the bathymetry of the ERA5 wave model
  lon0 = -98.5, lat0 = 17.5        ! cell (1,1) of a regular lon-lat grid
  dlon = 0.25,  dlat = 0.25        ! its spacing in degrees
  ...
/
&FORCING
  winds = .T., air_density = .T., seaice = .T.
  waves = .T.                      ! wave spectrum from the ERA5 wave partitions
  source = 'era5'                  ! prepare the missing input files from ERA5
  ...
/

The input files are prepared from the analysis-ready, cloud-optimized ERA5 store of Google, read anonymously over HTTPS, on the grid of the namelist: the bathymetry, the hourly wind, air density and sea ice, and the wave partitions, the wind sea and three swells. The model rebuilds its initial spectrum from the partitions, with the significant wave height, mean period, mean direction and directional spread of each, and in a regional domain it propagates the spectrum of the ERA5 partitions in from its open boundaries throughout the run.

The umwm executable prepares the files with the Python package (below) installed with the dependencies of ERA5, pip install '.[era5]'; set UMWM_PYTHON to the Python interpreter that has it if that is not python3. The Python command-line tool prepares them as well, when it runs the model or beforehand, for example before a parallel run on nodes without network access:

./umwm --case era5-helene                    # hurricane Helene in the Gulf of Mexico
umwm run --case era5-global-1deg             # the global ocean at 1 degree
umwm init --case era5-helene runs/gulf       # a run directory to edit
umwm forcing era5 runs/gulf                  # prepare its input files
cd runs/gulf && mpiexec -n 8 ../../umwm      # and run it in parallel

The example cases era5-helene, era5-global-1deg and era5-global-0.25deg are templates for other regions and times. DOCS.md describes the spectra of the partitions, the open boundaries, and how to add another data source, such as NCEP GFS with its WAVEWATCH III wave model.

Python interface

The umwm Python package configures and runs the model from Python, with the model state and the output available as NumPy arrays and xarray datasets, and provides the umwm command-line tool. The time integration runs in the compiled Fortran library, so there is no performance cost. Install it from the repository (it builds the library with the same Makefile configuration, for example UMWM_MPI=yes pip install . for MPI or UMWM_GPU=yes pip install . for NVIDIA GPUs), or set up an environment with uv:

pip install .
uv sync && uv run umwm info

import umwm

config = umwm.Config()                        # the defaults of namelists/main.nml
config.domain.stoptimestr = "2026-01-01 12:00:00"
config.forcing_constant.wspd0 = 15.0

with umwm.Domain(config, workdir="runs/case1") as domain:
    domain.run()
    swh = domain.grid_field("swh")            # NumPy array (ny, nx)
    output = domain.output()                  # xarray.Dataset of the output files

umwm init runs/case1 -s domain.stoptimestr="2026-01-01 12:00:00"
umwm run runs/case1
umwm output runs/case1 --stats
umwm run --case jonswap          # an example case, in ./jonswap

See python/README.md for the documentation of the package and the command-line tool.

Papers

Papers about or using UMWM:

Thanks

UMWM development is currently supported by the NSF Award 2543464.

Previously, UMWM was supported by the NSF Award 1745384, the Gulf of Mexico Research Initiative, and the National Oceanographic Partnership Program.

Many thanks as well to the open source contributors:

This site is open source. Improve this page.