NWP atmosphere-ocean coupling on gpu

Use Cases
1

Hi there,

In a project on the Santis machine, we used to compile icon-nwp on both cpu and gpu targets for high resolution coupled runs on hybrid cpu-gpu architecture using the nwp physics. Compiling for gpu was working fine, provided we used vdiff, but the MR Add coupling directives after zeroth time step introduced the following check in mo_nh_stepping.f90, making it fail.

#ifdef _OPENACC
        CALL finish('mo_nh_interface_nwp', 'nwp_couple_ocean is not &
        & available on GPU')
#endif

I tried adding an additional condition, namely IF (atm_phy_nwp_config(1)%inwp_turb /= ivdiff) like it’s done in mo_nh_interface_nwp.90 but I’m then falling in subsequent similar traps calling finish when #ifdef _OPENACC.

Is there still a way to compile the master branch on gpu with nwp physics, vdiff and atmosphere-ocean coupling?

Cheers,
Matthieu

2

Coupling icon-nwp on GPUs to the ocean on CPUs is not really a supported configuration of ICON. Roland Wirth last year implemented the necessary changes to vdiff for us last year, so that we could try it out (Sign in · GitLab). But at MPI-M it was decided not to further pursue this configuration and since it seemed like nobody is using it and there are no tests for it I guess the functionality is not properly maintained. But maybe Roland can tell you what is needed to make it work again. And if it is a configuration you want to continue to use it would be good to also create a test for it to make sure it continues to work.

Cheers,
Claudia

3

To enable coupling for the GPU configuration would require some additional synchronisation between CPU and GPU memory. But since till now there was no demand for it, asserts were added to guard against undefined behaviour.

Cheers,
Moritz

4

Thanks @cfrauen and @k202077 for your feedback. This setup is probably not going to be supported for long on our side either so I guess we’ll stick with the commit working at the moment for the current project.

Cheers,
Matthieu