Soil integration Tests fail when run on GPU

[imreddyTeja]

Describe the bug

Running the Soil integration test, specifically the "Phase change source term" test, causes a CUDA error

Phase change source term, FT = Float32: Error During Test at /home/treddy/ClimaLand.jl/test/standalone/Soil/soiltest.jl:644
  Got exception outside of a @test
  InvalidIRError: compiling MethodInstance for ClimaCoreCUDAExt.knl_copyto!(::ClimaCore.DataLayouts.VF{Float32, 200, CUDA.CuDeviceMatrix{Float32, 1}}, ::Base.Broadcast.Broadcasted{ClimaCore.DataLayouts.VFStyle{200, CUDA.CuArray{Float32, N, CUDA.DeviceMemory} where N}, NTuple{5, Base.OneTo{Int64}}, typeof(ClimaCore.RecursiveApply.radd), Tuple{ClimaCore.DataLayouts.VF{Float32, 200, CUDA.CuDeviceMatrix{Float32, 1}}, Base.Broadcast.Broadcasted{ClimaCore.DataLayouts.VFStyle{200, CUDA.CuArray{Float32, N, CUDA.DeviceMemory} where N}, Nothing, typeof(ClimaCore.RecursiveApply.rsub), Tuple{Base.Broadcast.Broadcasted{ClimaCore.DataLayouts.VFStyle{200, CUDA.CuArray{Float32, N, CUDA.DeviceMemory} where N}, Nothing, typeof(phase_change_source), Tuple{ClimaCore.DataLayouts.VF{Float32, 200, CUDA.CuDeviceMatrix{Float32, 1}}, ClimaCore.DataLayouts.VF{Float32, 200, CUDA.CuDeviceMatrix{Float32, 1}}, ClimaCore.DataLayouts.VF{Float32, 200, CUDA.CuDeviceMatrix{Float32, 1}}, Base.Broadcast.Broadcasted{ClimaCore.DataLayouts.VFStyle{200, CUDA.CuArray{Float32, N, CUDA.DeviceMemory} where N}, Nothing, typeof(thermal_time), Tuple{Base.Broadcast.Broadcasted{ClimaCore.DataLayouts.VFStyle{200, CUDA.CuArray{Float32, N, CUDA.DeviceMemory} where N}, Nothing, typeof(volumetric_heat_capacity), Tuple{ClimaCore.DataLayouts.VF{Float32, 200, CUDA.CuDeviceMatrix{Float32, 1}}, ClimaCore.DataLayouts.VF{Float32, 200, CUDA.CuDeviceMatrix{Float32, 1}}, Float32, Tuple{ClimaLand.Parameters.LandParameters{Float32, Thermodynamics.Parameters.ThermodynamicsParameters{Float32}, SurfaceFluxes.Parameters.SurfaceFluxesParameters{Float32, SurfaceFluxes.UniversalFunctions.BusingerParams{Float32}, Thermodynamics.Parameters.ThermodynamicsParameters{Float32}}, Insolation.Parameters.InsolationParameters{Float32}}}}}, ClimaCore.DataLayouts.VF{Float32, 200, CUDA.CuDeviceMatrix{Float32, 1}}, ClimaCore.DataLayouts.VF{Float32, 200, CUDA.CuDeviceMatrix{Float32, 1}}}}, Float32, Float32, Tuple{vanGenuchten{Float32}}, Tuple{ClimaLand.Parameters.LandParameters{Float32, Thermodynamics.Parameters.ThermodynamicsParameters{Float32}, SurfaceFluxes.Parameters.SurfaceFluxesParameters{Float32, SurfaceFluxes.UniversalFunctions.BusingerParams{Float32}, Thermodynamics.Parameters.ThermodynamicsParameters{Float32}}, Insolation.Parameters.InsolationParameters{Float32}}}}}}}}}, ::ClimaCore.DataLayouts.UniversalSize{1, 1, 200, Int64}) resulted in invalid LLVM IR
  Reason: unsupported dynamic function invocation (call to matric_potential)

To Reproduce

Ideally a minimal reproducible example:

include("test/standalone/Soil/soiltest")

Or even better, add it as a unit test, and open pull request.

Project

If not using the `examples` project: ``` paste your Project.toml here. ``` ``` paste your Manifest.toml here. ```

System details

Any relevant system information:

• Julia version: v"1.10.0"
• operating system : "Red Hat Enterprise Linux" "8.10 (Ootpa)"
• modules loaded on cluster (module list)

1. openmpi/4.1.5-mpitrampoline 2) julia/1.10.0 3) cuda/julia-pref 4) common

Related issues / PRs

Please add any relevant links.