Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dynamic dispatch in Proj.proj_trans #104

Closed
ctessum opened this issue Apr 16, 2024 · 7 comments
Closed

dynamic dispatch in Proj.proj_trans #104

ctessum opened this issue Apr 16, 2024 · 7 comments

Comments

@ctessum
Copy link

ctessum commented Apr 16, 2024

Hello,

I'm using Proj in some code where I'm trying to avoid allocations and dynamic dispatches for performance. To assist with this, I'm using the AllocCheck.jl packages, which finds allocations and dynamic dispatches in the code. So this example:

using Proj
using AllocCheck

const pj = Proj.proj_create_crs_to_crs(
    "EPSG:4326",  # source
    "+proj=utm +zone=32 +datum=WGS84",  # target, also EPSG:32632
)
const a = Proj.proj_coord(12, 55)

Proj.proj_trans(pj, Proj.PJ_FWD, a)

@check_allocs checkf(a) = Proj.proj_trans(pj, Proj.PJ_FWD, a)

try
    checkf(a)
catch err
    err.errors
end

gives the following error message:

Dynamic dispatch in ....julia/packages/Proj/YfK57/src/libproj.jl:494
  | @ccall libproj.proj_trans(P::Ptr{PJ}, direction::PJ_DIRECTION, coord::Coord)::Coord

Stacktrace:
 [1] proj_trans
   @ ~/.julia/packages/Proj/YfK57/src/libproj.jl:494 [inlined]
 [2] var"##checkf#260"(a::Coord)
 ....

I don't know enough about @ccall to be able to tell what the issue is, if there is one. Any suggestions would be appreciated!

@rafaqz
Copy link
Member

rafaqz commented Apr 16, 2024

Did you try looking at a profile? E.g. ProfileView.jl? It will show allocs and instability in yellow and red

@ctessum
Copy link
Author

ctessum commented Apr 16, 2024

I tried looking at at with JET, but nothing shows up there. It seems like it's something specific to that @CCall line, and my guess is that maybe JET doesn't analyze @CCall but maybe AllocCheck.jl does, but I really don't know.

@rafaqz
Copy link
Member

rafaqz commented Apr 16, 2024

I mean a sampling profiler like ProfileView.jl. Thats the best tool for this job because it will instantly show where the allocations are coming from. Just run that code a lot of times inside a function with a loop in it, and put @profview in front of the function call.

@ctessum
Copy link
Author

ctessum commented Apr 16, 2024

I mean, AllocCheck is pretty clear about where it's happening, line 494 of Proj.jl. I just don't know why it's a dynamic dispatch, or how it would be fixed.

@rafaqz
Copy link
Member

rafaqz commented Apr 17, 2024

But... the profile will show if its type instability (boxing) causing allocation, or just regular allocation, red or yellow in the profile. (Likely red given dynamic dispatch warning). It also gives context you need to understand type instability/dynamic dispatch.

They are totally unrelated reasons for allocation and totally different fixes.

If its red, use Cthulu.jl @descend to look around at types where you see problems in the profile. descend_clicked() after clicking the profile can help get there quicker.

If yellow, investigate possible allocations like Ref in code.

Compared to the ProfileView/Cthulhu combo AllocCheck is a blunt instrument. You need to see the context.

@asinghvi17
Copy link
Member

asinghvi17 commented Jun 16, 2024

I can't see this allocation or any instability in ProfileView from my end. I do see the same error from AllocCheck, though.

Perhaps the "dynamic dispatch" here is really triggering on the @ccall? A BenchmarkTools benchmarking of the code in that function shows no allocations, so this might be a false alarm.

julia> @benchmark Proj.proj_trans($pj, $(Proj.PJ_FWD), $a)
BenchmarkTools.Trial: 10000 samples with 911 evaluations.
 Range (min  max):  114.572 ns   2.863 μs  ┊ GC (min  max): 0.00%  0.00%
 Time  (median):     120.518 ns              ┊ GC (median):    0.00%
 Time  (mean ± σ):   123.166 ns ± 33.619 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▃  ▂▃▅█▄▄▄▆▆▅▄▂▂▁▁▁▁▁                                        ▂
  █▆▄█████████████████████▇▇▇▇▇▇▇▆▅▆▆▆▆▆▅▆▅▅▆▆▆▅▅▄▅▄▅▄▄▄▃▄▄▄▄▄ █
  115 ns        Histogram: log(frequency) by time       158 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

@rafaqz
Copy link
Member

rafaqz commented Jun 16, 2024

I can't see this allocation or any instability in ProfileView from my end

If there is nothing in the profile that suggests there isn't actually a problem. Time of 100ns and no allocs in @btime also suggests that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants