Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GeoArrays.read(ga, masked=false) should probably be default #124

Open
alex-s-gardner opened this issue Feb 20, 2023 · 4 comments
Open

GeoArrays.read(ga, masked=false) should probably be default #124

alex-s-gardner opened this issue Feb 20, 2023 · 4 comments

Comments

@alex-s-gardner
Copy link
Contributor

In all of my workflows I need to set masked=false when using GeoArrays.read otherwise I get really slow read times. Also masked=false results in a change in Type as missings are added. For this reason I think masked=false should be set as the default

@evetion
Copy link
Owner

evetion commented Feb 21, 2023

Could you specify how much slower? And are you only accessing a smaller part of a larger raster? Because masked=false is lazy and will only load into memory on the getindex later on, so initial load is fast, but accessing values is slower.

Also, there's an open issue #108 for handling masked/nodata values in masked=false, because now you need to handle those yourself.

Once I fixed that issue, I agree that everything could be lazy by default.

@alex-s-gardner
Copy link
Contributor Author

@evetion I'm working with subsets drawn from global .vrts, when masked=true the process slows by multiple orders of magnitude. Every once and awhile I forget to include masked=false and the pipeline comes to a halt.

@evetion
Copy link
Owner

evetion commented Feb 21, 2023

That sounds familiar indeed. How do you handle nodata/missing values in that use case? Or do you have a separate mask vrt?

@alex-s-gardner
Copy link
Contributor Author

It's depends a bit, sometimes I leave the no data value in the data and work with it that way... sometimes I fill the no_data value with NAN's after subsetting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants