Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RELEASE] cucim v24.10 #788

Merged
merged 21 commits into from
Oct 9, 2024
Merged

[RELEASE] cucim v24.10 #788

merged 21 commits into from
Oct 9, 2024

Conversation

raydouglass
Copy link
Member

❄️ Code freeze for branch-24.10 and v24.10 release

What does this mean?

Only critical/hotfix level issues should be merged into branch-24.10 until release (merging of this PR).

What is the purpose of this PR?

  • Update documentation
  • Allow testing for the new release
  • Enable a means to merge branch-24.10 into main for the release

raydouglass and others added 20 commits July 19, 2024 14:50
Forward-merge branch-24.08 into branch-24.10
…ration (#757)

The following section of the dependencies.yaml was being ignored:
```
  develop:
    common:
      - output_types: [conda, requirements, pyproject]
        packages:
          - pre-commit
          - black
          - ruff
          - isort
```
I noticed this because changing any values here and then running `rapids-dependency-file-generator` did not cause any corresponding change in the `pyproject.toml`.

The root cause is the files section was defining `py_develop` twice (lines 64 and 72). The first time pointed to the desired `develop:` section shown above, but the second, duplicate one pointed to the `docs:` section instead. I renamed the second file target to `py_docs` to avoid ignoring the developer dependencies section.

I tested locally by making changes to the develop: section to verify that they would now show up in the `pyproject.toml`.

Ideally we could have `rapids-dependency-file-generator` raise an error or print a warning if duplicate names like this were found.

Authors:
  - Gregory Lee (https://github.com/grlee77)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)
  - https://github.com/jakirkham

URL: #757
Forward-merge branch-24.08 into branch-24.10
This PR updates pre-commit hooks to the latest versions that are supported without causing style check errors.

Authors:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: #760
Remove the `pytest-lazy-fixture` plugin and use the replacement `pytest-lazy-fixtures` package instead.

Address  #755

Authors:
  - Gigon Bae (https://github.com/gigony)

Approvers:
  - https://github.com/jakirkham

URL: #756
…scalars (#764)

This change avoids an issue with an internal `numpy.can_cast` call in CuPy's kernel fusion code when using NumPy 2.0. With this change all cuCIM tests passed using NumPy 2.0.1 and the CuPy 13.3dev branch.

Fixes #742

Authors:
  - Gregory Lee (https://github.com/grlee77)

Approvers:
  - https://github.com/jakirkham
  - Gigon Bae (https://github.com/gigony)

URL: #764
Contributes to rapidsai/build-planning#88

Finishes the work of dropping Python 3.9 support.

This project stopped building / testing against Python 3.9 as of rapidsai/shared-workflows#235.
This PR updates configuration and docs to reflect that.

## Notes for Reviewers

### How I tested this

Checked that there were no remaining uses like this:

```shell
git grep -E '3\.9'
git grep '39'
git grep 'py39'
```

And similar for variations on Python 3.8 (to catch things that were missed the last time this was done).

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Gregory Lee (https://github.com/grlee77)
  - https://github.com/jakirkham
  - Bradley Dice (https://github.com/bdice)
  - Jake Awe (https://github.com/AyodeAwe)

URL: #766
This PR removes the NumPy<2 pin which is expected to work for
RAPIDS projects once CuPy 13.3.0 is released (CuPy 13.2.0 had
some issues preventing the use with NumPy 2).

Authors:
  - Sebastian Berg (https://github.com/seberg)
  - https://github.com/jakirkham

Approvers:
  - James Lamb (https://github.com/jameslamb)
  - Gregory Lee (https://github.com/grlee77)
  - https://github.com/jakirkham

URL: #762
This PR updates rapidsai/pre-commit-hooks to the version 0.4.0.

Authors:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: #772
We currently bundle some custom 2D convolution kernels not present in CuPy. We have additional testing of these internel kernels, but I found that on the Windows platform a subset of these tests currently fail.

This MR disables auto-selection of these kernels on the Windows platform (will always fall back to CuPy's `cupyx.scipy.ndimage` kernels in this case). The test cases that are currently known to fail are skipped for now on Windows. Longer term it would be good to resolve the issues with the kernels, restore the test cases and enable use of these kernels on Windows.

Authors:
  - Gregory Lee (https://github.com/grlee77)

Approvers:
  - https://github.com/jakirkham

URL: #770
Contributes to rapidsai/build-planning#40

This PR adds support for Python 3.12.

## Notes for Reviewers

This is part of ongoing work to add Python 3.12 support across RAPIDS.
It temporarily introduces a build/test matrix including Python 3.12, from rapidsai/shared-workflows#213.

A follow-up PR will revert back to pointing at the `branch-24.10` branch of `shared-workflows` once all
RAPIDS repos have added Python 3.12 support.

### This will fail until all dependencies have been updates to Python 3.12

CI here is expected to fail until all of this project's upstream dependencies support Python 3.12.

This can be merged whenever all CI jobs are passing.

Authors:
  - James Lamb (https://github.com/jameslamb)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #773
Recently RAPIDS added wheel tests on Rocky Linux 8. This requires a change to support installing dependencies with `yum` on Rocky Linux 8.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - James Lamb (https://github.com/jameslamb)

URL: #774
While adding support for Python 3.12 in #773, we found a problem where `GPUtil` does not support Python 3.12 (#775).

This PR removes the dependency on `GPUtil` and replaces it with `cupy`, which is already a dependency.

Closes #775. Closes #776.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - James Lamb (https://github.com/jameslamb)
  - Gregory Lee (https://github.com/grlee77)
  - Gigon Bae (https://github.com/gigony)

URL: #777
…rmance of blob detectors and `corner_peaks`) (#782)

It seems that we can get a large speedup in calls to `peak_local_max` when the default `min_distance=1` is specified. A bottleneck in that function can be the call to `ensure_spacing` which falls back to the CPU, but we don't need to call that function at all unless `min_distance>1` (no coordinates can be excluded if min_distance=1, so it is pointless to call it).

I proposed the same changes upstream in scikit-image/scikit-image#7548. See additional comments and measurements made there.

There should be no change in behavior with this MR, it is purely a performance improvement.

Authors:
  - Gregory Lee (https://github.com/grlee77)

Approvers:
  - Gigon Bae (https://github.com/gigony)

URL: #782
CuPy 13.3 removed dependence on jitify (see: cupy/cupy#8473)

Make the same change in our vendored code (but only if CuPy >= 13.3).

Authors:
  - Gregory Lee (https://github.com/grlee77)

Approvers:
  - https://github.com/jakirkham

URL: #781
Libraries such as jbig, libwebp-base, and zlib are not used for the functionality of cuCIM. 
These codec libraries are linked to libtiff only when available at build time.

This patch explicitly disables external codecs and removes unnecessary dependencies from the `dependencies.yaml`.

With this patch, I can confirm that they are not requested at all.
```
--  Support for external codecs:
--   ZLIB support:                       OFF (requested)  (availability)
--   Pixar log-format algorithm:         OFF (requested) FALSE (availability)
--   JPEG support:                       OFF (requested) TRUE (availability)
--   Old JPEG support:                   OFF (requested) TRUE (availability)
--   JPEG 8/12 bit dual mode:            OFF (requested) FALSE (availability)
--   ISO JBIG support:                   OFF (requested) FALSE (availability)
--   LZMA2 support:                      OFF (requested)  (availability)
--   ZSTD support:                       OFF (requested)  (availability)
--   WEBP support:                       OFF (requested) FALSE (availability)
```

Authors:
  - Gigon Bae (https://github.com/gigony)

Approvers:
  - James Lamb (https://github.com/jameslamb)
  - https://github.com/jakirkham

URL: #785
@raydouglass raydouglass requested review from a team as code owners October 4, 2024 19:46
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 84.21053% with 3 lines in your changes missing coverage. Please review.

Project coverage is 93.56%. Comparing base (2c63e75) to head (289e340).
Report is 53 commits behind head on main.

Files with missing lines Patch % Lines
.../src/cucim/skimage/_shared/version_requirements.py 33.33% 2 Missing ⚠️
.../src/cucim/skimage/filters/_separable_filtering.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #788      +/-   ##
==========================================
+ Coverage   93.49%   93.56%   +0.07%     
==========================================
  Files         129      129              
  Lines       10039    10044       +5     
==========================================
+ Hits         9386     9398      +12     
+ Misses        653      646       -7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@raydouglass raydouglass merged commit 16b3e5b into main Oct 9, 2024
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants