Build with the Python limited API. #108
Closed
+11
−6
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I noticed discussion on #21 around supported Python versions last week and it got me thinking about whether pynvjitlink really needs Python version-specific packages. pynvjitlink uses the C API directly, and it uses a pretty minimal subset, so we can actually just build it with the limited API and get a package compatible across Python versions. I tested the built wheel on 3.11 and 3.12 and tests pass just fine, at least locally.
The main blocker is that we require the limited API as of Python 3.11 (namely support for the buffer protocol) to make this switch. We just dropped support for Python 3.9, so we're likely a year out from dropping 3.10. If we do think that the limited API is a path work exploring, our two options are to simply keep this PR around as a reminder until we drop 3.10, or to bifurcate our build into a 3.10 path and a 3.11+ path. I don't think the latter would be too painful to maintain. It would require three things that I can think of off the top of my head: 1) updating test jobs to download the abi3 wheel for Python>3.11 instead of one specific to their version (so in general, some of the improvements needed for rapidsai/build-planning#43); 2) specifying
wheel.py-api
via a config-settings flag in build_wheel.sh for Python 3.10; and 3) updating the conda recipes accordingly to set the right run exports for the abi3 packages.A middle ground might be to do nothing now, but the next time someone asks for support for Python 3.13 we switch to the limited API instead.
Since pynvjitlink has users beyond just RAPIDS, making the change here seems beneficial so that pynvjitlink can uncouple from the Python upgrade cycle of RAPIDS.