Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build wheels without build isolation #1473

Merged
merged 31 commits into from
Oct 22, 2024
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
b983143
test wheel-build sccache fixes
jameslamb Oct 11, 2024
9e2ceef
fiddle with options
jameslamb Oct 11, 2024
8c747f5
run fewer build jobs
jameslamb Oct 11, 2024
7cf37ee
build with --no-build-isolation
jameslamb Oct 11, 2024
4477b70
paths
jameslamb Oct 11, 2024
d6f05c6
more paths
jameslamb Oct 11, 2024
d18c3f7
Merge branch 'wheel-build-sccache' of github.com:jameslamb/cuspatial …
jameslamb Oct 11, 2024
6c64e73
restore all CI
jameslamb Oct 11, 2024
cbfaf85
only pre-install build backend
jameslamb Oct 14, 2024
9744bcf
Merge branch 'wheel-build-sccache' of github.com:jameslamb/cuspatial …
jameslamb Oct 14, 2024
8256e35
only run wheel builds
jameslamb Oct 14, 2024
28e34eb
GHA fix
jameslamb Oct 14, 2024
8251aaa
go back to installing everything
jameslamb Oct 14, 2024
130e761
re-enable all CI
jameslamb Oct 14, 2024
29b005e
move PIP_CONSTRAINT stuff back, remove 'declare'
jameslamb Oct 14, 2024
fc8d575
Merge branch 'wheel-build-sccache' of github.com:jameslamb/cuspatial …
jameslamb Oct 14, 2024
ec06a4a
Merge branch 'branch-24.12' into wheel-build-sccache
bdice Oct 16, 2024
cb6b07d
only use an un-isolated build for libcuspatial
jameslamb Oct 16, 2024
75eaf12
Merge branch 'wheel-build-sccache' of github.com:jameslamb/cuspatial …
jameslamb Oct 16, 2024
7b87cd6
Merge branch 'branch-24.12' of github.com:rapidsai/cuspatial into whe…
jameslamb Oct 17, 2024
f59a067
test with new rapids-dependency-file-generator
jameslamb Oct 17, 2024
1199e31
enable all CI
jameslamb Oct 17, 2024
5e54225
shell variable
jameslamb Oct 17, 2024
9f979b5
actually turn off build isolation
jameslamb Oct 17, 2024
aa62971
more fiddling with PIP_NO_BUILD_ISOLATION
jameslamb Oct 17, 2024
b4cd177
ugh ok yeah it really is PIP_NO_BUILD_ISOLATION=0
jameslamb Oct 17, 2024
69fa9e8
try more libcuspatial builds
jameslamb Oct 17, 2024
8b841c2
restore all CI
jameslamb Oct 17, 2024
e5d02b9
use published rapids-dependency-file-generator packages
jameslamb Oct 22, 2024
dd7a7a6
run another build, hopefully with better cache hit rate
jameslamb Oct 22, 2024
336bc27
print sccache stats for conda builds
jameslamb Oct 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ repos:
- id: verify-copyright
- id: verify-alpha-spec
- repo: https://github.com/rapidsai/dependency-file-generator
rev: v1.13.11
rev: v1.16.0
vyasr marked this conversation as resolved.
Show resolved Hide resolved
hooks:
- id: rapids-dependency-file-generator
args: ["--clean"]
Expand Down
10 changes: 9 additions & 1 deletion ci/build_wheel.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,15 @@ RAPIDS_PY_CUDA_SUFFIX="$(rapids-wheel-ctk-name-gen ${RAPIDS_CUDA_VERSION})"

cd "${package_dir}"

python -m pip wheel . -w dist -vvv --no-deps --disable-pip-version-check
rapids-logger "Building '${package_name}' wheel"
python -m pip wheel \
-w dist \
-v \
--no-deps \
--disable-pip-version-check \
.

sccache --show-adv-stats
vyasr marked this conversation as resolved.
Show resolved Hide resolved

mkdir -p final_dist
python -m auditwheel repair \
Expand Down
24 changes: 23 additions & 1 deletion ci/build_wheel_libcuspatial.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,26 @@

set -euo pipefail

ci/build_wheel.sh libcuspatial python/libcuspatial cpp
package_name="libcuspatial"

rapids-logger "Generating build requirements"
matrix_selectors="cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION};cuda_suffixed=true"

rapids-dependency-file-generator \
--output requirements \
--file-key "py_build_${package_name}" \
--file-key "py_rapids_build_${package_name}" \
--matrix "${matrix_selectors}" \
| tee /tmp/requirements-build.txt

rapids-logger "Installing build requirements"
python -m pip install \
-v \
--prefer-binary \
-r /tmp/requirements-build.txt

# build with '--no-build-isolation', for better sccache hit rate
# 0 really means "add --no-build-isolation" (ref: https://github.com/pypa/pip/issues/5735)
export PIP_NO_BUILD_ISOLATION=0
vyasr marked this conversation as resolved.
Show resolved Hide resolved

ci/build_wheel.sh "${package_name}" python/libcuspatial cpp
Loading