Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade databricks-sql-connector to allow 3.2.0 or 3.3.0 #775

Open
stephen-bowser opened this issue Aug 20, 2024 · 1 comment
Open

upgrade databricks-sql-connector to allow 3.2.0 or 3.3.0 #775

stephen-bowser opened this issue Aug 20, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@stephen-bowser
Copy link

Describe the feature

It would be helpful if dbt-databricks was compatible with databricks-sql-connector 3.3.0 or 3.2.0. This would help since these versions are compatible with pyarrow (">=14.0.1,<17"), whereas 3.1.0 is restricted to ^14.0.1

Describe alternatives you've considered

N/A

Additional context

N/A

Who will this benefit?

Anyone who is interested in using the latest versions of pyarrow. In particular, the latest versions (0.19.x) of deltalake require more modern versions of pyarrow than what is currently enabled by dbt-databricks.

@stephen-bowser stephen-bowser added the enhancement New feature or request label Aug 20, 2024
@benc-db
Copy link
Collaborator

benc-db commented Aug 20, 2024

This will come in 1.9; we've become conservative about when we upgrade our dependency versions due to major issues in the migration to 3.0, since users on Databricks Workflows will always get latest available versions if we don't pin them down.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants