fix: race condition on cluster creation (#751)

Signed-off-by: Juri Krainjukov <[email protected]>
databricks · Jul 31, 2024 · 38aefbf · 38aefbf
1 parent 669c5b8
commit 38aefbf
Show file tree

Hide file tree

Showing 2 changed files with 4 additions and 1 deletion.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -6,6 +6,7 @@
 - Always use lower case when gathering metadata (since objects are stored internally as lower case regardless of how we create them) ([742](https://github.com/databricks/dbt-databricks/pull/742))
 - Persist table comments for python models ([743](https://github.com/databricks/dbt-databricks/pull/743))
 - Stop cursor destructor warnings ([744](https://github.com/databricks/dbt-databricks/pull/744))
+- Race condition on cluster creation. (thanks @jurasan!) ([751](https://github.com/databricks/dbt-databricks/pull/751))
 
 ## dbt-databricks 1.8.4 (July 17, 2024)
 

diff --git a/dbt/adapters/databricks/python_submissions.py b/dbt/adapters/databricks/python_submissions.py
@@ -395,7 +395,9 @@ def start_cluster(self) -> None:
             json={"cluster_id": self.cluster_id},
         )
         if response.status_code != 200:
-            raise DbtRuntimeError(f"Error starting terminated cluster.\n {response.content!r}")
+            current_status = self.get_cluster_status().get("state", "").upper()
+            if current_status not in ["RUNNING", "PENDING"]:
+                raise DbtRuntimeError(f"Error starting terminated cluster.\n {response.content!r}")
 
         self._wait_for_cluster_to_start()