how to stop databricks using powershell code

Winnie Yap 20 Reputation points
2025-04-28T16:03:44.38+00:00

Hi Everyone,

I would like to execute a runbook that checks the total costs and shuts down Azure databricks when the costs exceed a certain threshold. I think this can be done using powershell code but I am not sure. Can someone please offer some advice and/or share code to do this?

Best regards,
Winnie

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,413 questions
0 comments No comments
{count} votes

Accepted answer
  1. hossein jalilian 10,580 Reputation points
    2025-04-28T16:52:08.72+00:00

    Hello Winnie Yap,

    Thanks for posting your question in the Microsoft Q&A forum.

    1. You can combine Azure Cost Management budgets, Azure Automation runbooks, and the Databricks PowerShell module.
    2. Set Up a Cost Budget and Trigger: In Azure Cost Management, create a budget for your subscription or resource group, and configure an action group that triggers when your budget threshold is reached.
    3. Create an Azure Automation Runbook: In the Azure Portal, create an Azure Automation account if you don’t already have one.
    4. Install the Databricks PowerShell Module: Use the DatabricksPS module to manage Databricks resources via API
    5. Write the PowerShell Script to Stop Databricks Clusters: Here’s a sample script you can use in your runbook. You’ll need your Databricks workspace URL and a personal access token
         
         $accessToken = "<Your_Databricks_PAT>"
         $apiUrl = "https://<your-region>.azuredatabricks.net"
         
         
         Set-DatabricksEnvironment -AccessToken $accessToken -ApiRootUrl $apiUrl
         
         
         $clusters = Get-DatabricksCluster | Where-Object { $_.state -eq "RUNNING" }
         
         
         foreach ($cluster in $clusters) {
             Stop-DatabricksCluster -ClusterId $cluster.cluster_id
         }
         
      
    6. Automate the Trigger: When your cost budget threshold is reached, Azure will trigger your Automation runbook via the action group and webhook you set up in step 1

    Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful


0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.