How to select performance mode for Azure Databricks Delta Live Tables

Question

How to select performance mode for Azure Databricks Delta Live Tables

Quan Sun 21

Hi everyone,

For triggered pipelines, you can select the serverless compute performance mode using the Performance optimized setting in the pipeline scheduler. When this setting is disabled, the pipeline uses standard performance mode.

we can select 2 performance modes for pipelines (i.e. delta live tables).

But I tried everything, I was not able to find the location to select/disable/enable performance modes. Could you please tell me how to do so? Or did I misunderstand anything?

Btw, in my Azure databricks workspace, I can use serverless compute for both pipelines and jobs.

Thanks in advance.

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Krupal Bandari 80 Reputation points Microsoft External Staff

2025-04-28T08:54:38.5266667+00:00

Hi @Quan Sun,

It seems like you’re trying to figure out how to select or switch between performance modes for Delta Live Tables in Azure Databricks, but you’re having a hard time finding the option, even though the documentation says it should be there.

To change the performance mode, you normally need to go into the settings of the pipeline you’re working with. In your Azure Databricks workspace:

Go to the Jobs or Pipelines section.

Find and select the pipeline where you want to update the performance mode.

Inside the pipeline settings, look for something like "Performance Optimization" or "Serverless Compute". There, you should see options to switch between Performance optimized and Standard mode, usually by toggling a setting. Once you make your changes, don't forget to save or update the pipeline so it actually takes effect.

If you’re not seeing these options, it might be because of permission issues or maybe a mismatch in the Databricks version you're using.

Reference: https://learn.microsoft.com/en-us/azure/databricks/dlt/serverless-dlt https://learn.microsoft.com/en-us/azure/databricks/jobs/run-serverless-jobs#select-a-serverless-https://learn.microsoft.com/en-us/azure/databricks/dlt/pipeline-mode I hope this information helps.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Quan Sun 21 Reputation points

2025-04-29T00:40:16.3233333+00:00

Hi Krupal, thanks for your answer.

'Inside the pipeline settings, look for something like "Performance Optimization" or "Serverless Compute"' could you please share a screenshot of this? I still cannot see it.

Thanks.
Quan Sun 21 Reputation points

2025-04-30T02:33:31.12+00:00

Thanks Krupal. You saved my day.

Now I understand what happened.

We created our dlt pipelines before Feb 5, 2025, in pipeline settings summary, we can see below. These old pipelines don't have performance optimized mode by default which take 4-6 mins to start up.

But the newly created dlt pipelines have performance optimized mode by default and they can start up almost immediately.
Krupal Bandari 80 Reputation points Microsoft External Staff

2025-04-30T03:18:45.7566667+00:00

@Quan Sun
Just checking in to see if the below answer helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Accepted answer

0 additional answers

Your answer

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Krupal Bandari 80 Reputation points Microsoft External Staff

2025-04-28T08:54:38.5266667+00:00

Hi @Quan Sun,

It seems like you’re trying to figure out how to select or switch between performance modes for Delta Live Tables in Azure Databricks, but you’re having a hard time finding the option, even though the documentation says it should be there.

To change the performance mode, you normally need to go into the settings of the pipeline you’re working with. In your Azure Databricks workspace:

Go to the Jobs or Pipelines section.

Find and select the pipeline where you want to update the performance mode.

Inside the pipeline settings, look for something like "Performance Optimization" or "Serverless Compute". There, you should see options to switch between Performance optimized and Standard mode, usually by toggling a setting. Once you make your changes, don't forget to save or update the pipeline so it actually takes effect.

If you’re not seeing these options, it might be because of permission issues or maybe a mismatch in the Databricks version you're using.

Reference: https://learn.microsoft.com/en-us/azure/databricks/dlt/serverless-dlt https://learn.microsoft.com/en-us/azure/databricks/jobs/run-serverless-jobs#select-a-serverless-https://learn.microsoft.com/en-us/azure/databricks/dlt/pipeline-mode I hope this information helps.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Quan Sun 21 Reputation points

2025-04-29T00:40:16.3233333+00:00

Hi Krupal, thanks for your answer.

'Inside the pipeline settings, look for something like "Performance Optimization" or "Serverless Compute"' could you please share a screenshot of this? I still cannot see it.

Thanks.
Quan Sun 21 Reputation points

2025-04-30T02:33:31.12+00:00

Thanks Krupal. You saved my day.

Now I understand what happened.

We created our dlt pipelines before Feb 5, 2025, in pipeline settings summary, we can see below. These old pipelines don't have performance optimized mode by default which take 4-6 mins to start up.

But the newly created dlt pipelines have performance optimized mode by default and they can start up almost immediately.
Krupal Bandari 80 Reputation points Microsoft External Staff

2025-04-30T03:18:45.7566667+00:00

@Quan Sun
Just checking in to see if the below answer helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 1

@Quan Sun

In the past, Databricks provided a checkbox to manually enable "Performance Optimized" mode in the UI. But now, when you select Serverless compute with a Triggered pipeline, Databricks automatically runs it in Performance Optimized mode behind the scenes. That's why the checkbox no longer appears it's assumed based on your compute selection. User's image

I hope this information helps.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues

Share via

How to select performance mode for Azure Databricks Delta Live Tables

0 additional answers

Your answer