Service principals

This page has an overview of service principals in Azure Databricks. For how to manage service principals, see Manage service principals.

What is a service principal?

A service principal is a specialized identity in Azure Databricks designed for automation and programmatic access. Service principals give automated tools and scripts API-only access to Azure Databricks resources, providing greater security than using users accounts.

You can grant and restrict a service principal’s access to resources in the same way as you can an Azure Databricks user. For example, you can:

• Grant a service principal the account admin or workspace admin role
• Grant a service principal access to data using Unity Catalog.
• Add a service principal as a member to a group.

You can grant Azure Databricks users, service principals, and groups permissions to use a service principal. This allows users to run jobs as the service principal, instead of as their identity, which prevents jobs from failing if a user leaves your organization or a group is modified.

Benefits of using service principals:

• Security and stability: Automate jobs and workflows without relying on individual user credentials to reduce risks associated with user account changes or departures.
• Flexible permissions: Allow users, groups, or other service principals to delegate permissions to a service principal, enabling job execution on their behalf.
• API-Only identity: Unlike regular Databricks users, service principals are designed solely for API access and cannot log into the Databricks UI.

Databricks and Microsoft Entra ID service principals

Service principals can either be Azure Databricks managed service principals or Microsoft Entra ID managed service principals.

Azure Databricks managed service principals can authenticate to Azure Databricks using Databricks OAuth authentication and personal access tokens. Microsoft Entra ID managed service principals can authenticate to Azure Databricks using Databricks OAuth authentication and Microsoft Entra ID tokens. For more information on authentication for service principals, see Manage tokens for a service principal.

Azure Databricks managed service principals are managed directly within Azure Databricks. Microsoft Entra ID managed service principals are managed in Microsoft Entra ID, which requires additional permissions. Databricks recommends that you use Azure Databricks managed service principals for Azure Databricks automation and that you use Microsoft Entra ID managed service principals in cases where you must authenticate with Azure Databricks and other Azure resources at the same time.

To create a Azure Databricks managed service principal, skip this section and continue reading with Who can manage and use service principals?.

To use Microsoft Entra ID managed service principals in Azure Databricks, an admin user must create a Microsoft Entra ID application in Azure. To create a Microsoft Entra ID managed service principal, see MS Entra service principal authentication.

Who can manage and use service principals?

To manage service principals in Azure Databricks, you must have one of the following: the account admin role, the workspace admin role, or the manager or user role on a service principal.

• Account admins can add service principals to the account and assign them admin roles. They can also assign service principals to workspaces, as long as those workspaces use identity federation.
• Workspace admins can add service principals to an Azure Databricks workspace, assign them the workspace admin role, and manage access to objects and functionality in the workspace, such as the ability to create clusters or access specified persona-based environments.
• Service Principal Managers can manage roles on a service principal. The creator of a service principal becomes the service principal manager. Account admins are service principal managers on all service principals in an account.
• Service Principal Users can run jobs as the service principal. The job runs using the identity of the service principal, instead of the identity of the job owner. For more information, see Manage identities, permissions, and privileges for Databricks Jobs.

Users with the Service Principal Manager role do not inherit the Service Principal User role. If you want to use the service principal to execute jobs, you need to explicitly assign yourself the service principal user role, even after creating the service principal.

For information on how to grant the service principal manager and user roles, see Roles for managing service principals.

Sync service principals to your Azure Databricks account from your Microsoft Entra ID tenant

You can sync Microsoft Entra ID service principals automatically from your Microsoft Entra ID tenant to your Azure Databricks account using automatic identity management (Public Preview). Databricks uses Microsoft Entra ID as the source, so any changes to users or group memberships are respected in Azure Databricks. For instructions, see Sync users and groups automatically from Microsoft Entra ID.

SCIM provisioning does not support syncing service principals.