Big Data Pools - Create Or Update

Reference

Service:: Synapse

API Version:: 2021-06-01

Create a Big Data pool.
Create a new Big Data pool.

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Synapse/workspaces/{workspaceName}/bigDataPools/{bigDataPoolName}?api-version=2021-06-01

With optional parameters:

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Synapse/workspaces/{workspaceName}/bigDataPools/{bigDataPoolName}?api-version=2021-06-01&force={force}

URI Parameters

Name	In	Required	Type	Description
bigDataPoolName	path	True	string	Big Data pool name
resourceGroupName	path	True	string minLength: 1 maxLength: 90	The name of the resource group. The name is case insensitive.
subscriptionId	path	True	string minLength: 1	The ID of the target subscription.
workspaceName	path	True	string	The name of the workspace.
api-version	query	True	string minLength: 1	The API version to use for this operation.
force	query		boolean	Whether to stop any running jobs in the Big Data pool

Request Body

Name	Required	Type	Description
location	True	string	The geo-location where the resource lives
properties.autoPause		AutoPauseProperties	Spark pool auto-pausing properties Auto-pausing properties
properties.autoScale		AutoScaleProperties	Spark pool auto-scaling properties Auto-scaling properties
properties.cacheSize		integer (int32)	The cache size
properties.customLibraries		LibraryInfo[]	List of custom libraries/packages associated with the spark pool.
properties.defaultSparkLogFolder		string	The default folder where Spark logs will be written.
properties.dynamicExecutorAllocation		DynamicExecutorAllocation	Dynamic Executor Allocation
properties.isAutotuneEnabled		boolean	Enable Autotune Whether autotune is required or not.
properties.isComputeIsolationEnabled		boolean	Whether compute isolation is required or not.
properties.libraryRequirements		LibraryRequirements	Spark pool library version requirements Library version requirements
properties.nodeCount		integer (int32)	The number of nodes in the Big Data pool.
properties.nodeSize		NodeSize	The level of compute power that each node in the Big Data pool has.
properties.nodeSizeFamily		NodeSizeFamily	The kind of nodes that the Big Data pool provides.
properties.provisioningState		string	The state of the Big Data pool.
properties.sessionLevelPackagesEnabled		boolean	Whether session level packages enabled.
properties.sparkConfigProperties		SparkConfigProperties	Spark pool Config Properties Spark configuration file to specify additional properties
properties.sparkEventsFolder		string	The Spark events folder
properties.sparkVersion		string	The Apache Spark version.
tags		object	Resource tags.

Responses

Name	Type	Description
200 OK	BigDataPoolResourceInfo	OK
202 Accepted	BigDataPoolResourceInfo
Other Status Codes	ErrorResponse

Examples

Create or update a Big Data pool

Sample request

HTTP

PUT https://management.azure.com/subscriptions/01234567-89ab-4def-0123-456789abcdef/resourceGroups/ExampleResourceGroup/providers/Microsoft.Synapse/workspaces/ExampleWorkspace/bigDataPools/ExamplePool?api-version=2021-06-01

{
  "tags": {
    "key": "value"
  },
  "location": "West US 2",
  "properties": {
    "sparkVersion": "3.3",
    "nodeCount": 4,
    "nodeSize": "Medium",
    "nodeSizeFamily": "MemoryOptimized",
    "autoScale": {
      "enabled": true,
      "minNodeCount": 3,
      "maxNodeCount": 50
    },
    "autoPause": {
      "enabled": true,
      "delayInMinutes": 15
    },
    "sparkEventsFolder": "/events",
    "libraryRequirements": {
      "content": "",
      "filename": "requirements.txt"
    },
    "defaultSparkLogFolder": "/logs",
    "isAutotuneEnabled": false
  }
}

Sample response

Status code:: 200

{
  "id": "/subscriptions/01234567-89ab-4def-0123-456789abcdef/resourceGroups/ExampleResourceGroup/providers/Microsoft.Synapse/workspaces/ExampleWorkspace/bigDataPools/ExamplePool",
  "type": "Microsoft.Synapse/workspaces/bigDataPools",
  "location": "West US 2",
  "name": "ExamplePool",
  "tags": {
    "key": "value"
  },
  "properties": {
    "provisioningState": "Provisioning",
    "sparkVersion": "3.3",
    "nodeCount": 4,
    "nodeSize": "Medium",
    "nodeSizeFamily": "MemoryOptimized",
    "autoScale": {
      "enabled": true,
      "minNodeCount": 3,
      "maxNodeCount": 50
    },
    "autoPause": {
      "enabled": true,
      "delayInMinutes": 15
    },
    "creationDate": "1970-01-01T00:00:00Z",
    "sparkEventsFolder": "/events",
    "libraryRequirements": {
      "time": "1970-01-01T00:00:00Z",
      "content": "",
      "filename": "requirements.txt"
    },
    "defaultSparkLogFolder": "/logs",
    "lastSucceededTimestamp": "1970-01-01T10:00:00Z",
    "isAutotuneEnabled": false
  }
}

Status code:: 202

{
  "id": "/subscriptions/01234567-89ab-4def-0123-456789abcdef/resourceGroups/ExampleResourceGroup/providers/Microsoft.Synapse/workspaces/ExampleWorkspace/bigDataPools/ExamplePool",
  "type": "Microsoft.Synapse/workspaces/bigDataPools",
  "location": "West US 2",
  "name": "ExamplePool",
  "tags": {
    "key": "value"
  },
  "properties": {
    "provisioningState": "Provisioning",
    "sparkVersion": "3.3",
    "nodeCount": 4,
    "nodeSize": "Medium",
    "nodeSizeFamily": "MemoryOptimized",
    "autoScale": {
      "enabled": true,
      "minNodeCount": 3,
      "maxNodeCount": 50
    },
    "autoPause": {
      "enabled": true,
      "delayInMinutes": 15
    },
    "creationDate": "1970-01-01T00:00:00Z",
    "sparkEventsFolder": "/events",
    "libraryRequirements": {
      "time": "1970-01-01T00:00:00Z",
      "content": "",
      "filename": "requirements.txt"
    },
    "defaultSparkLogFolder": "/logs",
    "isAutotuneEnabled": false
  }
}

Status code:: default

{
  "error": {
    "code": "Error code",
    "message": "Error message"
  }
}

Definitions

Name	Description
AutoPauseProperties	Spark pool auto-pausing properties
AutoScaleProperties	Spark pool auto-scaling properties
BigDataPoolResourceInfo	Big Data pool
ConfigurationType	The type of the spark config properties file.
DynamicExecutorAllocation	Dynamic Executor Allocation Properties
ErrorAdditionalInfo	The resource management error additional info.
ErrorDetail	The error detail.
ErrorResponse	Error response
LibraryInfo	Information about a library/package created at the workspace level.
LibraryRequirements	Spark pool library version requirements
NodeSize	The level of compute power that each node in the Big Data pool has.
NodeSizeFamily	The kind of nodes that the Big Data pool provides.
SparkConfigProperties	Spark pool Config Properties

AutoPauseProperties

Object

Spark pool auto-pausing properties

Name	Type	Description
delayInMinutes	integer (int32)	Number of minutes of idle time before the Big Data pool is automatically paused.
enabled	boolean	Whether auto-pausing is enabled for the Big Data pool.

AutoScaleProperties

Object

Spark pool auto-scaling properties

Name	Type	Description
enabled	boolean	Whether automatic scaling is enabled for the Big Data pool.
maxNodeCount	integer (int32)	The maximum number of nodes the Big Data pool can support.
minNodeCount	integer (int32)	The minimum number of nodes the Big Data pool can support.

BigDataPoolResourceInfo

Object

Big Data pool

Name	Type	Description
id	string	Fully qualified resource ID for the resource. Ex - /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/{resourceProviderNamespace}/{resourceType}/{resourceName}
location	string	The geo-location where the resource lives
name	string	The name of the resource
properties.autoPause	AutoPauseProperties	Spark pool auto-pausing properties Auto-pausing properties
properties.autoScale	AutoScaleProperties	Spark pool auto-scaling properties Auto-scaling properties
properties.cacheSize	integer (int32)	The cache size
properties.creationDate	string (date-time)	The time when the Big Data pool was created.
properties.customLibraries	LibraryInfo[]	List of custom libraries/packages associated with the spark pool.
properties.defaultSparkLogFolder	string	The default folder where Spark logs will be written.
properties.dynamicExecutorAllocation	DynamicExecutorAllocation	Dynamic Executor Allocation
properties.isAutotuneEnabled	boolean	Enable Autotune Whether autotune is required or not.
properties.isComputeIsolationEnabled	boolean	Whether compute isolation is required or not.
properties.lastSucceededTimestamp	string (date-time)	The time when the Big Data pool was updated successfully.
properties.libraryRequirements	LibraryRequirements	Spark pool library version requirements Library version requirements
properties.nodeCount	integer (int32)	The number of nodes in the Big Data pool.
properties.nodeSize	NodeSize	The level of compute power that each node in the Big Data pool has.
properties.nodeSizeFamily	NodeSizeFamily	The kind of nodes that the Big Data pool provides.
properties.provisioningState	string	The state of the Big Data pool.
properties.sessionLevelPackagesEnabled	boolean	Whether session level packages enabled.
properties.sparkConfigProperties	SparkConfigProperties	Spark pool Config Properties Spark configuration file to specify additional properties
properties.sparkEventsFolder	string	The Spark events folder
properties.sparkVersion	string	The Apache Spark version.
tags	object	Resource tags.
type	string	The type of the resource. E.g. "Microsoft.Compute/virtualMachines" or "Microsoft.Storage/storageAccounts"

ConfigurationType

Enumeration

The type of the spark config properties file.

Value	Description
Artifact
File

DynamicExecutorAllocation

Object

Dynamic Executor Allocation Properties

Name	Type	Description
enabled	boolean	Indicates whether Dynamic Executor Allocation is enabled or not.
maxExecutors	integer (int32)	The maximum number of executors alloted
minExecutors	integer (int32)	The minimum number of executors alloted

ErrorAdditionalInfo

Object

The resource management error additional info.

Name	Type	Description
info	object	The additional info.
type	string	The additional info type.

ErrorDetail

Object

The error detail.

Name	Type	Description
additionalInfo	ErrorAdditionalInfo[]	The error additional info.
code	string	The error code.
details	ErrorDetail[]	The error details.
message	string	The error message.
target	string	The error target.

ErrorResponse

Object

Error response

Name	Type	Description
error	ErrorDetail	The error object.

LibraryInfo

Object

Information about a library/package created at the workspace level.

Name	Type	Description
containerName	string	Storage blob container name.
creatorId	string	Creator Id of the library/package.
name	string	Name of the library.
path	string	Storage blob path of library.
provisioningStatus	string	Provisioning status of the library/package.
type	string	Type of the library.
uploadedTimestamp	string (date-time)	The last update time of the library.

LibraryRequirements

Object

Spark pool library version requirements

Name	Type	Description
content	string	The library requirements.
filename	string	The filename of the library requirements file.
time	string (date-time)	The last update time of the library requirements file.

NodeSize

Enumeration

The level of compute power that each node in the Big Data pool has.

Value	Description
Large
Medium
None
Small
XLarge
XXLarge
XXXLarge

NodeSizeFamily

Enumeration

The kind of nodes that the Big Data pool provides.

Value	Description
HardwareAcceleratedFPGA
HardwareAcceleratedGPU
MemoryOptimized
None

SparkConfigProperties

Object

Spark pool Config Properties

Name	Type	Description
configurationType	ConfigurationType	The type of the spark config properties file.
content	string	The spark config properties.
filename	string	The filename of the spark config properties file.
time	string (date-time)	The last update time of the spark config properties file.

Share via

Big Data Pools - Create Or Update

URI Parameters

Request Body

Responses

Examples

Create or update a Big Data pool

Sample request

Sample response

Definitions

AutoPauseProperties

AutoScaleProperties

BigDataPoolResourceInfo

ConfigurationType

DynamicExecutorAllocation

ErrorAdditionalInfo

ErrorDetail

ErrorResponse

LibraryInfo

LibraryRequirements

NodeSize

NodeSizeFamily

SparkConfigProperties