Create, configure, and use an environment in Fabric

A Microsoft Fabric environment is a consolidated item for all your hardware and software settings. In an environment, you can select different Spark runtimes, configure your compute resources, install libraries from public repositories or a local directory, and set other settings.

This article presents an overview of how to create, configure, and use an environment.

Create an environment

You can create new environments from multiple entry points:

• Standard entry point: In the creation hub, or the New section of your workspace, you can find the option to create new environments like other Fabric items.
• Create during selection: When you select the environment for your notebook, Spark job definition, and the workspace default, you can find the option to create a new environment.

Configure an environment

An environment has three major components:

• Spark compute, which includes Spark runtime.
• Libraries.
• Resources.

The Spark compute and library configurations are required for publishing to be effective. Resources are shared storage that can change in real time. For more information, see Save and publish changes.

Configure Spark compute

For an environment, you can choose from various Spark runtimes with their own default settings and preinstalled packages. To view the available runtimes, go to the Home tab of the environment and select Runtime. Select the runtime that best suits your needs.

Fabric Spark compute provides unparalleled speed and efficiency running on Spark and requirement-tailored experiences. In your environment, you can choose from various pools created by workspace admins and capacity admins. You can further adjust the configurations and manage Spark properties to be effective in Spark sessions. For more information, see Spark compute configuration settings in Fabric environments.

Manage libraries

Each Spark runtime provides built-in libraries. With the Fabric environment, you can also install libraries from public sources or upload custom libraries that you or your organization built. After you successfully install the libraries, they're available in your Spark sessions. For more information, see Library management in Fabric environments. For the best practices for managing libraries in Fabric, see Manage Apache Spark libraries in Fabric.

Use resources

The Resources section in an environment facilitates the ability to manage small resources during the development phase. Files uploaded to the environment are accessible across notebooks when they're attached. For more information, see Manage the resources in a Fabric environment.

Save and publish changes

On the Home tab, you can easily find Save and Publish. They're activated when there are unsaved or unpublished pending changes in the Libraries and Spark compute sections.

When pending changes are in the Libraries and Spark compute sections, you also see a banner that prompts you with Save and Publish. The functionalities are the same as for the buttons:

• The unsaved changes are lost if you refresh or leave the browser open. Select Save to make sure that your changes are recorded before you leave. Saving doesn't apply the configuration but caches the changes in the system.
• To apply the changes to Libraries and Spark compute, select Publish. The Pending changes page appears for final review before publishing. Next, select Publish all to initiate configuration in the Fabric environment. This process might take some time, especially if library changes are involved.
• To cancel a publishing process, select View progress in the banner and cancel the operation.
• A notification appears upon publishing completion. An error notification occurs if there are any issues during the process.

An environment accepts only one Publish action at a time. You can't make changes to the Libraries or Spark compute sections during an ongoing Publish action.

Publishing doesn't affect adding, deleting, or editing the files and folders in the Resources section. The actions to manage resources are in real time. The Publish action doesn't block changes in the Resources section.

Share an existing environment

Fabric supports sharing an item with different levels of permissions.

When you share an environment item, recipients automatically receive Read permission. With this permission, they can explore the environment's configurations and attach it to notebooks or Spark jobs. For smooth code execution, make sure to grant Read permissions for attached environments when you share notebooks and Spark job definitions.

You can also share the environment with Share and Edit permissions. Users with Share permission can continue sharing the environment with others. Meanwhile, recipients with Edit permission can update the environment's content.

Attach an environment

You can attach a Fabric environment to your Data Engineering/Science workspaces or your notebooks and Spark job definitions.

Attach an environment as a workspace default

Select Workspace settings > Data Engineering/Science > Spark settings to see the Environment tab.

Workspace admins can define the default workload for entire workspaces. The values configured here are effective for notebooks and Spark job definitions that attach to Workspace settings.

The Set default environment toggle can enhance the user experience. By default, this toggle is set to Off. If there's no default Spark property or library required as the workspace default, you can define the Spark runtime in this circumstance. If you want to prepare a default Spark compute and libraries for the workspace, you can switch the toggle to On and easily attach an environment as the workspace default. This option makes all configurations in the environment effective as the Workspace settings.

Attach an environment to a notebook or a Spark job definition

The environment is available on both the Notebook and Spark Job Definition tabs. When notebooks and Spark job definitions are attached to an environment, they can access its libraries, compute configurations, and resources. The Explorer lists all available environments that are shared with you, are from the current workspace, and are from other workspaces to which you have access.

If you switch to a different environment during an active session, the newly selected environment doesn't take effect until the next session.

When you attach an environment from another workspace, both workspaces must have the same capacity and network security settings. Although you can select environments from workspaces with different capacities or network security settings, the session fails to start.

When you attach an environment from another workspace, the compute configuration in that environment is ignored. Instead, the pool and compute configurations default to the settings of your current workspace.

Related content

• Spark compute configuration settings in Fabric environments
• Library management in Fabric environments