Application lifecycle management for healthcare data solutions in Microsoft Fabric

Developing enterprise solutions is often complex and requires collaboration between multiple teams. This process includes working in development environments and deploying to downstream environments for testing and production. Microsoft Fabric provides tools to support application lifecycle management (ALM), including Git integration and deployment pipelines. These tools:

• Standardize communication and collaboration among development team members.
• Facilitate frequent product releases with continuous updates and efficient delivery methods.

For more information on Microsoft Fabric ALM support, see Lifecycle management in Microsoft Fabric. Review the prerequisite steps in Tutorial: Lifecycle management in Fabric.

Since healthcare data solutions in Microsoft Fabric are built on the Fabric platform, you can use the same ALM tools to implement the solutions. These tools help manage updates and additions to lakehouses, pipelines, and notebooks across development teams through source code control.

Workflow

Before deploying healthcare data solutions in Microsoft Fabric, your development teams should define a suitable ALM approach. Establishing a clear strategy early in the project ensures consistency, minimizes rework, and reduces errors during development and deployment to production environments.

Microsoft Fabric offers multiple ALM approaches that you can adopt. For more information, see Choose the best Fabric CI/CD workflow option for you. For healthcare data solutions in Microsoft Fabric, we recommend reviewing Option 3 - Deploy using Fabric deployment pipelines.

This option connects Git only during the development stage. After this stage, deployments occur directly between development, test, and production workspaces using Fabric deployment pipelines. During the development phase, the ALM helper notebook ensures that healthcare data solutions components remain in sync with feature branches until you commit them to the primary Git branch (master or main). When ready for downstream deployment, use the deployment pipeline APIs to integrate this process into an Azure release pipeline or GitHub workflow. These APIs support automated tests, approvals, and build and release processes similar to other ALM options.

We recommend this method when:

• You're using source control only for development purposes.
• Deployment rules and available APIs are sufficient to effectively manage configurations between stages.
• You want to use other functionalities such as viewing changes and deployment history in Fabric.

For more information about deployment pipelines in Microsoft Fabric, see What are deployment pipelines?

ALM helper notebook

Microsoft Fabric provides comprehensive ALM support, but you need some manual steps to run healthcare data solutions capabilities after syncing a workspace with ALM. To simplify these steps, healthcare data solutions provide another notebook called healthcare_msft_alm_helper.

Running the ALM helper notebook helps you:

• Automate reconfiguring deployment parameters
• Update notebook metadata
• Create delta tables
• Create folder structures identical to the bronze lakehouse folders
• Copy sample data

However, the notebook doesn't facilitate copying or moving any customer data. It also doesn't replicate folder structures found in the silver or gold lakehouses. The automation saves time, reduces errors, and streamlines the deployment of customized healthcare data solutions across environments. For example, during development, the ALM helper notebook helps you maintain environment consistency throughout feature development.

Run the ALM helper notebook periodically as you develop in feature branches and deploy to production. If you prefer running the notebook selectively, review the steps within the notebook to determine when it needs to run. For example, you might need a notebook run due to a late capability addition. You can bring the feature work into a workspace with the foundational capabilities deployed, deploy the OMOP capability, and then sync the workspaces using the Azure DevOps to Git integration feature. Because target feature workspaces won't initially include the OMOP lakehouse and its relevant tables, and the admin lakehouse deployment configuration is incomplete, running the ALM helper notebook is necessary. This run ensures that all workspaces remain in sync with the correct healthcare data solutions elements.

When you're ready for a release, use the ALM helper notebook during downstream workspace deployments, such as transitioning from development to staging, test, and production. You can either perform this process manually or invoke the notebook through the Fabric APIs. This workflow ensures that healthcare data solutions-specific elements deploy to the correct target workspaces.

For more information about this notebook and the execution instructions, see Healthcare data solutions in Microsoft Fabric ALM Helper.

To learn more about Microsoft Fabric API support for notebook execution, see Manage and execute notebooks in Fabric with APIs.

Prerequisites

• Deploy healthcare data solutions in Microsoft Fabric to the source workspace. This workspace is the starting point for pushing changes.
• Sync the source workspace to your ADO repository.
• Create a new destination workspace. This workspace receives updates.
• Sync the destination workspace to your ADO repository.

For more information on these prerequisite steps, see README.md in the healthcare data solutions ALM helper repository.

Download notebook

1. Go to the GitHub repository Healthcare data solutions in Microsoft Fabric ALM helper.

2. Select the file named sample-notebooks/healthcare_msft_alm_helper.ipynb.

3. From the file menu options, select Download raw file.

4. Save the file locally.

5. Alternately, sync the repository locally using Git tools such as GitHub Desktop or Visual Studio Code.

Import notebook

Follow these steps to import the notebook into your source workspace:

1. Open the source Fabric workspace.

2. From the top menu, select Import > Notebook > From this computer.

   

3. In the Import steps pane, select Upload.

4. Select the healthcare_msft_alm_helper.ipynb notebook downloaded in the previous section.

5. After importing the notebook, verify that it appears in your source workspace.

Run the notebook

After deploying the notebook, follow these steps to execute the code within the notebook:

1. Open the imported notebook.

2. In the Parameters code block, enter the values as described in the notebook instructions.

3. To locate the workspace identifier:

   1. Open the workspace main view.
   2. In the URL, select the GUID following the URL parameter groups. For example, https://app.powerbi.com/groups/11111111-2222-3333-4444-555555555555/list.

4. To locate the Lakehouse identifier:

   1. Open the lakehouse main view.
   2. In the URL, select the GUID following the URL parameter lakehouses. For example, https://app.powerbi.com/groups/11111111-2222-3333-4444-555555555555/lakehouses/55555555-4444-3333-2222-11111111111111.

5. Here's a sample:

   

6. Select the control next to the code block and select Run cell.

7. After completion, select Run cell on the code block in the section titled Helper methods.

8. Next, run the code blocks relevant to your deployment in the source workspace. For example, if you haven't deployed the healthcare data solutions sample data, you don't have to execute the code within Copy data assets.

Validate notebook results

After you execute each notebook section, validate the results by following these steps for each section:

• Save the deployment parameters to the destination workspace:

  1. Run and capture the log output from the code block in the Copy deployment parameters configuration section.
  2. Verify that the deploymentParametersConfiguration.json file in the target workspace admin lakehouse reflects values from the previous step.

• Copy system data:

  • In the logs following the code block for this section, verify that the output reports the following messages:
    • Successfully copied internal resources to destination workspace.
    • Successfully copied libraries to destination workspace.
  • In the target workspace, open the environment and verify that the custom files are copied.

• Copy data assets:

  • In the target workspace bronze lakehouse, verify that the data assets match the assets in the source workspace bronze lakehouse.

• Create folders:

  • In the target workspace bronze lakehouse, verify that the folders match the assets in the source workspace bronze lakehouse.

• Publish environment:

  • In the target workspace, open the Fabric environment item from the list. Verify that publishing is complete. Publishing generally takes several minutes to complete. During this process, you should see a banner indicating that the environment publishing is currently underway.

• Update notebooks (optional):

  • Open the target workspace and verify if the healthcare data solutions notebooks are updated to reference the correct parent workspace. Also, check if they're linked to the correct lakehouse.
  • The updated notebooks should be prefixed with the solution name. For example, healthcare1_notebook-name.

• Set up delta tables (optional):

  • In the target workspace, verify that the tables in each workspace match.

Tutorial

This section outlines steps to enable Git integration and deployment pipelines between source and target workspaces and highlights how to use the ALM helper notebook. The example demonstrates how to sync between a development (Dev) and a testing (Test) workspace and use the ALM helper notebook to synchronize healthcare data solutions specific components.

First, deploy healthcare data solutions to the Dev workspace, connect it to a Git repository, and commit the deployed resources. Next, deploy a Test workspace, connect it to the Git repository, and pull the committed resources. Then, deploy and execute the ALM helper notebook to ensure that all healthcare data solutions components are in sync. Finally, create a new notebook in the Dev workspace and commit the changes back to Azure DevOps for posting to the target workspace.

Step 1: Set up the Dev workspace

To prepare the workspace for Git integration and ALM support:

1. Create a new source workspace in your Fabric tenant. For this example, let's name it HDS ALM Dev.

2. Follow the instructions in Deploy healthcare data solutions to deploy the initial healthcare data solutions component to this workspace.

3. Before deploying the feature capabilities, view the workspace lineage after the initial deployment. On the workspace view, select the Lineage view button under Workspace settings.

   

4. You should see only the Healthcare item listed here.

   

Step 2: Set up Git connection

The following steps explain how to connect the source workspace to a Git repository. Make sure you set up a Git repository based on the guidance in Get started with Git integration.

1. Go to the source workspace and select Workspace settings > Git integration.

2. Under Git provider, select Azure DevOps > Connect.

3. In the Connect Git repository and branch pane, provide the following values:

   • Organization
   • Project
   • Git repository
   • Branch
   • Git folder

4. For this example, let's use a git folder named HLS_ALM_Sync and a branch named HLS_ALM.

   

5. Select Connect.

6. The workspace should now connect to your Git repository. Verify if the Git status value displays Synced for your healthcare data solutions component.

   

Step 3: Deploy healthcare data foundations and commit to Git

These steps explain how to deploy the foundational capability to the HDS ALM Dev workspace and commit the new components to Git.

1. In the HDS ALM Dev workspace, open your healthcare solution.

2. Select Setup your solution.

3. Follow the steps in Deploy healthcare data foundations to deploy the foundational components. For this exercise, you can skip deploying the sample data.

4. After the deployment, on the main workspace view, you should see the new items deployed with an updated Git status value. Because of these new updates, your healthcare solution git status value now changes to Uncommitted.

5. After deployment completes, select Source control.

   

6. In the source control pane, select all the items in the list, enter a commit message, and select Commit.

7. After committing, verify if you can see the updates posted to your Git repository. The repository now includes the newly deployed solution items.

   

Step 4: Deploy the Test workspace

The following steps explain how to deploy the HDS ALM Dev 2 Test workspace and sync the components from Git. After deployment, connect this workspace to the same Git repository used in Step 2: Set up Git connection, and pull the changes.

1. Create a new target workspace in your Fabric tenant. For this example, name it HDS ALM Dev 2.

2. In the target workspace, select Workspace settings > Git integration.

3. Under Git provider, select Azure DevOps > Connect.

4. In the Connect Git repository and branch pane, enter the following values. Make sure you provide the same Git repository details used in the HDS ALM Dev workspace.

   • Organization
   • Project
   • Git repository
   • Branch
   • Git folder

5. Select Connect.

6. After the sync completes, verify that the Git status value for your healthcare solution item displays Synced.

7. The previously committed healthcare solution items should now sync to the HDS ALM Dev 2 workspace.

Step 5: Deploy the ALM helper notebook

These steps explain how to deploy the ALM helper notebook that provides ALM support for Fabric and healthcare data solutions.

1. Go to the GitHub repository sample Healthcare data solutions in Microsoft Fabric ALM Helper.

2. Download the ALM helper notebook from the Git repository. See Download notebook.

3. Follow the repository instructions to deploy the notebook to the HDS ALM Dev workspace.

4. Open the notebook and complete the parameters section using values from the HDS ALM Dev and HDS ALM Dev 2 workspaces.

5. Navigate through the notebook and execute each step.

6. Go to the HDS ALM Dev 2 workspace and validate the updates. This validation depends on the executed notebook steps. For example, review if the HDS ALM Dev 2 workspace's healthcare#_msft_bronze lakehouse contains the relevant bronze folders.

Step 6: Update HDS ALM Dev and sync changes

The following steps explain how to create a new notebook in the HDS ALM Dev workspace and sync it to the target HDS ALM Dev 2 workspace.

1. In the HDS ALM Dev workspace, select New item and search for Notebook.

2. Select Notebook from the list of available items.

3. Open the new notebook and rename it to Hello World.

4. In the notebook code block, enter some code. For example, add print ('Hello world') and save the changes.

   

5. Select Source control to view the updated files.

   

6. Select the Hello World notebook, and select Commit.

7. In your Azure DevOps repository, verify if the Hello World notebook appears in the repository.

   

8. Open the HDS ALM Dev 2 workspace, and select Source control. View the available updates, including the Hello World notebook.

9. Select Update all.

10. View the incoming changes to the solution components.

11. Verify if the new Hello World notebook is available in the HDS ALM Dev 2 workspace.

Related information

• What is lifecycle management in Microsoft Fabric?
• What are deployment pipelines?
• Choose the best Fabric CI/CD workflow option for you