Get started with Jupyter notebooks and MSTICPy in Microsoft Sentinel

This article describes how to run the Getting Started Guide For Microsoft Sentinel ML Notebooks notebook, which sets up basic configurations for running Jupyter notebooks in Microsoft Sentinel and provides examples for running simple queries.

The Getting Started Guide for Microsoft Sentinel ML Notebooks notebook uses MSTICPy, a powerful Python library designed to enhance security investigations and threat hunting within Microsoft Sentinel notebooks. It provides built-in tools for data enrichment, visualization, anomaly detection, and automated queries, helping analysts streamline their workflow without extensive custom coding.

For more information, see Use notebooks to power investigations and Use Jupyter notebooks to hunt for security threats.

Prerequisites

Before you begin, make sure you have the required permissions and resources.

Install and run the Getting Started Guide notebook

This procedure describes how to launch your notebook with Microsoft Sentinel.

1. For Microsoft Sentinel in the Defender portal, select Microsoft Sentinel > Threat management > Notebooks. For Microsoft Sentinel in the Azure portal, under Threat management, select Notebooks.

2. From the Templates tab, select A Getting Started Guide For Microsoft Sentinel ML Notebooks .

3. Select Create from template.

4. Edit the name and select the Azure Machine Learning workspace as appropriate.

5. Select Save to save it to your Azure Machine Learning workspace.

6. Select Launch notebook to run the notebook. The notebook contains a series of cells:

   • Markdown cells contain text and graphics with instructions for using the notebook
   • Code cells contain executable code that performs the notebook functions

7. At the top of the page, select your Compute.

8. Continue by reading markdown cells and running code cells in order, using the instructions in the notebook. Skipping cells or running them out of order might cause errors later in the notebook.

   Depending on the function being performed, the code in the cell might run quickly, or it might take some time to complete. When the cell is running, the play button changes to a loading spinner, and the status is displayed at the bottom of the cell, together with the elapsed time.

   The first time you run a code cell, it may several a few minutes to start the session, depending on your compute settings. A Ready indication is shown when the notebook is ready to run code cells. For example:

   

The Getting Started Guide For Microsoft Sentinel ML Notebooks notebook includes sections for the following activities:

The code in the Getting Started Guide For Microsoft Sentinel ML Notebooks launches the MpConfigEdit tool, which has series of tabs for configuring your notebook environment. As you make changes in MpConfigEdit tool, make sure to save your changes before continuing. Settings for the notebook are stored in the msticpyconfig.yaml file, which is automatically populated with initial details for your workspace.

Make sure to read through the markdown cells carefully so that you understand the process completely, including each of the settings and the msticpyconfig.yaml file. Next steps, extra resources, and frequently asked questions from the Azure Sentinel Notebooks wiki are linked from the end of the notebook.

Customize your queries (optional)

The Getting Started Guide For Microsoft Sentinel ML Notebooks notebook provides sample queries for you to use when learning about notebooks. Customize the built-in queries by adding more query logic, or run complete queries using the exec_query function. For example, most built-in queries support the add_query_items parameter, which you can use to append filters or other operations to the queries.

1. Run the following code cell to add a data frame that summarizes the number of alerts by alert name:

   from datetime import datetime, timedelta
   
   qry_prov.SecurityAlert.list_alerts(
      start=datetime.utcnow() - timedelta(28),
       end=datetime.utcnow(),
       add_query_items="| summarize NumAlerts=count() by AlertName"
   )
   

2. Pass a full Kusto Query Language (KQL) query string to the query provider. The query runs against the connected workspace, and the data returns as a panda DataFrame. Run:

   # Define your query
   test_query = """
   OfficeActivity
   | where TimeGenerated > ago(1d)
   | take 10
   """
   
   # Pass the query to your QueryProvider
   office_events_df = qry_prov.exec_query(test_query)
   display(office_events_df.head())
   
   

For more information, see:

• The MSTICPy query reference
• Running MSTICPy pre-defined queries

Apply guidance to other notebooks

The steps in this article describe how to run the Getting Started Guide for Microsoft Sentinel ML Notebooks notebook in your Azure Machine Learning workspace via Microsoft Sentinel. You can also use this article as guidance for performing similar steps to run notebooks in other environments, including locally.

Several Microsoft Sentinel notebooks don't use MSTICPy, such as the Credential Scanner notebooks, or the PowerShell and C# examples. Notebooks that don't use MSTICpy don't need the MSTICPy configuration described in this article.

Try out other Microsoft Sentinel notebooks, such as:

• Configuring your Notebook Environment
• A Tour of Cybersec notebook features
• Machine Learning in Notebooks Examples
• The Entity Explorer series, including variations for accounts, domains and URLs, IP addresses, and Linux or Windows hosts.

For more information, see:

• Jupyter notebooks with Microsoft Sentinel hunting capabilities
• Advanced configurations for Jupyter notebooks and MSTICPy in Microsoft Sentinel
• Create your first Microsoft Sentinel notebook (Blog series)
• Linux Host Explorer Notebook walkthrough (Blog)

Related content

For more information, see:

• MSTICPy documentation
• Jupyter Notebooks: An Introduction
• The Infosec Jupyterbook
• Why use Jupyter for Security Investigations
• Security Investigations with Microsoft Sentinel & Notebooks