How can I access the Synapse REST API from Spark notebooks in a Synapse workspace deployed with a private endpoint?

Hello , Welcome to MS Q&A

To enable private access to an Azure Synapse workspace via REST calls using Managed Private Endpoints and Private DNS, you'll first configure a Managed Private Endpoint within your Synapse workspace, which creates a private IP address in your Managed Virtual Network. Then, you'll integrate this endpoint with a private DNS zone to ensure that your application uses the private IP for communication. This setup allows secure, private access to your Synapse workspace without exposing it to the public internet. 

1. Here's a more detailed breakdown:
   1. 1. Enable Managed Virtual Network: If your Synapse workspace doesn't already have a Managed Virtual Network, ensure you enable it. This is a prerequisite for using Managed Private Endpoints. 
   2. 2. Create a Managed Private Endpoint:
      • Open your Azure Synapse workspace in the Azure portal. 
      • Navigate to the Managed private endpoints section within the Manage tab. 
      • Create a new Managed private endpoint, specifying the target resource (e.g., Azure Storage, Azure Cosmos DB, etc.). 
      • The endpoint will be assigned a private IP address within your Managed Virtual Network. 
   3. 3. Configure Private DNS Integration:
      • In your Synapse workspace's settings, locate the "Private endpoint connections" section under "Security". 
      • For each private endpoint connection, select "Integrate with private DNS zone" to link it to your existing or new private DNS zone. 
      • This will ensure that your application uses the private IP address when resolving the FQDN of your Synapse workspace. 
   4. 4. Update DNS settings:
      • You'll need to update your DNS settings to resolve the FQDN of your Synapse workspace to the private IP address of the Managed Private Endpoint. 
      • This can be done using Private DNS Zones, Azure Private Resolver, or by modifying the host file (for testing). 
   5. 5. Test the connection:
      • Use a REST client (like Postman or the curl command) to make requests to your Synapse workspace, ensuring that the connection is using the private IP address and DNS records. 

Pls check this link for ref -->https://docs.azure.cn/en-us/synapse-analytics/security/how-to-connect-to-workspace-with-private-links

Kindly let us know if any further ques

Pls accept if it helps

Thanks

Deepanshu

I think I need to clarify a bit. The issue isn't that I can't access the REST API through the private endpoint. I do have a private endpoint configured, and when I use Postman or python code from my local machine while connected to our corporate VPN, I can access the API without any problems. The issue only arises when I try to access the API from a Spark notebook inside Synapse, it throws an error saying, 'Public Access Disabled'.

Also, if I am connected to the corporate VPN and do

nslookup myworkspace.dev.azuresynapse.com in my laptop, it resolves to the proper private ip.

Edit: I am trying to access the same Synapse workspace through the REST API from within that Synapse environment itself.

@Kartik Rana

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

When Synapse is deployed with private endpoint and public network access disabled, the REST API must be accessed through a self-hosted integration runtime or from a compute that is also inside the same Virtual Network.

External internet access will not work.

You need to use a Virtual Network integrated client or private DNS resolution.

Although the Synapse workspace itself is private, Spark pools inside Synapse still reach out to the Synapse control plane (*.dev.azuresynapse.net) over public endpoints by design. When public network access is disabled, these calls are blocked even internally. Currently, the recommended solutions are to either use a self-hosted integration runtime inside the same Virtual Network, or route the API call through a Virtual Network-integrated service such as an Azure Function or Logic App. Direct REST API access from within Spark notebooks is not supported under these restrictions.

(Reference: Azure Synapse Analytics connectivity settings)

I guess we can close this, since there's currently no way to do this from a notebook. I'll try to perform the operation using Logic Apps, as suggested.

Also, is this something that might be supported in the future?