Find Log Analytics Key of Azure Log Analytics Instance

 Issue: Find the keys to connect to a log analytics instance in Azure.

Solution:  You can find this under Log Analytics Workspace > Agents > Log Analytics Agent Instructions

Synapse Analytics ADF Error - Failed to encrypt sub-resource payload or linked service credentials [Solved]

Issue: While trying to publish an HTTP or REST linked service connection from master to live, we get an error if it is going through a self hosted Integration run time to an on-premise network. 

Publishing error

Failed to encrypt sub-resource payload

and error is: Failed to encrypted linked service credentials on self-hosted IR 'selfHostedIR', reason is: NotFound, error message is: No online instance..

Cause: When you have a HTTP call that happens inside an on-premise network and you use a self hosted integration run time to connect to this, the first time the linked service is published it has to validate the connection and hence requires the Self Hosted IR to be up and running. 

Solution: The solution is to have the self hosted IR( Azure VM in our case ) up and running while trying to publish from master to live. This enables the validation of the linked service connection, this fixed it for us.

Please do let us know if this fix worked for you. 

Storage Event Trigger Error - Register Azure Event Grid resource provider

Issue: Trying to create a storage event trigger fails in Synapse/ Azure Data Factory with the below error 

Register Azure Event Grid resource provider to your subscription before creating an event trigger.

Cause: This is caused because Azure Event Grid resource provider is not added or enabled under your Azure Subscription. 

If you are using Azure Synapse Analytics, then you also need to have the Microsoft.DataFactory resource provider regsitered.

Resolution: You can follow the steps to register event grid and data factory resource provider to your subscriptions.

Step 1: go to portal.azure.com and select the subscription where you want to create the storage event trigger.

Step 2: under the subscription go to settings > resource providers.

Step 3: Search and Register the two resource Providers 

Registering Microsoft.DataFactory resource provider

Registering Microsoft.EventGrid resource provider

Try again and now it should be working. Let us know if this worked for you.

Self Hosted Integration Runtime sharing between Synapse Workspaces

Scenario: We have to connect to an on-premise SQL Server/Oracle database and have three synapse workspaces in Azure, DEV, UAT and PROD in three different subscriptions. 

Current- Architecture

To connect to these sources on the on-premise networks we have virtual networks set up on Azure, gateways and site to site connections to local network.

The downside of this is having to maintain a lot more from an administrator perspective,

1. Three Virtual Networks 
2. Peering between three networks
3. Three seperate Virtual Machines to host the respective Azure Self Hosted IRs

Instead of this, we thought of simplifying it and having only one Virtual Network, one VM where self hosted integration runtime is hosted and the IR is shared by all three Synapse workspaces, see below architecture, 

Proposed Architecture with Shared Self Hosted IR.

Now comes the road-block here, sharing integration run-time is not yet available for Synapse Analytics ( reference - https://learn.microsoft.com/en-us/azure/synapse-analytics/data-integration/concepts-data-factory-differences#available-features-in-adf--azure-synapse-analytics  )

 

This seems to be something in the backlog for the Synapse Analytics Product team, I will be updating here if this comes out in any new releases. This will be really cool to have. 

Error 403 - Failed to load resources in Azure Synapse Studio

Error: User with Owner access at subscription level getting a failed error while opening Synapse Studio, and not able to see any pipelines, linked service etc

Failed to load one or more resources due to no access, error code 403

Root Cause: Even if you have owner level privileges at the subscription level, you still  have to assign rights inside Synapse Studio.

Fix: Go to Synapse Studio > Manage tab

Go to Access Control > Click on Add

Add the user you want to give Administrator access to 

Now log out and login back to Azure Portal and back to Synapse Studio, now you would be able to see all the resources. 

Authorization Failed error while creating Azure Data Factory Instance

Issue description: New user not able to create an Azure Data Factory instance despite having owner or contributor group access at resource group level.

Error - The client '' with object id '' does not have authorization to perform action '' over scope '/subscriptions/' or the scope is invalid. If access was recently granted, please refresh your credentials. (Code: AuthorizationFailed) (Code: AuthorizationFailed)  

Root Cause - The root cause of this issue is that, when you setup a subscription, only a default set of Microsoft Providers are by default registered. 

Fix: Go to Subscriptions > Resource Providers and add/register the 'Microsoft.DataFactory' provider.