Azure Data Factory artifacts can be edited and deployed using the Azure portal. However, as an enterprise solution, one would want the capability to edit and publish these artifacts using Visual Studio. Let’s check are options available to publish using Visual Studio.
Assuming you have the created a Data Factory project in Visual Studio and extracted the artifacts to this project:
Right click on the Project Name in Solution Explorer and select Publish.
Sign in using your Azure credentials.
If you have Admin privileges on the Data Factory resource, you can select an existing Data Factory and proceed to Publish.
If you do not have Admin privileges, you cannot publish using Visual Studio
In an enterprise environment, you may not want all the developers to have admin privileges. But developers need the capability to publish changes. This can be achieved using PowerShell scripts.
Here are the steps:
- Open Azure Powershell
- Enter Add-AzureAccount. This will open a window to sign into Azure. Use your credentials.
- Enter Get-AzureSubscription to get a list of subscriptions
- Select-AzureSubscription <SubscriptionName> to select the subscription
- Navigate to the project folder where the Azure Data Factory artifacts are located using cd command
- Enter Login-AzureRmAccount and login
- Let’s create a variable so store the Resource Name and Azure Data Factory name
$var=Get-AzureRmDataFactory -ResourceGroupName <ResourceName> -Name <AzureDataFactoryName>
- To publish a dataset enter: New-AzureRmDataFactoryDataset $df -File .\<DatasetName>.json
If a dataset exists, a warning message appears. Select Y to continue to publish
- Similarly, New-AzureRmDataFactoryPipeline and New-AzureRmDataFactoryLinkedService can be used to publish a pipeline and linked service respectively
Here a few helpful resources:
Install Azure Powershell: https://azure.microsoft.com/en-us/documentation/articles/powershell-install-configure/
Azure Data Factory Cmdlets: https://msdn.microsoft.com/en-us/library/dn820234.aspx
business intelligist 