Just wanted to do a quick video to cover the process of deploying ADF artifacts from Development environment to higher environments like QA, etc. these are the basic steps to get it done (please watch the video for detailed instructions) https://youtu.be/7fsIwWheyDk Create a new project in Devops if there is no project set up alreadyMake … Continue reading How to Deploy Azure Data Factory (ADF) from Dev to QA using Devops
Now, before we get into it, I have to admit that I am somewhat of a Data Science sceptic. Not that I do not believe in Data Science, quite the opposite, I think it is a critical field in data analytics world that will eventually mature enough to be the dominant force in insight generation. … Continue reading We Don’t Need Data Scientists?!?
Should Microsoft be worried that Salesforce has just acquired Slack? In my opinion – YES, in fact it should be so worried that it must double its investments in Power BI.
This post is a beginning to a series of articles about building analytical capabilities in Azure using data lake, Databricks and Power BI. On the surface, those technologies seem like they were specifically designed to complement each other as they provide a set of foundational capabilities necessary to develop scalable and cost-effective business intelligence … Continue reading How to make Azure Databricks work with Azure Data Lake Storage Gen2 and Power BI
Recently, I had a chance to work with Azure Analysis Services (AS) sourcing data from Azure Data Warehouse (DW) external tables. Optimizing the processing of the Azure Analysis Services partitions to use with the Azure DW external tables is a bit different from working with the regular (physical) data tables, and I will discuss the … Continue reading Optimizing Azure Analysis Services Partitions for Azure Data Warehouse External Tables
Visual Studio 2017 and the SQL Server Data Tool (SSDT), a tool that comes with Visual Studio, finally integrates the now familiar Query Editor (Power Query) capability to work with the Analysis Services Tabular Model. Query Editor is a data ingestion and transformation tool used by Excel Power Pivot as well as Power BI. Now, … Continue reading Quirks of Visual Studio Query Editor working with Analysis Services
I have run into this a few times now and every time it took me a while to figure out what's going on, so I figured if I wrote about this on my blog, maybe I would not forget about it next time. Microsoft has a great article here that details how to setup Azure … Continue reading Giving access to Azure Active Directory Application to Enable Polybase
In an earlier post, we talked about self-service process to hydrate the Data Lake Store. We also mentioned the need to use Power Shell to load data files larger than 2Gb. Here is the Power Shell script: Provide your credentials to login to Azure: $MyAzureName = "<YourAzureUsername>"; $MyAzurePassword = ConvertTo-SecureString '<YourAzurePassword>' -AsPlainText -Force; $AzureRMCredential = New-Object System.Management.Automation.PSCredential($MyAzureName, … Continue reading Power Shell script to load data in to Data Lake Store
Azure Data Factory artifacts can be edited and deployed using the Azure portal. However, as an enterprise solution, one would want the capability to edit and publish these artifacts using Visual Studio. Let’s check are options available to publish using Visual Studio. Assuming you have the created a Data Factory project in Visual Studio and … Continue reading How to Publish Azure Data Factory Artifacts