Azure Data Factory (ADF) is a fully managed, serverless data integration solution for ingesting, preparing, and transforming all your data at scale. It enables every organization in every industry to use it for a rich variety of use cases: data Engineering, migrating their on-premises SSIS packages to Azure, operational data integration, analytics, ingesting data into data warehouses, and more.
ADF is an essential service in all data related activities in Azure. It is flexible and powerful Platform as a Service, offering multitude of connectors and integration capabilities. It is a heart of ETL in Azure. This tool can support extremely large volume data. We often propose and use it in Dynamics 365 implementations as a core service with data migration or during the lifecycle of the project itself.
In the world of big data, raw, unorganized data is often stored in relational, non-relational, and other storage systems. However, on its own, raw data doesn't have the proper context or meaning to provide meaningful insights to analysts, data scientists, or business decision makers.
Big data requires service that can orchestrate and operationalize processes to refine these enormous stores of raw data into actionable business insights. Azure Data Factory is a managed cloud service that's built for these complex hybrid extract-transform-load (ETL), extract-load-transform (ELT), and data integration projects.
ADF is always the good place also for such Hybrid/On Premise scenarios where you can setup multiple data sources with such kind of data structures / connectors : including famous Azure Data Lake Storage :
In a way, ADF is also the good product that make easier to build SQL Server Integration Services (SSIS) pipeline. In fact, you could also do both in a single Data Pipeline for Dynamics 365 projects.
Logic Apps are very similar to Power Automate ; but for us it has more flexibility on ALM Process such as the deployment part and Azure Dev Ops feature : including version control and CI/CD : that’s why for integration process we would always go through Logic Apps rather than Power Automate. But you can also export any flow from Power Automate you already created to transfer it on Logic Apps template.
Meaning, you need to be careful also for the governance side, and Logic Apps are more accurate to be used for small volumes - transactional process with few basic transformations. (like Power Automate)
Logic Apps are more powerful in terms of connectors than ADF : (450 versus 90 for ADF)
To summarize - Azure Logic Apps is a cloud service that helps you schedule, automate, and orchestrate tasks, business processes, and workflows when you need to integrate apps, data, systems, and services across enterprises or organizations. Logic Apps simplifies how you design and build scalable solutions for app integration, data integration, system integration, enterprise application integration (EAI), and B2B communication, whether in the cloud, on premises, or both. All ADF or Logic Apps can be added to an ALM process with GIT Version control on Azure Dev Ops.
For Logic Apps the price can be explored and calculated very quickly and easily. But be very careful, our advice is - go straight to the Integration Service environment dedicated for you. Otherwise, you will be in a shared global pool of Logic Apps. In term of response time and since Logic Apps is more attending for Synchronous fast flow, the price can go up very quickly - as well as if you have multiple actions and connectors (premium one) in multiple flows.
The Azure Data Factory price list is much more complex than Logic Apps. But it also very cheap, especially if you need those data pipelines only for a data migration project. You pay per use (like a lot of Azure components) but it's easier to run batches to extract, transform and load billions of rows data coming from different sources.
Like Logic Apps, you could deploy your own self-hosted integration runtime, and not a shared one with multiple customers. Consider also to have multiple Virtual Machines in Scale-set and High Availability.
Many resources are available on Azure Docs and Azure Data Factory YouTube Channel to help the technical community get started with ADF. In addition, you can get started by using the Azure Data Factory Learning Path (part of Azure Learn). Or, contact us and let us do the leg work!