Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
This guide is for Azure Data Factory and Synapse Gen2 pipeline creators. It explains how to modernize and migrate your pipelines to Data Factory in Microsoft Fabric.
Overview
Microsoft Fabric is a unified platform for self-service and IT-managed enterprise data. It supports scalable, secure, and accessible solutions for organizations of all sizes.
Data Factory in Fabric offers advanced features and capabilities, enabling data integration developers to consolidate their solutions within Fabric. This guide answers common questions about migrating to Fabric pipelines, including:
- What features are available in Fabric pipelines?
- How do Fabric pipelines differ from Azure Data Factory?
- How do you migrate existing pipelines?
Note
The decision to migrate depends on your specific requirements. Evaluate the benefits carefully to make an informed choice. Currently, there are no plans to deprecate Azure Data Factory or Synapse Gen2 for data ingestion. However, Fabric pipelines are the focus for enterprise data ingestion.
Enterprise and self-service data integration
Fabric simplifies discovery, collaboration, and management by bringing resources together. Central IT teams can operationalize data movement and transformation services while integrating self-service tools. Key features include:
- Office 365 and Teams activities for communication.
- Seamless Power BI model refreshes.
- Governance tools for data lineage and monitoring.
- Generative AI with Copilot for intelligent pipeline creation and error resolution.
This shared platform streamlines workflows, helping organizations scale their data solutions efficiently.
Fabric capacities
Fabric's distributed architecture handles high loads, spikes, and concurrency effectively. Larger Fabric capacity SKUs offer better performance and throughput.
Feature comparison
Here’s a comparison of features in Azure Data Factory and Fabric Data Factory:
| Feature | Azure Data Factory | Fabric Data Factory |
|---|---|---|
| Pipeline activities | ||
| Office 365 and Teams activities to send messages and support collaboration | No | Yes |
| Connections to Power BI semantic models and Dataflow Gen2 for consistent data refreshes | No | Yes |
| Validation to ensure pipelines only run after meeting criteria or timing out | Yes | Yes1 |
| Execute SQL Server Integration Services (SSIS) packages for data integration | Yes | Planned |
| Data transformation | ||
| Visual data transformations with Apache Spark clusters using Mapping Dataflows | Yes | No2 |
| Visual data transformations with the Fabric compute engine using Power Query in Dataflow Gen2 | No | Yes |
| Connectivity | ||
| Support for all Data Factory data sources | Yes | In progress3 |
| Scalability | ||
| Scheduled runs for seamless pipeline execution | Yes | Yes |
| Multiple runs for a single pipeline | Yes | Planned |
| Tumbling window triggers for nonoverlapping time windows | Yes | Planned |
| Event triggers to automate pipeline runs based on events | Yes | Yes4 |
| Artificial intelligence | ||
| Copilot for Data Factory for intelligent pipeline generation and error suggestions | No | Yes |
| Content management | ||
| Data lineage view to understand pipeline dependencies | No | Yes |
| Deployment pipelines to manage content lifecycles | No | Yes |
| Platform scalability and resiliency | ||
| Premium capacity for increased scale and performance | No | Yes |
| Multi-Geo support for regional and organizational data residency needs | Yes | Yes |
| Security | ||
| Virtual network (VNet) connectivity for seamless integration | No | Yes |
| On-premises data gateway for secure access to on-premises data | No | Yes |
| Azure service tags for simplified network security rule updates | Yes | Yes |
| Governance | ||
| Content endorsement to promote high-quality items | No | Yes |
| Microsoft Purview integration for managing and governing items | Yes | Yes |
| Microsoft Purview Information Protection sensitivity labels and Microsoft Defender for Cloud Apps integration for data loss prevention | No | Yes |
| Monitoring and diagnostic logging | ||
| Log pipeline execution events for monitoring and troubleshooting | Yes | Planned |
| Monitoring hub for tracking Fabric items | No | Yes |
| Microsoft Fabric Capacity Metrics app for monitoring capacities | No | Yes |
| Audit log to track user activities across Fabric and Microsoft 365 | No | Yes |
1 Use the Get metadata, Until, and If condition activities for equivalent metadata retrieval of the Validation activity.
2 Use the Invoke remote pipeline function to execute mapping data flow activities. See the Invoke Pipeline activity.
3 For supported connectors, see Pipeline support.
4 Use Fabric eventstreams and Reflex to trigger execution run events. See Pipeline event triggers.
Considerations
There are some other considerations to factor into your planning before migrating to Fabric pipelines.
Licensing
Fabric pipelines require at least a Microsoft Fabric (Free) license to author in a premium capacity workspace. Learn more in Fabric licenses.
Roadmap
For updates and planned features, see the Microsoft Fabric release plan documentation.