Reza Rad (RADACAD) [MVP]

Data Analytics MyRole

Hot News Focus

Microsoft Fabric

[URL] Worldwide Content Releastionship

<div><div data-sp-canvascontrol="" data-sp-canvasdataversion="1.0" data-sp-controldata="&#123;&quot;position&quot;&#58;&#123;&quot;layoutIndex&quot;&#58;1,&quot;zoneIndex&quot;&#58;1,&quot;sectionIndex&quot;&#58;1,&quot;sectionFactor&quot;&#58;12,&quot;controlIndex&quot;&#58;0&#125;,&quot;controlType&quot;&#58;3,&quot;id&quot;&#58;&quot;2eb8b735-c0fb-4b90-8cf9-49e5afcb5068&quot;,&quot;webPartId&quot;&#58;&quot;d1d91016-032f-456d-98a4-721247c305e8&quot;,&quot;rteInstanceId&quot;&#58;&quot;92cfee40-9667-4fc6-ba55-2bc80b1c8d58&quot;,&quot;addedFromPersistedData&quot;&#58;true,&quot;reservedHeight&quot;&#58;659,&quot;reservedWidth&quot;&#58;1178&#125;"><div data-sp-webpart="" data-sp-webpartdataversion="1.11" data-sp-webpartdata="&#123;&quot;id&quot;&#58;&quot;d1d91016-032f-456d-98a4-721247c305e8&quot;,&quot;instanceId&quot;&#58;&quot;2eb8b735-c0fb-4b90-8cf9-49e5afcb5068&quot;,&quot;title&quot;&#58;&quot;Image&quot;,&quot;description&quot;&#58;&quot;Add an image, picture or photo to your page including text overlays and ability to crop and resize images.&quot;,&quot;audiences&quot;&#58;[],&quot;serverProcessedContent&quot;&#58;&#123;&quot;htmlStrings&quot;&#58;&#123;&#125;,&quot;searchablePlainTexts&quot;&#58;&#123;&#125;,&quot;imageSources&quot;&#58;&#123;&quot;imageSource&quot;&#58;&quot;/sites/M365-CDN/SiteAssets/SitePages/Content-21558/4249919829-workloads-access-data.jpeg&quot;&#125;,&quot;links&quot;&#58;&#123;&#125;,&quot;customMetadata&quot;&#58;&#123;&quot;imageSource&quot;&#58;&#123;&quot;siteId&quot;&#58;&quot;40ef054e-5ec7-4552-ac92-09bbd4faa5e1&quot;,&quot;webId&quot;&#58;&quot;35f81bac-1b1d-4f8c-a038-0302a1a6ae55&quot;,&quot;listId&quot;&#58;&quot;870ffa3e-eb32-4905-82ab-9e57adb99968&quot;,&quot;uniqueId&quot;&#58;&quot;a21f13aa-f296-4646-a7f3-aeb7440f7f4d&quot;,&quot;width&quot;&#58;1100,&quot;height&quot;&#58;633&#125;&#125;&#125;,&quot;dataVersion&quot;&#58;&quot;1.11&quot;,&quot;properties&quot;&#58;&#123;&quot;id&quot;&#58;&quot;d3c857d9-df74-48b1-8a2a-dedb1a6e9105&quot;,&quot;linkUrl&quot;&#58;&quot;&quot;,&quot;isInlineImage&quot;&#58;true,&quot;siteId&quot;&#58;&quot;40ef054e-5ec7-4552-ac92-09bbd4faa5e1&quot;,&quot;webId&quot;&#58;&quot;35f81bac-1b1d-4f8c-a038-0302a1a6ae55&quot;,&quot;listId&quot;&#58;&quot;870ffa3e-eb32-4905-82ab-9e57adb99968&quot;,&quot;uniqueId&quot;&#58;&quot;a21f13aa-f296-4646-a7f3-aeb7440f7f4d&quot;,&quot;imgHeight&quot;&#58;633,&quot;imgWidth&quot;&#58;1100,&quot;imageSourceType&quot;&#58;2,&quot;alignment&quot;&#58;&quot;Center&quot;,&quot;fileName&quot;&#58;&quot;workloads-access-data.jpeg&quot;,&quot;fixAspectRatio&quot;&#58;false,&quot;altText&quot;&#58;&quot;&quot;&#125;,&quot;containsDynamicDataSource&quot;&#58;false&#125;"><div data-sp-componentid="d1d91016-032f-456d-98a4-721247c305e8"></div><div data-sp-htmlproperties=""><img data-sp-prop-name="imageSource" src="/sites/M365-CDN/SiteAssets/SitePages/Content-21558/4249919829-workloads-access-data.jpeg" data-sp-cust-siteId="40ef054e-5ec7-4552-ac92-09bbd4faa5e1" data-sp-cust-webId="35f81bac-1b1d-4f8c-a038-0302a1a6ae55" data-sp-cust-listId="870ffa3e-eb32-4905-82ab-9e57adb99968" data-sp-cust-uniqueId="a21f13aa-f296-4646-a7f3-aeb7440f7f4d" data-sp-cust-width="1100" data-sp-cust-height="633" /></div></div></div><div data-sp-canvascontrol="" data-sp-canvasdataversion="1.0" data-sp-controldata="&#123;&quot;position&quot;&#58;&#123;&quot;layoutIndex&quot;&#58;1,&quot;zoneIndex&quot;&#58;1,&quot;sectionIndex&quot;&#58;1,&quot;sectionFactor&quot;&#58;12,&quot;controlIndex&quot;&#58;0&#125;,&quot;controlType&quot;&#58;3,&quot;id&quot;&#58;&quot;e8d4ba55-a523-4395-b2b0-0752e5cd079b&quot;,&quot;webPartId&quot;&#58;&quot;d1d91016-032f-456d-98a4-721247c305e8&quot;,&quot;rteInstanceId&quot;&#58;&quot;92cfee40-9667-4fc6-ba55-2bc80b1c8d58&quot;,&quot;addedFromPersistedData&quot;&#58;true,&quot;reservedHeight&quot;&#58;464,&quot;reservedWidth&quot;&#58;1178&#125;"><div data-sp-webpart="" data-sp-webpartdataversion="1.11" data-sp-webpartdata="&#123;&quot;id&quot;&#58;&quot;d1d91016-032f-456d-98a4-721247c305e8&quot;,&quot;instanceId&quot;&#58;&quot;e8d4ba55-a523-4395-b2b0-0752e5cd079b&quot;,&quot;title&quot;&#58;&quot;Image&quot;,&quot;description&quot;&#58;&quot;Add an image, picture or photo to your page including text overlays and ability to crop and resize images.&quot;,&quot;audiences&quot;&#58;[],&quot;serverProcessedContent&quot;&#58;&#123;&quot;htmlStrings&quot;&#58;&#123;&#125;,&quot;searchablePlainTexts&quot;&#58;&#123;&#125;,&quot;imageSources&quot;&#58;&#123;&quot;imageSource&quot;&#58;&quot;/sites/M365-CDN/SiteAssets/SitePages/Content-21558/1132096484-Datawarehouse.jpg&quot;&#125;,&quot;links&quot;&#58;&#123;&#125;,&quot;customMetadata&quot;&#58;&#123;&quot;imageSource&quot;&#58;&#123;&quot;siteId&quot;&#58;&quot;40ef054e-5ec7-4552-ac92-09bbd4faa5e1&quot;,&quot;webId&quot;&#58;&quot;35f81bac-1b1d-4f8c-a038-0302a1a6ae55&quot;,&quot;listId&quot;&#58;&quot;870ffa3e-eb32-4905-82ab-9e57adb99968&quot;,&quot;uniqueId&quot;&#58;&quot;3c6ab195-6c43-4fd1-8d19-7ef9b7e11cea&quot;,&quot;width&quot;&#58;2048,&quot;height&quot;&#58;1152&#125;&#125;&#125;,&quot;dataVersion&quot;&#58;&quot;1.11&quot;,&quot;properties&quot;&#58;&#123;&quot;id&quot;&#58;&quot;b75d81ff-357e-46cd-b6fe-83cbe6c7a09d&quot;,&quot;linkUrl&quot;&#58;&quot;&quot;,&quot;isInlineImage&quot;&#58;true,&quot;siteId&quot;&#58;&quot;40ef054e-5ec7-4552-ac92-09bbd4faa5e1&quot;,&quot;webId&quot;&#58;&quot;35f81bac-1b1d-4f8c-a038-0302a1a6ae55&quot;,&quot;listId&quot;&#58;&quot;870ffa3e-eb32-4905-82ab-9e57adb99968&quot;,&quot;uniqueId&quot;&#58;&quot;3c6ab195-6c43-4fd1-8d19-7ef9b7e11cea&quot;,&quot;imgHeight&quot;&#58;1152,&quot;imgWidth&quot;&#58;2048,&quot;imageSourceType&quot;&#58;2,&quot;resizeCoefficient&quot;&#58;0.6604414261460102,&quot;resizeDesiredWidth&quot;&#58;778,&quot;alignment&quot;&#58;&quot;Center&quot;,&quot;fileName&quot;&#58;&quot;Datawarehouse.jpg&quot;,&quot;fixAspectRatio&quot;&#58;false,&quot;altText&quot;&#58;&quot;&quot;&#125;,&quot;containsDynamicDataSource&quot;&#58;false&#125;"><div data-sp-componentid="d1d91016-032f-456d-98a4-721247c305e8"></div><div data-sp-htmlproperties=""><img data-sp-prop-name="imageSource" src="/sites/M365-CDN/SiteAssets/SitePages/Content-21558/1132096484-Datawarehouse.jpg" data-sp-cust-siteId="40ef054e-5ec7-4552-ac92-09bbd4faa5e1" data-sp-cust-webId="35f81bac-1b1d-4f8c-a038-0302a1a6ae55" data-sp-cust-listId="870ffa3e-eb32-4905-82ab-9e57adb99968" data-sp-cust-uniqueId="3c6ab195-6c43-4fd1-8d19-7ef9b7e11cea" data-sp-cust-width="2048" data-sp-cust-height="1152" /></div></div></div><div data-sp-canvascontrol="" data-sp-canvasdataversion="1.0" data-sp-controldata="&#123;&quot;controlType&quot;&#58;4,&quot;id&quot;&#58;&quot;92cfee40-9667-4fc6-ba55-2bc80b1c8d58&quot;,&quot;position&quot;&#58;&#123;&quot;layoutIndex&quot;&#58;1,&quot;zoneIndex&quot;&#58;1,&quot;sectionIndex&quot;&#58;1,&quot;sectionFactor&quot;&#58;12,&quot;controlIndex&quot;&#58;1&#125;,&quot;addedFromPersistedData&quot;&#58;true&#125;"><div data-sp-rte=""><p>Data Factory in Microsoft Fabric integrates Azure Data Factory (ADF) and Power Query Dataflows into a single, comprehensive data integration solution. ADF, traditionally a data ingestion tool with powerful control flow capabilities, gets a transformational upgrade through the integration with Power Query. Power Query simplifies data transformations with its user-friendly interface and powerful data transformation language, M.</p><p><br></p><p>This fusion in Data Factory effectively eliminates the earlier trade-off between simplicity and scalability in data transformations. Now, both Power Query's transformation abilities and ADF's control flow capabilities and scalability can be leveraged simultaneously, making Data Factory a state-of-the-art ETL software-as-a-service offering.</p><p><br></p><p>Data Factory comes equipped with a wide range of data connectors, catering to numerous data sources. This vast selection enables Data Factory to extract data from diverse locations, making it a versatile tool in data integration. The creation of dataflows within Data Factory involves the transformation and loading of this extracted data into the desired destinations.</p><div tabindex="-1" data-cke-widget-wrapper="1" data-cke-filter="off" class="cke_widget_wrapper cke_widget_block cke_widget_inlineimage cke_widget_wrapper_webPartInRteInlineImage cke_widget_wrapper_webPartInRteClear cke_widget_wrapper_webPartInRteAlignCenter cke_widget_wrapper_webPartInRte" data-cke-display-name="div" data-cke-widget-id="1" role="region" aria-label="Inline image in RTE. Use Alt + F11 to go to toolbar. Use Alt + P to open the property pane."><div data-webpart-id="image" class="webPartInRte webPartInRteAlignCenter webPartInRteClear webPartInRteInlineImage cke_widget_element" data-cke-widget-data="%7B%22classes%22%3A%7B%22webPartInRteInlineImage%22%3A1%2C%22webPartInRteClear%22%3A1%2C%22webPartInRteAlignCenter%22%3A1%2C%22webPartInRte%22%3A1%7D%7D" data-cke-widget-upcasted="1" data-cke-widget-keep-attr="0" data-widget="inlineimage" data-instance-id="2eb8b735-c0fb-4b90-8cf9-49e5afcb5068" title=""></div></div><p>Dataflows can be further enhanced by embedding them within Data Pipelines. A Data Pipeline is essentially a control flow execution component. The inclusion of Data Pipelines brings added control and monitoring to the ETL process, enhancing the efficiency and transparency of data integration tasks within the Microsoft Fabric ecosystem.</p><p><br></p><ul><li><a data-cke-saved-href="https&#58;//radacad.com/what-is-microsoft-fabric-and-why-it-is-a-big-deal" href="https&#58;//radacad.com/what-is-microsoft-fabric-and-why-it-is-a-big-deal">What is Microsoft Fabric</a>&#160;</li><li><a data-cke-saved-href="https&#58;//radacad.com/enabling-microsoft-fabric-in-your-tenant" href="https&#58;//radacad.com/enabling-microsoft-fabric-in-your-tenant">How to enable Fabric in your organization</a>&#160;</li><li><a data-cke-saved-href="https&#58;//radacad.com/microsoft-fabric-lakehouse" href="https&#58;//radacad.com/microsoft-fabric-lakehouse">What is Lakehouse, and a sample Dataflow Gen2 created to load data into it.</a>&#160;</li><li><a data-cke-saved-href="https&#58;//radacad.com/getting-started-with-data-pipelines-in-fabric-data-factory" href="https&#58;//radacad.com/getting-started-with-data-pipelines-in-fabric-data-factory">Getting started with Data Pipelines in Microsoft Fabric Data Factory</a>&#160;</li><li><a data-cke-saved-href="https&#58;//radacad.com/getting-started-with-dataflow-in-microsoft-fabric-data-factory" href="https&#58;//radacad.com/getting-started-with-dataflow-in-microsoft-fabric-data-factory">Getting started with Dataflows in Microsoft Fabric Data Factory</a>&#160;</li></ul><p><br></p><h3><b>Microsoft Fabric&#58; A Versatile Data A</b><b>nalytics Solution</b></h3><p>Microsoft Fabric is an end-to-end Data Analytics software-as-a-service offering from Microsoft. With its workloads covering various aspects of data analytics, it provides a unified, easy-to-use platform that can accommodate a variety of business intelligence and data analysis needs.</p><p>Microsoft Fabric encapsulates several Microsoft services, including the powerful Data Factory, which serves as the data integration component. Data Factory draws on the strengths of Azure Data Factory and Power Query Dataflows to ensure seamless, scalable, and efficient data integration.</p><p>Microsoft Fabric provides an integrated environment for managing and analyzing data, enabling organizations to gain insights and make data-driven decisions. Whether it's data integration, data warehousing, or data visualization, Microsoft Fabric offers a comprehensive suite of services to manage your end-to-end data journey.</p><p><br></p><h3><b>Understanding Azure Data F</b><b>actory and Power Query Dataflows</b></h3><p>To fully leverage the power of Data Factory in Microsoft Fabric, it's vital to understand the foundational components it's built upon - Azure Data Factory and Power Query Dataflows.</p><p>Azure Data Factory, initially developed as an in-cloud ETL solution, has evolved into a comprehensive data integration service. Although it lacked a robust data transformation engine at its inception, it has been extremely effective for data ingestion. Data Pipelines, a key component of ADF, allows for the execution of a group of activities as a flow, providing a strong foundation for control flow capabilities in Data Factory.</p><p>On the other hand, Power Query Dataflows was designed to provide a user-friendly data transformation engine for Power BI. However, it has now evolved beyond Power BI to form an integral part of the Power Platform, serving both data transformation and migration needs.</p><p>In Data Factory, these two technologies are combined, resulting in an enhanced ETL service that not only enables scalable data ingestion and transformation but also provides superior control over the ETL execution process.</p><div tabindex="-1" data-cke-widget-wrapper="1" data-cke-filter="off" class="cke_widget_wrapper cke_widget_block cke_widget_inlineimage cke_widget_wrapper_webPartInRteInlineImage cke_widget_wrapper_webPartInRteClear cke_widget_wrapper_webPartInRteAlignCenter cke_widget_wrapper_webPartInRte cke_widget_focused cke_widget_selected" data-cke-display-name="div" data-cke-widget-id="2" role="region" aria-label="Inline image in RTE. Use Alt + F11 to go to toolbar. Use Alt + P to open the property pane."><div data-webpart-id="image" class="webPartInRte webPartInRteAlignCenter webPartInRteClear webPartInRteInlineImage cke_widget_element" data-cke-widget-data="%7B%22classes%22%3A%7B%22webPartInRteInlineImage%22%3A1%2C%22webPartInRteClear%22%3A1%2C%22webPartInRteAlignCenter%22%3A1%2C%22webPartInRte%22%3A1%7D%7D" data-cke-widget-upcasted="1" data-cke-widget-keep-attr="0" data-widget="inlineimage" data-instance-id="e8d4ba55-a523-4395-b2b0-0752e5cd079b" title=""></div></div><h3><b>Deepening </b><b>Your Knowledge on Data Factory</b></h3><p>To make the best use of Data Factory in Microsoft Fabric, one needs to dig deeper into its functionalities and capabilities. Start by understanding its foundational elements, Azure Data Factory and Power Query Dataflows, and how they contribute to the functioning of Data Factory.&#160;</p><p>Explore the range of data connectors available and understand how to leverage them for different data sources. Learn how to create and manage Dataflows using the Power Query Editor online, and familiarize yourself with the M language for advanced data transformations.</p><p>Understanding the workings of Data Pipelines and their role in the control flow of the ETL process is equally important. Learn about the</p><p><br></p><h3><a data-cke-saved-href="https&#58;//radacad.com/what-is-data-factory-in-microsoft-fabric" href="https&#58;//radacad.com/what-is-data-factory-in-microsoft-fabric">https&#58;//radacad.com/what-is-data-factory-in-microsoft-fabric</a></h3></div></div><div data-sp-canvascontrol="" data-sp-canvasdataversion="1.0" data-sp-controldata="&#123;&quot;position&quot;&#58;&#123;&quot;layoutIndex&quot;&#58;1,&quot;zoneIndex&quot;&#58;1,&quot;sectionIndex&quot;&#58;1,&quot;sectionFactor&quot;&#58;12,&quot;controlIndex&quot;&#58;2&#125;,&quot;controlType&quot;&#58;3,&quot;id&quot;&#58;&quot;62b390f6-2677-47e8-9084-5a958edca2b6&quot;,&quot;webPartId&quot;&#58;&quot;6410b3b6-d440-4663-8744-378976dc041e&quot;,&quot;addedFromPersistedData&quot;&#58;true,&quot;reservedHeight&quot;&#58;177,&quot;reservedWidth&quot;&#58;1188&#125;"><div data-sp-webpart="" data-sp-webpartdataversion="1.0" data-sp-webpartdata="&#123;&quot;id&quot;&#58;&quot;6410b3b6-d440-4663-8744-378976dc041e&quot;,&quot;instanceId&quot;&#58;&quot;62b390f6-2677-47e8-9084-5a958edca2b6&quot;,&quot;title&quot;&#58;&quot;Link&quot;,&quot;description&quot;&#58;&quot;Add a link with a rich preview such as an image or text brought in from the linked video or web content.&quot;,&quot;audiences&quot;&#58;[],&quot;serverProcessedContent&quot;&#58;&#123;&quot;htmlStrings&quot;&#58;&#123;&#125;,&quot;searchablePlainTexts&quot;&#58;&#123;&quot;title&quot;&#58;&quot;What is Data Factory in Microsoft Fabric&quot;,&quot;description&quot;&#58;&quot;Learn what is Data Factory workload in Microsoft Fabric, what components it includes, what is the origin of it, and what it does.&quot;&#125;,&quot;imageSources&quot;&#58;&#123;&quot;imageURL&quot;&#58;&quot;https&#58;//radacad.com/wp-content/uploads/2023/05/Data-Factory.png&quot;&#125;,&quot;links&quot;&#58;&#123;&quot;url&quot;&#58;&quot;https&#58;//radacad.com/what-is-data-factory-in-microsoft-fabric&quot;&#125;&#125;,&quot;dataVersion&quot;&#58;&quot;1.0&quot;,&quot;properties&quot;&#58;&#123;&quot;id&quot;&#58;&quot;85b82ac5-ca93-4abe-93cf-00ddc3cc911c&quot;,&quot;linkPreviewComponentMode&quot;&#58;2,&quot;cachedMetaDataUpdateTimeStamp&quot;&#58;1685434034276,&quot;listId&quot;&#58;&quot;&quot;,&quot;webId&quot;&#58;&quot;&quot;,&quot;uniqueId&quot;&#58;&quot;&quot;,&quot;siteId&quot;&#58;&quot;&quot;,&quot;previewContent&quot;&#58;&#123;&#125;&#125;,&quot;containsDynamicDataSource&quot;&#58;false&#125;"><div data-sp-componentid="6410b3b6-d440-4663-8744-378976dc041e"></div><div data-sp-htmlproperties=""><div data-sp-prop-name="title" data-sp-searchableplaintext="true">What is Data Factory in Microsoft Fabric</div><div data-sp-prop-name="description" data-sp-searchableplaintext="true">Learn what is Data Factory workload in Microsoft Fabric, what components it includes, what is the origin of it, and what it does.</div><img data-sp-prop-name="imageURL" src="https&#58;//radacad.com/wp-content/uploads/2023/05/Data-Factory.png" /><a data-sp-prop-name="url" href="https&#58;//radacad.com/what-is-data-factory-in-microsoft-fabric"></a></div></div></div><div data-sp-canvascontrol="" data-sp-canvasdataversion="1.0" data-sp-controldata="&#123;&quot;controlType&quot;&#58;0,&quot;pageSettingsSlice&quot;&#58;&#123;&quot;isDefaultDescription&quot;&#58;true,&quot;isDefaultThumbnail&quot;&#58;true,&quot;globalRichTextStylingVersion&quot;&#58;0,&quot;rtePageSettings&quot;&#58;&#123;&quot;contentVersion&quot;&#58;4&#125;,&quot;isEmailReady&quot;&#58;false&#125;&#125;"></div></div>

What is Data Factory in Microsoft Fabric

Microsoft Fabric is an end-to-end data analytics solution in the cloud, and one of its workloads is called Data Factory. In this article, you will learn what Da