Skip to content

BRK353 Breakout Fri 20 Nov 10:15 PST

AI-ready data integration with Fabric Data Factory

VIDYA Shanmugam, Wee hyong Tok

Summary

From the recording

Microsoft product leaders covered Fabric Data Factory updates. Mirroring security roles replication for Snowflake is in public preview. Mapping data flows are generally available in Dataflow Gen2, and DBT Core V2 (Fusion) is in public preview. Outlook activity is generally available. Business actions activity now connects to close to 80 systems. A View in Fabric readiness assessment helps upgrade Azure Data Factory pipelines. Caveat: part two of the Azure Data Factory upgrade journey is only promised in a few months, and the tumbling window trigger is only coming soon in private preview.

Session description

Microsoft's catalogue description

See how Fabric Data Factory helps teams connect, transform, govern, and operationalize enterprise data for AI. Learn patterns for pipelines, Dataflows, Copy Jobs, Mirroring, OneLake integrations, data quality, and analytics engineering with dbt to deliver trusted data for copilots, agents, and analytics. Additionally, learn how to modernize your data estate by migrating Azure Data Factory workloads to Microsoft Fabric.

Demo timestamps

Microsoft's player does not support links to a specific time, so open the recording and skip to the time shown.

  1. Skip to 0:07:46

    Snowflake mirroring with security roles into Fabric

    Snowflake sales and orders data is shown with existing security roles; user Aaron can reach only orders, products and sales tables. Data and the security model are mirrored into Fabric.

  2. Skip to 0:14:03

    Snowflake copy job into Real-Time Intelligence

    A copy job named Snowflake Ticket Ingestion is pointed at a Snowflake ticket-transactions table, so high-value bookings in a sports-fan ticketing example can raise real-time alerts.

  3. Skip to 0:21:24

    Redesigned Power Query editor in Power BI Desktop

    One-minute walkthrough of the redesigned editor: diagram view, schema view, column actions, query-folding indicators, query plans and dark mode.

  4. Skip to 0:23:12

    Mapping data flows in Dataflow Gen2

    Run MDF transforms builds Spark-scale low-code transformations in Dataflow Gen2, including upgrading existing Azure Data Factory or Synapse mapping data flows. A run succeeds in about five minutes; Monitoring Hub shows 150 million orders joined with 15 million customers.

  5. Skip to 0:26:15

    dbt Fusion on Fabric with GitHub Copilot

    A dbt project is created locally in VS Code against 100,000 order lines in a Fabric warehouse. GitHub Copilot generates sources, staging, a fact table, a sales mart and tests; a Fabric dbt job connects to the repo.

  6. Skip to 0:36:39

    Business actions activity in Fabric pipelines

    A built-in connector catalogue is used to call Stripe for a customer record by customer ID, first alone and then as part of a pipeline.

Watch on-demand (opens in a new tab)

← All sessions