Skip to content

Architecture overview

Purpose

The repository delivers a Microsoft Fabric retail demo with two active modes:

  1. Fabric-native historical setup through retail-setup and setup notebooks.
  2. Optional live Real-Time Intelligence (RTI) through stream-events.ipynb writing directly to Eventhouse and queried with KQL (Kusto Query Language).
flowchart LR
    subgraph Local[Operator and repository]
        Setup[setup.ps1 / setup.sh / setup.py]
        CLI[retail-setup]
        Deploy[Terraform + fabric-cicd + KQL apply]
    end

    subgraph Fabric[Microsoft Fabric workspace]
        SetupNB[setup-01..04]
        Stream[stream-events]
        Lake[(Lakehouse<br/>Silver ag / Gold au)]
        Event[(Eventhouse<br/>KQL database)]
        KQL[KQL functions, views, querysets]
        Pipes[Data Pipelines]
        Model[Direct Lake semantic model]
        Report[Power BI report]
        Ontology[Ontology]
        Agents[Data Agents]
    end

    Setup --> CLI --> Deploy
    Deploy --> SetupNB
    Deploy --> Stream
    Deploy --> Event
    Deploy --> Pipes
    Deploy --> Model
    Deploy --> Report
    SetupNB --> Lake
    Stream --> Event --> KQL
    Event --> Pipes --> Lake
    Lake --> Model --> Report
    Lake --> Ontology
    Event -->|TimeSeries bindings| Ontology
    Model --> Agents
    Ontology --> Agents

Primary historical path

setup-01 through setup-04 seed dictionaries, generate dimensions and facts, and build Gold directly in the Lakehouse. This path does not require ADLS parquet shortcuts or the retained historical-load notebook.

Optional live path

stream-events emits eighteen typed business event types to Eventhouse through the Spark Kusto connector, which is the Kusto software component for Apache Spark. KQL supplies the recent-event query path. Optional Eventhouse shortcuts and streaming transforms project events into Lakehouse Silver and Gold.

Direct Lake is the Power BI connection mode that reads Lakehouse tables directly from OneLake without importing a second copy.

Contract owners

Current support boundaries

  • Deployment profiles select exact, dependency-checked groups. The destructive reset group is excluded from every automatic profile.
  • Dashboard and rule assets are not yet guaranteed first-class deployable items.
  • The semantic model is Direct Lake and has 42 active tables, including six ML outputs.
  • fact_online_order_status is a streaming-only Silver output outside the base table contract.
  • Required full-demo setup, ML, Reporting, ontology, agents, task flow, and readiness have live evidence. Alternate authentication, separate core/standard profile proof, and recent optional streaming evidence remain in the owning module backlogs.