Jason Miles
edudatasci.net
Jason Miles
@edudatasci.net
Practicing Data & AI specialist in California. Opinions and views are my own.
Want your #MSFabric #MaterializedLakeView deployments to stop being “it worked in dev” stories? Treat your #Lakehouse like code: one SQL-only configuration notebook, promoted through your pipeline, idempotent in every environment—with CDF set intentionally in the final cell.
Ship Your Lakehouse Like Code: Deploying MLVs with a SQL-Only Configuration Notebook
If you’re building with Materialized Lake Views (MLVs), you’ve probably felt the tension: the definitions live in code, but the Lakehouse itself is an environment-specific artifact. That gap is where deployments get messy—schemas drift, tables don’t exist yet, and MLV refresh behavior looks “random” when it’s really just reacting to configuration. This post lays out a pattern that closes that gap cleanly: a lakehouse configuration notebook that you promote through your deployment pipeline and run in every environment to create schemas, tables, and MLVs idempotently—using SQL cells only. The key is that MLVs are treated as “definition-driven assets” that can be iterated in dev and re-stamped into test/prod with the same notebook.
edudatasci.net
January 21, 2026 at 4:15 PM
#Bitemporal isn’t extra history—it’s operational clarity: what was true, and what did we know, at the time. Here’s why #MSFabric/hashtag/MSFabric" class="hover:underline text-blue-600 dark:text-sky-400 no-card-link">#MSFabric Lakehouse on Delta is a powerful bitemporal implementation, plus how materialized lake views can own interval closure and when #AzureSQL belongs in the mix. #MSFabric
Delta First: Building Efficient Bitemporal Tables in Microsoft Fabric
In financial services, the questions that matter most are rarely answered by “the latest record.” Regulators, auditors, model validators, and operations teams want something more specific: what was true for the business at the time, and what did we know at the time? That’s bitemporal thinking—and it’s exactly the kind of problem where Microsoft Fabric’s Lakehouse on Delta becomes more than storage. It becomes a practical design advantage. In this post, I’m going to walk through what bitemporal tables actually require, why intervals matter (ValidFrom/ValidTo), and how to implement bitemporal efficiently in Fabric by leaning into #DeltaLake in the Lakehouse.
edudatasci.net
January 16, 2026 at 3:07 PM
DirectLake on OneLake is the semantic layer many Fabric teams want—but CI/CD is still catching up. Here’s a practical two-step deployment pattern using Sempy Labs + Variable Libraries to rebind DirectLake models automatically after promotion. #MicrosoftFabric #PowerBI #DirectLake #CICD
DirectLake on OneLake CI/CD: A Practical Two-Step Deployment Pattern with Sempy Labs + Variable Libraries
DirectLake on OneLake is one of those “this is what we’ve been waiting for” features in Microsoft Fabric—until you try to deploy it cleanly across Dev → Test → Prod and realize you’ve re-entered the world of post-deployment manual fixes. In this how-to, I’m going to do three things: Contrast DirectLake on SQL endpoints (the “classic” flavor) with DirectLake on OneLake (the newer flavor), and explain why OneLake is worth the trouble. Walk through the normal deployment pipeline approach that works well for DirectLake on SQL. Show a two-step, semi-automated approach for DirectLake on OneLake using: …
edudatasci.net
January 15, 2026 at 3:03 PM
#DirectLake isn’t “one mode,”it’s two. If your #MSFabric #PowerBI semantic model is slow, failing security tests, or behaving inconsistently, there’s a good chance you’re running the wrong DirectLake flavor (or falling back without realizing it).
Two Flavors of DirectLake: Over SQL vs. Over OneLake (and How to Switch Without Surprises)
DirectLake has a way of sounding wonderfully simple: “Power BI, but it reads the lake directly.” Then you build two semantic models that both say DirectLake, and they behave… differently. One falls back to DirectQuery when you least expect it. Another refuses to touch your SQL views. Security works for you, but not for your report consumers. Suddenly, “DirectLake” feels less like a feature and more like a riddle. The good news: this is explainable. And once you understand the two flavors—DirectLake over SQL and DirectLake over OneLake…
edudatasci.net
January 14, 2026 at 7:19 PM
Your #fraud and #AML signals are hiding in plain sight,inside the relationships your tables don’t model well. This post shows how #Graph in #MicrosoftFabric turns #OneLake data into a network you can query and explore visually—so investigations and insights start from connections, not joins.
From Tables to Networks: A Deep Dive into Graph in Microsoft Fabric for Financial Services Insights
Most financial services data is already “connected.” It just isn’t modeled that way. Fraud rings don’t show up as a single row. Money laundering doesn’t announce itself in one transaction. Counterparty exposure isn’t obvious from one booking. The meaningful signal lives in relationships: who shares an address, which accounts route funds through the same nodes, where devices and identities overlap, and how risk propagates through a network. Graph in Microsoft Fabric is designed for exactly that: turning your OneLake data into a connected model you can explore visually, query with GQL, and enrich with built-in graph algorithms—without standing up a separate graph stack and duplicating data.
edudatasci.net
January 13, 2026 at 3:04 PM
“Real‑time” isn’t a toggle—it’s a tax. This update breaks down the complexity and cost curve from trickle‑scale to internet‑class, plus what Ignite 2025 signals about mirroring, CDC, and integrated streaming stacks. #RealTimeAnalytics #StreamingData #DataPlatforms #CostOptimization
Real‑Time Data Isn’t Free: The Complexity and Cost Tradeoffs (From Trickle to Internet‑Class)
The first time someone asks for “real‑time,” it sounds like a small tweak: refresh the dashboard faster, trigger an alert sooner, show a counter that feels alive. In a data platform, that single request quietly changes everything—how you ingest, how you process, how you serve, and how you operate. This post keeps it practical. It frames real‑time as a freshness target (not a vibe), walks through the two taxes real‑time introduces—architectural complexity and cost—and shows how patterns evolve as you scale from modest #StreamingData to internet‑class velocity. It also folds in recent Microsoft Ignite announcements that matter for real‑time platforms, including SQL Server 2025’s “change event streaming” and near real‑time analytics via OneLake/Fabric mirroring, plus the continued maturation of Microsoft Fabric’s Real‑Time Intelligence building blocks.
edudatasci.net
January 5, 2026 at 3:04 PM
#2025 was the year Microsoft stopped treating “data + AI + governance” as three separate initiatives. #MicrosoftFabric expanded into a true AI-era data estate (databases, OneLake interoperability, #FabricIQ, and agents), while #MicrosoftPurview pulled governance and security into the workflow.
2025 Year in Review: When Microsoft Fabric and Microsoft Purview Turned “Data + AI” Into a Governed Operating Model
By the end of 2025, the conversation around analytics stopped being about dashboards and started sounding a lot more like operations. The rise of autonomous and semi-autonomous agents put a sharper edge on an old truth: AI only becomes an enterprise capability when the underlying data is trusted, discoverable, and defensible. Microsoft Fabric and Microsoft Purview spent 2025 building toward that reality from opposite (but increasingly overlapping) sides of the house. Fabric pushed the platform forward—unifying workloads, expanding OneLake, and adding new intelligence and database capabilities designed for AI-era workloads. Purview tightened the governance and security loop—making data quality, cataloging, risk visibility, and policy enforcement feel less like a separate initiative and more like part of the daily flow.
edudatasci.net
December 26, 2025 at 3:04 PM
Stop rebuilding #SemanticModels just to rename 100 columns or repoint to a new Lakehouse.
This walkthrough shows a clean PBIP + TMDL folder workflow for #MicrosoftFabric semantic models—including how to retarget the entire model (or a single table) to a different Lakehouse. #PowerBI #DataModeling
Edit, Retarget, and Redeploy: A Practical TMDL Folder Workflow for Fabric Semantic Models
There’s a moment in every Fabric semantic model lifecycle where the “click it in the UI” approach stops scaling. It usually happens when you need to rename dozens (or hundreds) of fields to match a business glossary, or when Dev is stable and you’re ready to point the same model at a new Lakehouse for Test/Prod. That’s when the model stops being a diagram and starts being an artifact—something you want to treat like code. This guide reflows the whole workflow end-to-end, using the Fabric service Edit in Desktop experience to open the model, exporting it to a PBIP project stored as a TMDL folder, editing that folder externally (no scripting inside Power BI Desktop), and then getting those changes back into the service—
edudatasci.net
December 22, 2025 at 3:01 PM
Cloud migrations in Financial Services, Insurance, Wealth, and Professional Services often recreate the same old warehouse dynamics—just with better infrastructure. This post applies the SAMR lens to data platforms and shows how data products + design thinking help you move from “inventory” to…
From Warehouses to Products: SAMR for Your Cloud Data Platform
Financial Services, Insurance, Wealth Management, and Professional Services have a gift—and a curse—when it comes to data. The gift is that these industries know how to run critical systems with discipline. The curse is that we’re so good at controlling risk that we often rebuild the same constraints in every new platform we adopt. That’s why so many “modern cloud data platforms” in these sectors end up feeling like the old data warehouse with a new hosting model: better infrastructure, familiar bottlenecks.
edudatasci.net
December 16, 2025 at 3:05 PM
Trust debt hides in the “should‑do” list. See how policy‑aware digital workers plug into existing systems, finish the work that never makes the sprint, and hand back proof—updated for T+1, methane program shifts, BOI changes, and evolving data‑sharing rules. #Automation #WealthManagement #OilAndGas
From “Should‑Do” to Done: Digital Workers for Wealth, Energy, and Financial Services
Every enterprise carries a shadow backlog—the should‑do work that never beats the urgent. It’s the reconciliation that almost closes, the control that’s “fine for now,” the evidence that exists but isn’t filed where audit will accept it. None of these items is existential in isolation; together they become trust debt: silent risk, rework, slower decisions, and reputational drag. 2025 amplified the problem.
edudatasci.net
December 15, 2025 at 3:00 PM
From Chunks to Queries—Ignite 2025 Update: Fabric Data Agents, RAG, and the New IQ Layer

Monday, 9:02 a.m. The CFO pings: “What was Q3 gross margin by region—and did audit call out any risks?” Your RAG bot shines on PDFs and wiki pages, but it can’t compute a number you’d put on a KPI card. After…
From Chunks to Queries—Ignite 2025 Update: Fabric Data Agents, RAG, and the New IQ Layer
Monday, 9:02 a.m. The CFO pings: “What was Q3 gross margin by region—and did audit call out any risks?” Your RAG bot shines on PDFs and wiki pages, but it can’t compute a number you’d put on a KPI card. After Ignite 2025, the answer is cleaner than ever: let a Fabric Data Agent generate and run a governed query for the metric, and let your RAG retriever bring back the one‑sentence risk note.
edudatasci.net
December 12, 2025 at 3:43 PM
Microsoft Fabric makes medallion a first‑class citizen – but your data products don’t have to be medallion‑shaped. In a managed, domain‑driven world, inputs and outputs matter more than internal layers. This post shows how to treat medallion as a powerful option, not a mandate, with simple examples…
Fabric Is Medallion‑First, Not Medallion‑Only
If you work with Microsoft Fabric long enough, it’s easy to come away with the impression that “real” Fabric means “medallion everywhere.” The official docs walk through Bronze, Silver, and Gold patterns for lakehouses. The learning paths lean on medallion as the canonical example. Fabric clearly makes medallion a first‑class citizen. But that doesn’t mean your data platform – or your data products – must be medallion‑shaped.
edudatasci.net
December 11, 2025 at 3:00 PM
Your backlog tells you what to build. Your spec should tell you when it’s good enough to ship. Here’s how to make a three‑file spec drive code, tests, and SLOs—without slowing your team. #SpecDrivenDevelopment #Agile #DevOps #APIs
Spec‑Driven Development: Make the Specification the First Commit
If your acceptance criteria live in a comment thread, they’re not requirements—they’re opinions. Spec‑driven development (SDD) turns those opinions into executable truth so code, tests, docs, and operations move in lockstep. Building on our split between functional and nonfunctional requirements, this follow‑up introduces spec‑driven development: what it is, why it reduces drift, and how to run it inside agile without ceremony.
edudatasci.net
December 10, 2025 at 3:03 PM
Most “modern” workflows are just yesterday’s paper forms, rebuilt in browsers and automated with bots—sometimes still obeying the preferences of someone who retired fifty years ago. AI gives us a chance to stop automating those ghosts and start designing goal‑based processes that focus on outcomes.
From Substitution to Outcomes: How AI and SAMR Are Forcing a Rethink of Development Strategy
We like to say we’ve “transformed” how work gets done. But if you look closely at many enterprise systems, you still see the outline of a paper form hiding under a slick UI. We replaced paper with terminals, terminals with web apps, web apps with SaaS—and then pointed automation at the whole stack. In too many places, we’ve simply substituted one medium for another, without asking whether the underlying process still makes sense.
edudatasci.net
December 9, 2025 at 3:08 PM
Your organization doesn’t just have a #Copilot problem—it has an #AgentSprawl problem. This piece walks through how Entra’s #AgentRegistry and #MicrosoftPurview finally give you a control plane for AI #Agents: identity, inventory, and data policy that work together instead of apart.
How Entra’s Agent Registry and Purview Team Up to Conquer Agent Sprawl
AI agents are showing up everywhere in the enterprise: Copilot add‑ins, line‑of‑business copilots built in Studio, “helper” bots glued onto SaaS apps, home‑grown automations running in the background. Individually, each one looks harmless. Collectively, they turn into something more dangerous: agent sprawl. You get dozens (soon hundreds) of agents with overlapping responsibilities, inconsistent permissions, and no clear answer to a basic question: …
edudatasci.net
December 5, 2025 at 3:05 PM
Ontology gives your data a voice. Graph and agents give it a say in decisions. In part two of the series, we cover how Fabric IQ’s graph, data agents, and operations agents convert shared meaning into real‑time action. #MicrosoftFabric #FabricIQ #DataAgents #GraphAnalytics
Beyond the Ontology: How the Rest of Fabric IQ Turns Meaning into Action
Yesterday we went deep on Fabric IQ’s Ontology—the shared vocabulary that teaches Microsoft Fabric how your business actually talks. Today we’ll zoom out to everything else: the graph that lets insights travel across relationships, the agents that answer questions and watch your operations in real time, and the governance and integration that make it usable at scale.
edudatasci.net
December 4, 2025 at 3:01 PM
#Agents don’t just need more data—they need shared meaning. #MSFabric #FabricIQ's new #Ontology feature models your business concepts, binds them to live sources, and powers agents that can reason and act.
From Tables to Meaning: A Deep Dive into Microsoft Fabric IQ’s Ontology (Preview)
AI agents don’t fail for lack of data—they fail for lack of meaning. Microsoft Fabric IQ’s new ontology capability tackles that head‑on by modeling the business concepts, relationships, and rules that live across your estate, then binding them to live data so agents (and people) can ask better questions and take smarter action.
edudatasci.net
December 3, 2025 at 3:10 PM
SQL Server 2025 at Ignite: Why This Release Matters—and What to Do Next

In brief: SQL Server 2025 is generally available with built‑in AI, major developer conveniences, sturdier performance/availability behaviors, and licensing/edition changes that lower the cost of entry. Below I frame the…
SQL Server 2025 at Ignite: Why This Release Matters—and What to Do Next
In brief: SQL Server 2025 is generally available with built‑in AI, major developer conveniences, sturdier performance/availability behaviors, and licensing/edition changes that lower the cost of entry. Below I frame the release around three themes—AI + developer experience, performance + resilience, and product/edition shifts—and close with concrete first steps you can act on today.
edudatasci.net
December 1, 2025 at 3:02 PM
With #SAP #BusinessDataCloud Connect for #MSFabric, you can bring SAP #DataProducts into OneLake—and send Fabric insights back into SAP—bi‑directionally and zero‑copy. Add SAP Databricks, Azure Databricks, and Purview data products, and you have a practical, multi‑platform data‑product estate
SAP Business Data Cloud Connect for Microsoft Fabric: The New Backbone of Your Data‑Product Strategy
SAP and Microsoft have just taken away one of the biggest excuses for slow analytics and AI on SAP: “We can’t move that data safely or reliably enough.” At Microsoft Ignite 2025, they announced SAP Business Data Cloud (BDC) Connect for Microsoft Fabric—a new capability that lets you share SAP Business Data Cloud data products and Microsoft Fabric data sets bi‑directionally, with zero‑copy, and have those products show up natively in OneLake and back in BDC.
edudatasci.net
November 28, 2025 at 3:03 PM