Skip to content
Migration · Track 01

Migration toDatabricks.

Synapse, SSIS, ADF and T-SQL estates move to the Lakehouse as code your team owns: Delta tables under Unity Catalog, Lakeflow pipelines and jobs, and Databricks SQL beneath the same reports. Power BI stays exactly where it is.

01 / The Shift

From Estate to Lakehouse

Today

An estate running out of road

Hardcoded SSIS packages, ADF and Synapse pipelines, T-SQL nobody wants to touch, a dedicated pool with a retirement date, and every Power BI report sitting downstream of all of it.

Target

The Lakehouse, as code

Governed Delta tables under Unity Catalog, Lakeflow pipelines and jobs, and Databricks SQL beneath the same reports. All of it lives in Git and belongs to your team.

02 / On Databricks

Open tables, one catalog

01

Delta Lake

Tables stored as Delta on your own cloud storage: ACID transactions, time travel, schema enforcement and an open format any engine can read. Bronze, silver and gold are governed tables in the catalog, not folders in a lake, and the data never belongs to a service.

  • Open table format
  • ACID
  • Bronze · silver · gold
02

Unity Catalog

One catalog for tables, views, models and files, with permissions, lineage and audit in one place and mapped from the roles the estate already has. Delta Sharing lets you share tables with partners and other platforms without copying them.

  • Governance
  • Lineage
  • Delta Sharing
03

Serverless compute

SQL warehouses, jobs and notebooks on serverless compute that starts in seconds and is billed by use. Nothing is left running because it is slow to start, and nothing is sized by guesswork.

  • SQL warehouses
  • Jobs
  • Billed by use
04

Lakeflow Connect and Auto Loader

Managed, incremental ingestion from SQL Server and business applications into governed tables. Files in ADLS Gen2 are picked up once, as they land, with schema changes handled. No pipeline is written per source and nothing is polled.

  • Managed connectors
  • Incremental files
  • Schema evolution
05

Lakehouse Federation

SQL Server, Synapse or Snowflake can be queried through Unity Catalog without copying them first. This covers whatever stays where it is, during the move and afterwards, and removes the need for linked servers and interim extracts.

  • Query in place
  • Unity Catalog
  • Coexistence
03 / Pipelines to reports

Declared, scheduled, served

06

Lakeflow Declarative Pipelines

Pipelines that are declared, not scripted: streaming tables, materialized views and data-quality expectations that fail visibly. ADF pipelines, Synapse pipelines and SSIS packages arrive here as one pipeline where there used to be one per source, in Git.

  • Streaming tables
  • Materialized views
  • Expectations
07

Lakeflow Jobs

Schedules, dependencies, retries and alerts for every pipeline, notebook and query, in one place. ADF triggers and SQL Agent jobs become jobs your team can see and change.

  • Orchestration
  • Schedules
  • Alerts
08

Databricks SQL under Power BI

The SQL endpoint the reports read from: serverless, governed by the catalog and accelerated by Photon. The semantic models point at a SQL warehouse in place of the pool. Measures, reports and users stay as they are, and AI/BI Genie can query the same tables in plain language.

  • Serverless SQL
  • Photon
  • Reports unchanged
09

Asset Bundles and CI/CD

Every pipeline, job and catalog object is code in Git. It is deployed to development, test and production the same way, as a Databricks Asset Bundle, and reviewed like any other change. The repository is the platform.

  • DABs
  • Git
  • Environments
10

Built on Lakebridge

Databricks' own migration toolkit sits underneath: it inventories the estate, converts the T-SQL and reconciles source against target. Our method runs on top of it, working out what every package and procedure is for before an engineer decides where it goes.

  • Analyzer
  • Transpiler
  • Reconcile
04 / In five lines

What changes, and what stays

ADF · Synapse · SSIS

Pipelines become Lakeflow pipelines and jobs.

ADF pipelines, Synapse pipelines and SSIS packages are rebuilt as declarative pipelines with expectations and scheduled as jobs: one where there was one per source, all in Git.

Dedicated pool

The pool becomes Delta tables under Unity Catalog.

Schemas are mapped, history is backfilled and reconciled, and distribution is redesigned for Delta. The pool is frozen and retired once the reports read from the Lakehouse.

T-SQL

Procedures become Databricks SQL.

Converted with Lakebridge, with every dialect difference linked to its source and reviewed by a person. Anything that needs code becomes a notebook run as a job.

SQL Server · files

They arrive through Lakeflow Connect and Auto Loader.

Databases come through managed connectors and files are picked up as they land in ADLS Gen2, incrementally and under governance. What has to stay is queried in place through Lakehouse Federation.

Power BI

It reads a SQL warehouse; the reports stay.

The semantic models are repointed to Databricks SQL. Measures, reports and their users are untouched; only the source underneath changes.

05 / Private assessment

Tell us what the estate is made of

What we need
  1. 01An inventory of the estate: package, pipeline, procedure and table names, and where the reports read from.
  2. 02Who owns what, and which objects are already thought to be unused.
  3. 03Your Databricks position: workspace, catalog, cloud, and who administers it.
  4. 04The constraints that shape the cutover: maintenance windows, regulators, and the reports that cannot be interrupted.

No files or data at this stage