Case study · 02
Warehouse migration MCP tool
An MCP server that automates moving a data warehouse, starting with Snowflake to Databricks.
- Role
- Builder · Nuthan Technologies
- Status
- Shipped
- Read
- 1 min
- Stack
- PythonFastMCPFastAPIasyncioSQLiteReactSnowflakeDatabricks
2–3 days
of manual scripting replaced per migration
Problem
Every cross-warehouse migration meant two to three days of hand-written scripting, repeated for each project.
Approach
- 0110 MCP tools over stateless Streamable HTTP. The React UI is itself an MCP client, and any AI client can drive the same tools.
- 02No LLM in the migration path: every move is deterministic and auditable.
- 03An async job engine: submitting a migration returns a job ID immediately, a background task runs it, and a SQLite (WAL) job store feeds live logs over server-sent events.
- 04A streaming path moves 100k-row Arrow and Pandas chunks through a bounded queue for backpressure. An optional S3-Parquet staging path moves data warehouse to warehouse with no rows passing through Python.
- 05Per-warehouse work: Snowflake Arrow batches instead of LIMIT/OFFSET rescans, ROW_NUMBER paging for Synapse, Databricks COPY INTO with a fallback, and a type mapper across 5 sources and 4 destinations.
No LLM in the migration path: every move is deterministic and auditable.
Results
- Replaces 2–3 days of manual scripting per migration.
- Covers Snowflake, Databricks, BigQuery, Redshift and Synapse.
What I'd do next
- Add dry-run validation reports before any data moves.