Data Migration Plan
Existing Customers:
Analyze schema of existing tables and determine what changes are done in the new table schemas.
Alembic script for existing tables (pickup last value from existing tables and add in state_records table)
Silver table schema changes using Alembic. (new table models are created by dbt).
Using alembic make the existing models compatible with dbt model.
Before moving do we have new data schemas/documentations? (Old vs New Table Schemas (Revamped ETLs) )
Write alembic scripts for data migrations.
Check impacts in existing reports.
Run overall regression cycle (test by loading complete data and deploying existing reports).
New Customers: