Breadcrumbs

Upgrade Guide for Data Platform

Pause all the pipeline from the Data Platform UI before proceeding to upgrade

  1. Clone the transflux repository

git clone -b 5.1.0 https://efcx:RecRpsuH34yqp56YRFUb@gitlab.expertflow.com/cim/transflux.git $HOME/CX-5.1.0/transflux

  1. Replace the existing configs with new ones as per following

    1. transflux/dbt_schema/agent_state_summary_gold.yml with CX-5.1.0/transflux/dbt_schema/agent_state_summary_gold.yml

    2. transflux/config/bulk_upload_config_template.yaml with CX-5.1.0/transflux/config/bulk_upload_config_template.yaml

    3. transflux/config/gold_15_minutes_queries_config_template.yaml with CX-5.1.0/transflux/config/gold_15_minutes_queries_config_template.yaml

    4. transflux/config/data_migration_config.yaml with CX-5.1.0/transflux/config/data_migration_config.yaml

    5. transflux/config/activities_data_pipeline_config_template.yaml with CX-5.1.0/transflux/config/activities_data_pipeline_config_template.yaml

  2. Edit the existing transflux/config/tenants.yaml

Before

After (For MYSQL)

After (For MSSQL)

 
   
    TARGET_TYPE: "mysql"
    MYSQL:
      TARGET_HOST: "192.168.2.18"
      TARGET_PORT: "3306"
      TARGET_USERNAME: "monty"
      TARGET_PASSWORD: "Expertflow#143"
      TARGET_SSL_ENABLED: false
      TARGET_DATABASE: "hold_db"
    MSSQL:
      TARGET_HOST: "192.168.1.77"
      TARGET_PORT: "1433"
      TARGET_USERNAME: "sa"
      TARGET_PASSWORD: "Expertflow464"
      TARGET_SSL_ENABLED: false
      TARGET_DATABASE: "hold_db"

   
    TARGET_TYPE: "mysql"
    TARGET_HOST: "192.168.2.18"
    TARGET_PORT: "3306"
    TARGET_USERNAME: "monty"
    TARGET_PASSWORD: "Expertflow#143"
    TARGET_SSL_ENABLED: false
    TARGET_DATABASE: "hold_db"

   
    TARGET_TYPE: "mssql"
    TARGET_HOST: "192.168.1.77"
    TARGET_PORT: "1433"
    TARGET_USERNAME: "sa"
    TARGET_PASSWORD: "Expertflow464"
    TARGET_SSL_ENABLED: false
    TARGET_DATABASE: "hold_db"

  1. Delete the existing config maps and re-create them from transflux directory

# Delete existing config maps
k delete cm ef-transflux-config-cm -n expertflow
k delete cm ef-transflux-dbt-schema-cm -n expertflow

# Re-create the config maps
kubectl -n expertflow create configmap ef-transflux-config-cm --from-file=config
kubectl -n expertflow create configmap ef-transflux-dbt-schema-cm --from-file=dbt_schema

  1. Edit the file helm-values/cx-transflux-custom-values.yaml in the transflux directory

    image:
  repository: cim/cx-data-platform
  tag: 5.1.0

  2. Redeploy the solution

helm upgrade --install --namespace expertflow   --set global.efCxReleaseName="ef-cx"  cx-transflux --debug --values helm-values/cx-transflux-custom-values.yaml  expertflow/transflux --version 5.1.0

  1. Once the pod is up, perform Alembic Schema Migrations using the guide Alembic Schema Migration and use CX 5.1 version upgrade id available in attached document

  2. Once done, please go to Data Platform UI and trigger the Tenant-updates_5.1 named DAG and ensure from logs it passes successfully with no errors.