Breadcrumbs

(5.1.0) Upgrade Guide for Data Platform - Draft

Pause all the pipeline from the Data Platform UI before proceeding to upgrade

  1. Clone the transflux repository

git clone -b 5.1_f-Candidate https://efcx:RecRpsuH34yqp56YRFUb@gitlab.expertflow.com/cim/transflux.git $HOME/CX-5.1/transflux

  1. Replace the existing configs with new ones as per following

    1. transflux/dbt_schema/agent_state_summary_gold.yml with CX-5.1/transflux/dbt_schema/agent_state_summary_gold.yml

    2. transflux/config/bulk_upload_config_template.yaml with CX-5.1/transflux/config/bulk_upload_config_template.yaml

    3. transflux/config/gold_15_minutes_queries_config_template.yaml with CX-5.1/transflux/config/gold_15_minutes_queries_config_template.yaml

    4. transflux/config/data_migration_config.yaml with CX-5.1/transflux/config/data_migration_config.yaml

  2. Edit the existing transflux/config/tenants.yaml

Before

After (For MYSQL)

After (For MSSQL)

 
   
    TARGET_TYPE: "mysql"
    MYSQL:
      TARGET_HOST: "192.168.2.18"
      TARGET_PORT: "3306"
      TARGET_USERNAME: "monty"
      TARGET_PASSWORD: "Expertflow#143"
      TARGET_SSL_ENABLED: false
      TARGET_DATABASE: "hold_db"
    MSSQL:
      TARGET_HOST: "192.168.1.77"
      TARGET_PORT: "1433"
      TARGET_USERNAME: "sa"
      TARGET_PASSWORD: "Expertflow464"
      TARGET_SSL_ENABLED: false
      TARGET_DATABASE: "hold_db"

   
    TARGET_TYPE: "mysql"
    TARGET_HOST: "192.168.2.18"
    TARGET_PORT: "3306"
    TARGET_USERNAME: "monty"
    TARGET_PASSWORD: "Expertflow#143"
    TARGET_SSL_ENABLED: false
    TARGET_DATABASE: "hold_db"

   
    TARGET_TYPE: "mssql"
    TARGET_HOST: "192.168.1.77"
    TARGET_PORT: "1433"
    TARGET_USERNAME: "sa"
    TARGET_PASSWORD: "Expertflow464"
    TARGET_SSL_ENABLED: false
    TARGET_DATABASE: "hold_db"

  1. Delete the existing config maps and re-create them from transflux directory

# Delete existing config maps
k delete cm ef-transflux-config-cm -n expertflow
k delete cm ef-transflux-dbt-schema-cm -n expertflow

# Re-create the config maps
kubectl -n expertflow create configmap ef-transflux-config-cm --from-file=config
kubectl -n expertflow create configmap ef-transflux-dbt-schema-cm --from-file=dbt_schema

  1. Edit the file helm-values/cx-transflux-custom-values.yaml in the transflux directory

    image:
  repository: cim/cx-data-platform/build
  tag: 5-1-f-candidatebranch-d69c4689131e8129201984f15f943f4f2d434c50

  2. Redeploy the solution

helm upgrade --install --namespace expertflow   --set global.efCxReleaseName="ef-cx"  cx-transflux --debug --values helm-values/cx-transflux-custom-values.yaml  expertflow/transflux --version 5.1.0-rc.1

  1. Once the pod is up, perform Alembic Schema Migrations using the guide https://expertflow-docs.atlassian.net/wiki/x/jADsRQ and use CX 5.1 version upgrade id available in attached document

  2. Once done, please go to Data Platform UI and trigger the Tenant-updates_5.1 named dag and ensure from logs it passes successfully with no errors.