APISIX Upgrade Guide from CX4.9.4 to CX4.10

APISIX

To migrate the apisix from old deployment to new minified mode, please follow along these steps.

Delete all the apisix routes in expertflow namespace

kubectl -n expertflow delete ar $(kubectl -n expertflow get ar -o jsonpath='{.items[*]..metadata.name}')

Uninstall the existing apisix helm chart in ef-external
```
helm -n ef-external uninstall apisix
```

Deploy the apisix from the CX4.10 release

Clone the apisix values.yaml file

helm show values expertflow/apisix --version 2.10.0  > helm-values/apisix-custom-values.yaml

Update the apisix-custom-values.yaml file for below given parameters

YAML

global:
  ingressRouter: "devops.ef.com"   
  ingressClassName: "nginx"
  ingressTlsCertName: "ef-ingress-tls-secret"

Deploy the apisix using updated custom-values.yaml file

helm upgrade --install --namespace ef-external --values helm-values/apisix-custom-values.yaml apisix  expertflow/apisix --version 2.10.0

Verify the deployment of the apisix
```
kubectl -n ef-external get deploy
```

Do not use the existing apisix-custom-values.yaml file, as it is not relevant to current release.

APISIX Static Config Routes

APISIX Route Details in Static Config

Service Group	Route ID	Status	Priority	Host(s)	URI Path(s)	Upstream Service	Plugins & Key Configuration
Agent Manager	ef-cx-agent-manager-login-unsecured	Enabled	10	devops.ef.com	/agent-manager/agent/login	svc-agent-manager	proxy-rewrite: .../login -> /agent/login
Agent Manager	ef-cx-agent-manager-socketio-unsecured	Enabled	10	devops.ef.com	/agent-manager/socket.io/*	svc-agent-manager	websocket enabled. proxy-rewrite: .../socket.io/* -> /socket.io/*
Agent Manager	ef-cx-agent-manager-send-sms-otp-unsecured	Enabled	10	devops.ef.com	/agent-manager/agent/send-sms-otp*	svc-agent-manager	proxy-rewrite: .../send-sms-otp* -> /agent/send-sms-otp*
Agent Manager	ef-cx-agent-manager-register-phone-unsecured	Enabled	10	devops.ef.com	/agent-manager/agent/register-phone*	svc-agent-manager	proxy-rewrite: .../register-phone* -> /agent/register-phone*
Agent Manager	ef-cx-agent-manager-validate-otp-unsecured	Enabled	10	devops.ef.com	/agent-manager/agent/validate-otp*	svc-agent-manager	proxy-rewrite: .../validate-otp* -> /agent/validate-otp*
Agent Manager	ef-cx-agent-manager-secured	Enabled	0	devops.ef.com	/agent-manager/*	svc-agent-manager	proxy-rewrite: Strips /agent-manager/ prefix.
Bot Framework	ef-cx-bot-framework-main	Enabled	0	devops.ef.com	/bot-framework/*	svc-bot-framework	proxy-rewrite: Strips /bot-framework/ prefix.
CCM	ef-cx-ccm-agents-unsecured	Enabled	10	devops.ef.com	/ccm/agents/*	svc-ccm	proxy-rewrite: Strips /ccm/ prefix.
CCM	ef-cx-ccm-channels-svc-id-unsecured	Enabled	10	devops.ef.com	/ccm/channels/service-identifier/*	svc-ccm	proxy-rewrite: Strips /ccm/ prefix.
CCM	ef-cx-ccm-widget-configs-unsecured	Enabled	10	devops.ef.com	/ccm/widget-configs/*	svc-ccm	proxy-rewrite: Strips /ccm/ prefix.
CCM	ef-cx-ccm-main-secured	Enabled	0	devops.ef.com	/ccm/*	svc-ccm	proxy-rewrite: Strips /ccm/ prefix.
CCM	ef-cx-ccm-internal-wcm-access	Enabled	20	apisix-gateway.ef-external.svc.cluster.local	/internal-wcm-to-ccm/*	svc-ccm	Internal Route. limit-count: 10 req/sec per IP. proxy-rewrite: Strips /internal-wcm-to-ccm/ prefix.
CIM Customer	ef-cx-cim-customer-main	Enabled	0	devops.ef.com	/cim-customer/*	svc-cim-customer	proxy-rewrite: Strips /cim-customer/ prefix.
Conversation	ef-cx-conversation-manager-main	Enabled	0	devops.ef.com	/conversation-manager/*	svc-conversation-manager	proxy-rewrite: Strips /conversation-manager/ prefix.
Conversation	ef-cx-conversation-monitor-main	Enabled	0	devops.ef.com	/conversation-monitor/*	svc-conversation-monitor	proxy-rewrite: Strips /conversation-monitor/ prefix.
Customer Widget	ef-cx-customer-widget-main	Enabled	0	devops.ef.com	/customer-widget/*	svc-customer-widget	proxy-rewrite: Strips /customer-widget/ prefix. client-control: Max body size 6MB.
Customer Widget	ef-cx-customer-widget-assets	Enabled	0	devops.ef.com	/widget-assets/*	svc-customer-widget	cors: Allows all origins.
File Engine	ef-cx-file-engine-public-svg-downloads	Disabled	10	devops.ef.com	/file-engine/api/downloadFileStream	svc-file-engine	cors: Allows all origins. response-rewrite: Adds X-XSS-Protection header.
File Engine	ef-cx-file-engine-main	Enabled	0	devops.ef.com	/file-engine/*	svc-file-engine	proxy-rewrite: Strips /file-engine/ prefix. client-control: Max body size 6MB. response-rewrite: Adds X-XSS-Protection.
Reports	ef-cx-historical-reports-main	Enabled	0	devops.ef.com	/historical-reports/*	svc-historical-reports	proxy-rewrite: Strips /historical-reports/ prefix.
License Manager	ef-cx-license-manager-main	Enabled	0	devops.ef.com	/license-manager/*	svc-license-manager	proxy-rewrite: Strips /license-manager/ prefix.
Reports	ef-cx-realtime-reports-main	Enabled	0	devops.ef.com	/realtime-reports/*	svc-realtime-reports	proxy-rewrite: Strips /realtime-reports/ prefix.
Routing Engine	ef-cx-routing-engine-main	Enabled	0	devops.ef.com	/routing-engine/*	svc-routing-engine	proxy-rewrite: Strips /routing-engine/ prefix.
Team	ef-cx-team-announcement-main	Enabled	0	devops.ef.com	/team-announcement/*	svc-team-announcement	proxy-rewrite: Strips /team-announcement/ prefix.
Admin UI	ef-cx-unified-admin-main	Enabled	0	devops.ef.com	/unified-admin, /unified-admin/*	svc-unified-admin	proxy-rewrite: Strips /unified-admin/ prefix. cors: Allows all origins. client-control: Max body size 6MB.
Admin UI	ef-cx-unified-admin-default	Enabled	-10	devops.ef.com	/*	svc-unified-admin	Default fallback route. Catches any unmatched paths.
Admin UI	ef-cx-unified-admin-default-404-handler	Disabled	-200	devops.ef.com	/*	dummy-service-for-404	mocking: Returns a custom 404 HTML page. (Lowest priority handler)
WCM	ef-cx-web-channel-manager-main	Enabled	0	devops.ef.com	/web-channel-manager/*	svc-web-channel-manager	websocket enabled. proxy-rewrite: Strips /web-channel-manager/ prefix.
Business Logic	ef-cx-business-calendar-main	Enabled	0	devops.ef.com	/business-calendar/*	svc-business-calendar	proxy-rewrite: Strips /business-calendar/ prefix.
Agent UI	cx-agent-desk-unified-agent-main	Enabled	0	devops.ef.com	/unified-agent, /unified-agent/*	svc-unified-agent	proxy-rewrite: Strips /unified-agent/ prefix. cors: Allows all origins.
Agent UI	cx-agent-desk-unified-agent-assets	Enabled	0	devops.ef.com	/assets/*	svc-unified-agent	cors: Allows all origins. client-control: Max body size 6MB.
Grafana	cx-agent-desk-grafana-main	Enabled	0	devops.ef.com	/grafana, /grafana/*	svc-grafana	proxy-rewrite: Strips /grafana/ prefix.
Campaigns	cx-campaigns-campaigns-backend-main	Enabled	0	devops.ef.com	/campaigns/*	svc-campaigns-backend	proxy-rewrite: Strips /campaigns/ prefix.
Campaigns	cx-campaigns-campaign-studio-redirect	Enabled	1	devops.ef.com	/campaign-studio	svc-campaign-studio	redirect: Appends a trailing slash (/) to the URI.
Campaigns	cx-campaigns-campaign-studio-main	Enabled	0	devops.ef.com	/campaign-studio/*	svc-campaign-studio	proxy-rewrite: .../campaign-studio/* -> /red/*. cors: Allows all origins.
Campaigns	cx-campaigns-scheduled-activities-main	Enabled	0	devops.ef.com	/scheduler/*	svc-scheduled-activities	proxy-rewrite: Strips /scheduler/ prefix.
Channels	cx-channels-connect360-main	Enabled	0	devops.ef.com	/360connector/*	svc-connect360	proxy-rewrite: Strips /360connector/ prefix.
Channels	cx-channels-facebook-connector-main	Enabled	0	devops.ef.com	/facebook-connector/*	svc-facebook-connector	proxy-rewrite: Strips /facebook-connector/ prefix.
Channels	cx-channels-instagram-connector-main	Enabled	0	devops.ef.com	/instagram-connector/*	svc-instagram-connector	proxy-rewrite: Strips /instagram-connector/ prefix.
Channels	cx-channels-smpp-connector-main	Enabled	0	devops.ef.com	/smppconnector/*	svc-smpp-connector	proxy-rewrite: Strips /smppconnector/ prefix.
Channels	cx-channels-telegram-connector-main	Enabled	0	devops.ef.com	/telegram-connector/*	svc-telegram-connector	proxy-rewrite: Strips /telegram-connector/ prefix.
Channels	cx-channels-twilio-connector-main	Enabled	0	devops.ef.com	/twilio-connector/*	svc-twilio-connector	proxy-rewrite: Strips /twilio-connector/ prefix.
Channels	cx-channels-twitter-connector-main	Enabled	0	devops.ef.com	/twitter-connector/*	svc-twitter-connector	proxy-rewrite: Strips /twitter-connector/ prefix.
Channels	cx-channels-viber-connector-main	Enabled	0	devops.ef.com	/viber-connector/*	svc-viber-connector	proxy-rewrite: Strips /viber-connector/ prefix.
Channels	cx-channels-email-connector-main	Enabled	0	devops.ef.com	/email-connector/*	svc-email-connector	proxy-rewrite: Strips /email-connector/ prefix.
Channels	cx-channels-whatsapp-connector-main	Enabled	0	devops.ef.com	/whatsapp-connector/*	svc-whatsapp-connector	proxy-rewrite: Strips /whatsapp-connector/ prefix.
Channels	cx-channels-ms-email-connector-main	Enabled	0	devops.ef.com	/ms-email-connector/*	svc-ms-email-connector	proxy-rewrite: Strips /ms-email-connector/ prefix.
Channels	cx-channels-youtube-connector-main	Enabled	0	devops.ef.com	/youtube-connector/*	svc-youtube-connector	proxy-rewrite: Strips /youtube-connector/ prefix.
Channels	cx-channels-linkedin-connector-main	Enabled	0	devops.ef.com	/linkedin-connector/*	svc-linkedin-connector	proxy-rewrite: Strips /linkedin-connector/ prefix.
Eleveo	cx-eleveo-eleveo-api-main	Enabled	0	devops.ef.com	/eleveo-connector-api/*	svc-eleveo-api	proxy-rewrite: Strips /eleveo-connector-api/ prefix.
Eleveo	cx-eleveo-eleveo-runner-main	Enabled	0	devops.ef.com	/eleveo-recording-runner/*	svc-eleveo-runner	proxy-rewrite: Strips /eleveo-recording-runner/ prefix.
Surveys	cx-surveys-survey-backend-main	Enabled	0	devops.ef.com	/survey-backend/*	svc-survey-backend	proxy-rewrite: Strips /survey-backend/ prefix.
Surveys	cx-surveys-survey-studio-redirect	Enabled	1	devops.ef.com	/survey-studio	svc-survey-studio	redirect: Appends a trailing slash (/) to the URI.
Surveys	cx-surveys-survey-studio-main	Enabled	0	devops.ef.com	/survey-studio/*	svc-survey-studio	proxy-rewrite: .../survey-studio/* -> /red/*. cors: Allows all origins.

ApiSix Route Configuration

Max File Size

Change the plugin configuration for client-control and update the required upload size in below given routes under client-control.max_body_size

Route
cx-agent-desk-unified-agent-assets
ef-cx-customer-widget-main
ef-cx-file-engine-main
ef-cx-unified-admin-frontend-spa-routes

API Authentication

To enable API authentication on specific routes, add the following openid-connect plugin snippet under the plugins section of the route definition:

# add the following openid-connect section under plugins: to enable the authentication
         openid-connect:
           enable: true
           discovery: "http://keycloak.ef-external.svc/auth/realms/expertflow/.well-known/openid-configuration"
           realm: "expertflow"
           client_id: "cim"
           client_secret: "ef61df80-061c-4c29-b9ac-387e6bf67052"
           bearer_only: true
           token_signing_alg_values_expected: "RS256"
           set_access_token_header: false
           set_userinfo_header: false
           audience: ["cim", "account", "realm-management"]
           required_scopes: ["email", "profile"]

Only specific components support API authentication. Among those, only the routes named secured or main are designed to support API-level authentication.

Below is the list of components that support API authentication.

Following components are secure with API gateway:

Agent Manager
Bot framework
Customer Channel Manager
Cim Customer
Conversation Manager
Conversation Monitor
License Manager
Routing Engine
Team Announcement
File Engine
Unified Admin

The following APIs, belonging to secure components, are unauthenticated.

POST /agent-manager/agent/login
GET /agent-manager/socket.io/
POST /agent-manager/agent/refresh-token
POST /agent-manager/agent/send-sms-otp
POST /agent-manager/agent/register-phone
POST /agent-manager/agent/validate-otp
GET /ccm/widget-configs/{{WidgetIdentifier}}
GET /ccm/agents/{{}}
GET /ccm/channels/service-identifier/{{serviceIdentifier}}
POST /agent-manager/agent/send-sms-otp
POST /agent-manager/agent/register-phone
POST /agent-manager/agent/validate-otp
POST /unified-admin/forms/getAllFormTitles
POST /unified-admin/forms/{{formID}}

/unified-admin/keycloakLogin (all routes)
/unified-admin/tiny-url (all routes)
/unified-admin/formValidation (all routes)
/unified-admin/agent-desk-settings (all routes)
/unified-admin/locale-setting(all routes)

Rate-Limiting

By Default 10 messages per second are allowed. In order to change the default message count per second, please update the below given snippet in apisix-custom-values.yaml file for apixis under route ef-cx-ccm-internal-wcm-access

 limit-count:
   enable: true
   count: 10          # this parameter should be adjusted.              
   time_window: 1     # Time duration in seconds.
   key_type: "var"
   key: "remote_addr"
   rejected_code: 429
   rejected_msg: "{\"error\":\"Internal rate limit exceeded .\"}"
   show_limit_quota_header: true