Media-Bug Azure Transcription — Architecture Guide

1. Overview

1.1 Executive Summary

• Transcription runs inside EFSwitch via mod_azure_transcribe (media-bug + Microsoft Speech SDK). Recognition is performed by Azure Speech in the cloud.

• A separate systemd process, tmb-relay-azure (tmb_app_azure.py), is intentionally thin: ESL events, start/stop TX, map call fields, POST finals to CCM.

• TX attaches only on CHANNEL_BRIDGE (not early media). Default layout is dual_mono: two mono Azure streams per bridged call (caller + agent).

• The relay does not create the CX conversation; Voice Connector already reserved the agent / conversation. The relay only appends PLAIN messages to /ccm/message/receive.

• Multi-tenant CCM URL: https://{FS domain_name}.{CCM_ROOT_DOMAIN}/ccm, with CCM_BASE_URL as fallback.

• When EFCX APISIX auth is on: AUTH_ENABLED=true + API_USERNAME / API_PASS → POST {tenantFqdn}/unified-admin/keycloakLogin → Authorization: Bearer on CCM (shared helper relay/ccm_auth.py). Default is off (no regression on non-APISIX labs).

2. Architecture Diagram

2.1 Component flow

2.2 Sequence Diagram

1. Voice Connector has already created the CX conversation.

2. EFSwitch emits CHANNEL_BRIDGE (A+B UUIDs) to the relay over ESL.

3. Relay applies tenant / call-type / allowlist gates.

4. Relay starts uuid_azure_transcribe on caller and agent legs (dual_mono).

5. Azure returns recognition events; EFSwitch forwards finals to the relay.

6. If AUTH_ENABLED=true and no cached token: POST {fqdn}/unified-admin/keycloakLogin → cache data.token.

7. Relay POSTs CCM /message/receive with Bearer when auth is on.

8. On HTTP 401: invalidate cache, refresh token once, retry the CCM request once.

9. On hangup / destroy: stop azure_transcribe once per call.

3. Component Responsibilities

3.1 What the relay does not do

• Does not originate SIP or bridge calls

• Does not call StartConversation / create CX conversations

• Does not store long-term transcript archives (beyond operational logs)

• Does not replace Voice Connector APISIX NodePort / gateway install work

4. Call Lifecycle and Gating

4.1 Trigger

1. ESL CHANNEL_BRIDGE with A and B channel UUIDs

2. Resolve EFSwitch domain_name → tenant gate (TMB_DOMAIN_ALLOWLIST)

3. Classify call type (inbound / manual outbound / campaign); apply TMB_ENABLE_* flags

4. Optional inbound DID filter (TMB_ALLOWLIST; * = all DIDs on allowed tenants)

5. If TMB_MODE=observe, log only; if active, start TX (unless kill file DISABLE.azure exists)

6. Enforce TMB_MAX_CONCURRENT

4.2 Layout

4.3 Finals → CCM

• Only final recognition results are posted as conversation messages

• Payload shape matches prior Jambonz apps: conversation id, header.channelData, sender CONNECTOR vs AGENT, body.type=PLAIN

4.4 Hangup and recovery

• On hangup / channel destroy, stop media-bugs once per call

• Mid-call relay start recovers already-bridged calls (recover: CHANNEL_BRIDGE attach)

• Stopping the systemd unit stops live media-bugs; starting again re-attaches bridged calls when recovery is enabled

5. Multi-Tenant CCM Routing

CCM base = https://{fs_domain_name}.{CCM_ROOT_DOMAIN}/ccm

Example: FS domain_name=thirdparty, CCM_ROOT_DOMAIN=intelicx.ai → https://thirdparty.intelicx.ai/ccm.

• Matches CX IVR / Voice Connector tenant FQDN pattern

• CCM_BASE_URL is used when dynamic resolution is unavailable

• Tenant header tenantid is set when required by the deployment

6. APISIX / Keycloak Auth (CCC-2331)

Auth is opt-in and shared across Azure, Deepgram, and the relay/ package via CxApiAuth.

6.1 Token flow

1. Derive tenant FQDN from CCM base (…/ccm stripped) or fallback base

2. POST {fqdn}/unified-admin/keycloakLogin with JSON {username, password}

3. Read data.token; cache per FQDN

4. Attach Authorization: Bearer <token> on CCM HTTP calls

5. On 401: invalidate cache, refresh token once, retry the request once

6. Log markers: AUTH token acquired, AUTH token cache cleared, CX request unauthorized (401)

This mirrors Voice Connector (ecx_generic_connector) env names and login path so RMT can reuse the same Keycloak Expertflow-realm API user (admin + default roles).

6.2 Failure modes

7. Deployment Topology

8. Operational Configuration (key knobs)

Full RMT steps: Set up Azure live call transcription on an EFSwitch server

9. Capacity and Sizing Notes

• Each bridged call in dual_mono = 2 Azure concurrent recognition streams

• Plan Azure Speech quota and EFSwitch CPU for 2 × concurrent bridged calls

• Start with a conservative TMB_MAX_CONCURRENT (for example 3–10), raise after soak / load test

• Relay CPU/RAM is usually small vs EFSwitch + Speech SDK in-process cost

• Prefer horizontal Media Server capacity and Azure quota over multi-worker tricks inside the relay

10. Security Considerations

• Secrets live in .env.azure (mode 600 recommended); never commit keys or API passwords

• With APISIX on, use a dedicated Keycloak API user — not personal admin logins in shared docs

• Transcripts and ANI are PII; restrict log retention and Agent Desk access via CX roles

• ESL is local-only by design; do not expose 8021 publicly

11. Observability

12. Related Work and Alternatives

13. Related Documents

• EFSwitch Media-Bug Live Transcription: Real-Time Call Text on Agent Desk Without Jambonz — Feature Guide

• Set up Azure live call transcription on an EFSwitch server — RMT setup

• Real-Time Transcription — Jambonz path

• Voice Connector Deployment Guide

• CX Voice with APISIX Auth: Required NodePort Exposure for EFCX Services

• PII-Masker Architecture — Architecture style reference

• GitLab: rtc/media-bug-for-transcription