CX End-to-End Performance Test Results
The following table provides an overview of load/soak test results:
Test No. | Test Type | Test Duration | No. of Test Cycles | Agents | Conversations (Customer) | Messages | Databases | Host Cluster | Remarks | Raw Data, Graphs, & Details. | |||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Think Time | Max Tasks per Agent | Concurrent | Total | Message Reply Think Time | Ramp up | Life Time (before it ends and a new conversation starts in its place) | Concurrent | Total (same as number of total customers) | Queued/Waiting | Total | Throughput (messages per second) | Response Time | MongoDB | Redis | Memory without Cache | Avg CPU Utilization % | |||||||||||||||||||||
Login (Ramp up) | Message Reply | State Change (Login → Ready) | Average | 99th Percentile | Distribution | Memory (Max) | Max Query Operations (ops/s) | Max Document Operations | Avg Open Connections | Max Disk Writes per Second | Max Disk Reads per Second | Total Size | Memory | Max Total Items in DB | Avg Command Calls per Second | Avg Command Duration (ms) | Max Network IO | Min | Max | ||||||||||||||||||
Scenario: Log in all the agents via PEP Connector, and set chat MRD state to Ready for all agents. Start sending conversations afterward. So there is no queuing/waiting in this scenario. | |||||||||||||||||||||||||||||||||||||
1 | Load Test | 2 hours | 3 | 1 sec | 5 sec | 3-5 sec | 5 | 100 | 100 | 5 sec | 1 sec | 20 minutes | 100 | No issues noticed | |||||||||||||||||||||||
2 | Soak Test | 24 hours | 2 | 1 sec | 5 sec | 3-5 sec | 5 | 100 | 100 | 10 sec | 1 sec | 20 minutes | 100 | No issues noticed | |||||||||||||||||||||||
3 | Load Test | 15-30 minutes | 4 | 1 sec | 5 sec | 7-10 sec | 5 | 500 | 500 | 10 sec | 1 sec | 20 minutes | 2500 | Halted cause of issues | |||||||||||||||||||||||
Info: Test # 3 was stopped cause of Agent Manager's performance deterioration. The following tests are executed after fixing the issues. | |||||||||||||||||||||||||||||||||||||
4 | Soak Test | 5 hours | 2 | 1 sec | 5 sec | 7-10 sec | 5 | 500 | 500 | 10 sec | 1 sec | 20 minutes | 2500 | Halted cause of issues | |||||||||||||||||||||||
Info: Test # 4 was stopped cause of application disconnection from Redis. Tuned some Redis parameters before executing the following tests. | |||||||||||||||||||||||||||||||||||||
5 | Soak Test | 3 hours | 2 | 1 sec | 5 sec | 7-10 sec | 5 | 1000 | 1000 | 15 sec | 1 sec | 20 minutes | 5000 | Halted cause of issues | |||||||||||||||||||||||
Info: Test # 5 stopped again because of application disconnection from Redis. Tuned Redis further before executing further tests. | |||||||||||||||||||||||||||||||||||||
6 | Soak Test | 30 minutes | 5 | 1 sec | 5 sec | 7-10 sec | 5 | 1500 | 1500 | 20 sec | 1 sec | 20 minutes | 7500 | Halted cause of issues | |||||||||||||||||||||||
Info: Test # 6 stopped cause all of the APIs suddenly stopped responding. It was caused by an NGINX proxy at the DNS server. Resolved the issues before the following tests. | |||||||||||||||||||||||||||||||||||||
7 | Soak Test | 3.2 hour | 1 | 1 sec | 5 sec | 7-10 sec | 1 | 5000 | 5000 | 20 sec | 1 sec | 20 minutes | 5000 | 42348 | 3.09 GB | insert: 285 update: 86.9 query: 34.8 delete: 0.044 | inserted: 285 updated: 23.5 returned: 90.2 | 54.4 | 469 | 202 | 1.11 GiB | 63456 | get: 277 set: 113 append: 53.0 del: 28.4 | get: 58.5 set: 18.3 append: 9.03 del: 4.44 | 2.88 MiB | ~ 9 GiB | ~ 15 GiB | 34.0% | Halted cause of issues | ||||||||
8 | Soak Test | 3.5 hours | 1 | 1 sec | 5 sec | 7-10 sec | 1 | 3000 | 3000 | 20 sec | 1 sec | 20 minutes | 5000 | 42551 | 3.03 GB | insert: 175 update: 61.8 query: 58.1 delete: 0.037 | 49.0 | 1.00 GiB | 62684 | get: 211 set: 85.2 append: 40.8 del: 25.5 | mget: 227 get: 43.7 set: 14.9 append: 9.03 del: 4.07 | 3.06 MiB | ~7 GiB | ~13 GiB | 36.1% | Halted cause of issues | |||||||||||