Logging and Tracing Integration with OpenSearch

This guide, intended for the operations team, walks you through integrating your audit logging system with OpenSearch. It is divided into clear steps:

Deploy OpenSearch
Configure index, mappings, index patterns, and dashboards
Configure Fluent Bit Output

Deploy OpenSearch

Set up OpenSearch cluster in your Kubernetes environment using the following guide.

Opensearch and Opensearch Dashboard Deployment Guide

Key Outputs

OpenSearch cluster URL: http://{{baseURL}}:9200
OpenSearch Dashboards URL: http://{{baseURL}}:5601
Admin credentials configured
Cluster health: GREEN

Configure Index, Mappings, index patterns and dashboards

Follow this guide to create the audit logging index and dashboards in OpenSearch Dashboard

Index Configuration, Opensearch and Opensearch Dashboard Setup Guide

Key Outputs:

Index created: audit_log_index
Mappings configured
Index status: ACTIVE
Index pattern created: audit-logs-*
Time field configured: timestamp
Visualization created
Fields discoverable in Dashboards
Search functionality enabled

Configure Fluent Bit Output

Connect Fluent Bit to send audit logs to OpenSearch.

Configure Fluent Bit output plugin
Set OpenSearch endpoint URL
Configure authentication (if needed)
Enable retry and buffering policies

update the OUTPUT section in helm-values/cx-fluent-bit-custom-values.yaml

[OUTPUT]
      Name  opensearch
      Match audit.admin
      Host  [opensearch host]
      Port  9200
      Index audit_log_index
      HTTP_User [opensearch user]
      HTTP_Passwd [opensearch password]
      Logstash_Format Off
      Replace_Dots On
      Suppress_Type_Name On
      Retry_Limit 5
      tls [opensearch tls verification]
      tls.verify Off

Key Outputs:

Fluent Bit connected to OpenSearch
Logs being received successfully
Index naming pattern working
No connection errors in Fluent Bit logs