Benzai Logging-operator to Syslog
Introdcution
This implementation routes the logs from a collected Flow to Syslog Server.
Requirements
- Syslog server at a reachable location/URL at port 514 ( UDP and/or TCP)
- has enough storage to ingest and maintain a larger quantity of logs
These examples below are given for ClusterFlow, and ClusterOutput, however, this can also be deployed using Flow and Output which are namespaced only and should be created in the same namespace .
Cluster Output
Create the ClusterOutPut Resource
CODE
kubectl apply -f - <<"EOF"
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterOutput
metadata:
name: ef-external-to-syslog
spec:
syslog:
buffer:
total_limit_size: 2GB
flush_thread_count: 8
timekey: 10m
timekey_use_utc: true
timekey_wait: 1m
format:
app_name_field: kubernetes.pod_name
hostname_field: custom-cluster-name
log_field: message
rfc6587_message_size: false
host: < SYSLOG-SERVER-FQDN-IP>
insecure: true
port: 514
transport: udp
EOF
ClusterFlow
Create a the associated ClusterFlow
CODE
apiVersion: logging.banzaicloud.io/v1beta1
kind: ClusterFlow
metadata:
name: global-ns-clusterflow
spec:
match:
- exclude:
namespaces:
- kube-system
- select:
namespaces:
- ef-external
globalOutputRefs:
- ef-external-to-syslogAfter a settlement time, usually 2-5 minutes, the cluster starts sending logs to the syslog server which can be viewed in ` /var/log/remote/{date hierarchy}/{hour}.log` location on the syslog server.
Using syslogNG to syslog
CODE
apiVersion: logging.banzaicloud.io/v1beta1
kind: SyslogNGOutput
metadata:
name: syslog-output
namespace: default
spec:
syslog:
host: 10.20.9.89
port: 601
template: "$(format-json
--subkeys json.
--exclude json.kubernetes.labels.*
json.kubernetes.labels=literal($(format-flat-json --subkeys json.kubernetes.labels.)))\n"
tls:
ca_file:
mountFrom:
secretKeyRef:
key: ca.crt
name: syslog-tls-cert
cert_file:
mountFrom:
secretKeyRef:
key: tls.crt
name: syslog-tls-cert
key_file:
mountFrom:
secretKeyRef:
key: tls.key
name: syslog-tls-cert
transport: tls
reference: https://kube-logging.dev/docs/configuration/plugins/syslog-ng-outputs/