Skip to main content
Skip table of contents

Docker Logs Retention


HOSTThe system where all components are running in a containerized environment.
dockerdA master process for creating an isolated execution environment with all the boundaries and limits enforced using different namespaces. Every single instance instantiated by dockerd process is called a container.
ContainerAn isolated execution environment running applications inside.
rsyslogdA daemon process running on most of the Linux/Unix distributions, responsible for collecting logs from different applications and distributing them to respective logs files. For Example /var/log/message
journaldA master process for maintaining and managing system logs and activities.

Log Rotation Using JSON file format

To setup rotating logs, add the following lines in /etc/docker/daemon.json file (create the file if it doesn’t exist already) and restart the docker daemon using systemctl restart docker on all instances. In the following configuration, the maximum file size is kept to 20 MB for a maximum of 3 files.

"log-driver": "json-file",
"log-opts": {
   "max-size": "20m",  # Max size of the log files.  
   "max-file": "3"     # The maximum number of log files that can be present.

Please do not remove already existing ( if any ) lines in the /etc/docker/daemon.json file, as it may corrupt the docker deamon configuration. Consult Expertflow personnel for help on this.

Verifying Logs

Logs for each container are available in files as well as within the docker daemon. To see the logs for any container, execute docker ps, and get the id of the container. Use that id to see the logs using docker logs container_id. At any given time, the active machine would be the one with the highest keep alive priority, so the logs would be stored there.

Logs aggregation on the host

Container logs are flushed on the container restart. This section covers guidelines for aggregating logs of Expertflow components on the host.  All logs are forwarded to the host system's rsyslogd process. Rsyslogd aggregates logs based on container names and write them to respective files. These log files will be rotated by another application called logrotated to maintain a minimum of the last 30 files.

Requires Host Reboot

A system reboot is required after these configurations. 

  1. Create a root log directory on the host. 

    mkdir /var/log/dockerlfs
  2. make a copy of the currently running rsyslogd.conf using 

    cp /etc/rsyslog.conf{,-backup}
  3. in /etc/rsyslog.conf 
    1. add/update contents to look like below ( remove # at the start of these lines as they are already present in the file)  

      $ModLoad imtcp   
      $InputTCPServerRun 514
    2. Add/enable these parameters  in /etc/rsyslog.conf


      $ActionQueueFileName fwdRule1
      $ActionQueueSaveOnShutdown on
      $ActionQueueType LinkedList
      $ActionResumeRetryCount -1
      $imjournalRatelimitInterval 0
      $imjournalRatelimitBurst 0
  4. Create a new file /etc/rsyslog.d/docker_daemon.conf and add below the given contents. 


    $template DockerLogs, "/var/log/dockerlfs/daemon.log"
    if $programname startswith 'dockerd' then -?DockerLogs
    & stop
  5. Create a new file /etc/rsyslog.d/docker_container.conf add below-given contents.


    $template DockerContainerLogs,"/var/log/dockerlfs/%hostname%_%syslogtag:R,ERE,1,ZERO:.*container_name/([^\[]+)--end%.log"
    if $syslogtag contains 'container_name'  then -?DockerContainerLogs
    & stop
  6. Reconfigure journald - edit /etc/systemd/journald.conf and add/enable.

  7.  For log rotation, reconfigure. create a new file /etc/logrotate.d/dockerlogs and add the following.


    /var/log/dockerlfs/*.log {
      size 20M
      dateformat -%Y%m%d%H%s
      rotate 30
  8. Restart system services.

    systemctl restart rsyslog
    systemctl restart systemd-journald
  9. Check the status of configurations by 

    systemctl status rsyslog
    ● rsyslog.service - System Logging Service
       Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)
       Active: active (running) since Tue 2020-06-16 15:41:41 PKT; 6h ago
         Docs: man:rsyslogd(8)
     Main PID: 4161 (rsyslogd)
        Tasks: 8
       Memory: 12.8M
       CGroup: /system.slice/rsyslog.service
               └─4161 /usr/sbin/rsyslogd -n
    Jun 16 15:41:41 port1 systemd[1]: Starting System Logging Service...
    Jun 16 15:41:41 port1 rsyslogd[4161]:  [origin software="rsyslogd" swVersion="8.24.0-52.el7" x-pid="4161" x-info=""] start
    Jun 16 15:41:41 port1 systemd[1]: Started System Logging Service.
  10. Run ncat 


    nc -zvw 3000ms  localhost 514
    Ncat: Version 7.50 ( )
    Ncat: Connected to ::1:514.
    Ncat: 0 bytes sent, 0 bytes received in 0.05 seconds.

    or use lsof the command for the same purpose 


    # lsof -i:514
  11. Edit Docker daemon /etc/docker/daemon.json and add the following code snippet to enable forwarding container logs to the host. 


    	"log-driver": "syslog",
    	"log-opts": {
    	"syslog-address": "tcp://",
    	"tag": "container_name/{{.Name}}",
    	"labels": "production",
    	"syslog-facility": "daemon"

Please do not remove already existing ( if any ) lines in the /etc/docker/daemon.json file, as it may corrupt the docker deamon configuration. Consult Expertflow personnel for help on this.

  1. Restart the Docker services 

    // shutdown the solution 
    efutils all down
    // bring the solution up again 
    systemctl restart docker
    efutils all up 

Validate Configurations

File names may differ based upon the context and project name used, but the last part of file names will identify the component name.

  1. Restart the application stack to start sending their corresponding logs to the Docker daemon, which should now forward all container logs to the host. 

    cd /var/log/dockerlfs/
    ls -ltr
    total 11768
    -rw-------. 1 root root    6091 Jun 16 20:15 localhost_minio_1.log
    -rw-------. 1 root root    5281 Jun 16 20:15 localhost_bot-connector_1.log
    -rw-------. 1 root root   55335 Jun 16 20:15 localhost_ecm-services_1.log
    -rw-------. 1 root root   46237 Jun 16 20:16 localhost_mysql_1.log
    -rw-------. 1 root root   58450 Jun 16 20:16 localhost_database-connector_1.log
    -rw-------. 1 root root   85133 Jun 16 20:16 localhost_mre-microservice_1.log
    -rw-------. 1 root root   31446 Jun 16 20:16 localhost_activemq_1.log
    -rw-------. 1 root root   31465 Jun 16 20:17 localhost_ccm_1.log
    -rw-------. 1 root root   40939 Jun 16 20:19 daemon.log
    -rw-------. 1 root root  218661 Jun 16 20:19 localhost_eabc_1.log
    -rw-------. 1 root root   82426 Jun 16 20:19 localhost_ecm_1.log
    -rw-------. 1 root root  141140 Jun 16 22:05 localhost_mre-frontend_1.log
    -rw-------. 1 root root   86612 Jun 16 22:05 localhost_customer-gadget_1.log
    -rw-------. 1 root root   78237 Jun 16 22:05 localhost_ecm-frontend_1.log
    -rw-------. 1 root root   89137 Jun 16 22:05 localhost_agent-gadget_1.log
    -rw-------. 1 root root   37694 Jun 16 22:05 localhost_file-engine_1.log
    -rw-------. 1 root root   76818 Jun 16 22:05 localhost_mongo_1.log
    -rw-------. 1 root root 3318065 Jun 16 22:06 localhost_umm_1.log
    -rw-------. 1 root root 1473795 Jun 16 22:06 localhost_comm-server_1.log
    -rw-------. 1 root root 2558706 Jun 16 22:06 localhost_mre_1.log
    -rw-------. 1 root root 1538594 Jun 16 22:06 localhost_chat_1.log
    -rw-------. 1 root root 1538594 Jun 16 22:06 localhost_app-gateway_1.log
  2. Tail one or more files to see logs output 

    tail -f /var/log/dockerlfs/{localhost_file-engine_1.log,localhost_comm-server_1.log,localhost_umm_1.log}
    tail -f /var/log/dockerlfs/* for all logs to be tailed on your screen/terminal
JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.