DCImanager 6 saves information about the operation of services in log files. The data from log files can be used to diagnose the platform operation. Log files are stored in the /var/log directories of docker containers.

Log files list

Main log files on platform server

ContainerFile pathContents
dci_auth_back_1/var/log/licupdate.log logs of the license validation
logs of the main DCImanager 6 service
 /var/log/dci_location_check_update.loglogs of the update service for containers at the location. The service starts when the container is restarted and updates the containers at the location, if necessary
/var/log/dci_ansiblectl_location_<location id>.log
logs of creating and configuring a location
/var/log/dci_operationctl.loglog of operations performed at the location
dci_consumer_1/var/log/worker_daemon.loglogs of periodic equipment polls and obtaining statistical data

logs of the modules:

logs of address space management service
dci_ipmi_proxy_v2_client_1/var/log/ipmi_proxy_goproxy.loggoproxy service logs. The service transmits data to the VNC client and proxies Websocket requests to the VNC server
/var/log/ipmi_proxy_update_checker.loglogs of the update service for BMC proxy module at the location. The service starts each time the container is restarted and updates the module if necessary
vm_ldap_1/var/log/isp_ldap.loglogs of synchronization with the LDAP directory
migration of objects from DCImanager 5

The platform installation log is saved in the file /opt/ispsystem/dci/install.log.

The log files of the diagnostics operation are listed in the Diagnostics check article.

The authorization service logs are written to the standard output of the dci_auth_back4_1 container.

Main log files on location server

ContainerFile pathContents

/var/log/dci_switch_<id>_handler.loglogs of switch handlers
/var/log/dci_pdu_<id>_handler.loglogs of PDU handlers
/var/log/dci_ipmi_<id>_handler.loglogs of BMC handlers
/var/log/dci_intel_amt_handler.log logs of Intel AMT handlers

<id> — device ID in DCImanager 6. For example, for the switch with ID 3, the file is called dci_switch_3_handler.log

To view BMC logs, use the BMC id, not the server id. To get the BMC id, perform an API request:

GET https://domain.com/dci/v3/ipmi

domain.com — domain name or IP address of the server with the platform

Working with logs

Copy log file from a container

To copy a log file from the container to the current directory, run the command:

docker cp <container_name>:<path_to_log> ./

<container_name> — container name

<path_to_log> — log file path and name

Example command

docker cp dci_auth_back_1:/var/log/licupdate.log ./

View log files in a container

  1. Enter the required container: 

    docker exec -it <container_name> sh

    <container_name> — container name

    Example command

    docker exec -it dci_back sh
  2. View the log file using standard Linux utilities. For example, cat, tail, less, grep, etc.

    Example command: Find out when the switch was polled

    grep 'Switch status start' /var/log/dci_switch_1_handler.log
  3. Exit the container:


Viewing log files via stdout

Separate log files are not created for some services. Logs of these services are written to stdout. To view such logs, run the command: 

docker logs <container_name> --tail <lines> --since <period>

<container_name> — container name

<lines> — number of lines

<period> — logging period

Example command: Display the last 100 log lines of the authorization service created in the last 60 minutes

docker logs dci_consul_1 --tail 100 --since 60m

Collecting platform logs

To collect all the log files of the platform in one directory:

  1. Connect to the server with the platform via SSH.
  2. Create the bash script getlog.sh with the following content:

    BEGIN_DATE=$(date +'%Y-%m-%d %H:%M:%S' --date '-1 day')
    /bin/echo "Remove old files"
    /bin/rm -rf /home/logs && /bin/mkdir -p /home/logs/container && /bin/mkdir -p /home/logs/service && /bin/cd /home/logs
    /bin/echo "---===Step 1/3===---"
    /bin/echo "Get logs of the containers"
    for container in ${CONTAINERS[@]}
        /bin/echo "Get log of the container: ${container}"
        /bin/journalctl --no-pager CONTAINER_NAME=${container} --since "${SINCE}" > /home/logs/container/${container}.log
    /bin/echo "---===Step 2/3===---"
    /bin/echo "Get logs of the services"
    for service in ${SERVICES[@]}
        /bin/echo "Get log of the service: ${service}"
        /bin/journalctl --no-pager SYSLOG_IDENTIFIER=${service} --since "${SINCE}" > /home/logs/service/${service}.log
    /bin/echo "---===Step 3/3===---"
    ARCHIVE_NAME="logs_$(date +'%Y_%m_%d_%H_%M_%S').tar.gz"
    tar -cvzf /home/$ARCHIVE_NAME -C /home/logs .
    /bin/echo "Done"
  3. Run the script using one of the commands: 

    Before collecting logs, the script deletes all files from the /home/logs/ directory.

    Collect logs for the last day

    bash getlog.sh

    Collect logs since a certain date

    bash getlog.sh "YYYY-MM-DD"

    Collect logs since a certain date and time

    bash getlog.sh "YYYY-MM-DD HH:MM:SS"

    YYYYY-MM-DD — date from which logs should be collected

    HH:MM:SS — time from which logs should be collected

    The script:

    1. Saves container logs in the /home/logs/container/ directory.

    2. Saves service logs in the /home/logs/service/ directory.

    3. Creates an archive logs_<current_date>_.tar.gz in the /home/ directory.

Logging in journald

Logging settings

The platform uses the journald system service to collect logs. journald collects logs for all containers except dci_consul_1.

The journald service on the platform server and the location server is configured automatically. The service configuration is saved in the /usr/lib/systemd/journald.conf.d/dci.conf file. If the service has already been configured on the server, the configuration set by the platform will have a higher priority.

Configuring the journald service may fail if the syslog and rsyslog service settings on the server have been changed.

Configuration parameters set by the platform:

  • Storage=persistent — store files on disk;
  • Compress=yes — compress data in logs;
  • SystemMaxFileSize=500M — the maximum size of the log file is 500 Mb. When this value is reached, logs are rotated;
  • SystemMaxUse=20G — the maximum disk size for storing logs is 20 GB. When this value is reached, older files will be deleted.

Operations with logs

journald saves logs in the /var/log/journal/ directory. You can view the logs using the journalctl utility. Examples of how to use the utility:

Get id of containers for which logging is performed

journalctl --field CONTAINER_NAME

View container logs

journalctl -f CONTAINER_NAME=<container_id>

Get id of services for which logging is performed

journalctl --field SYSLOG_IDENTIFIER

View service logs

journalctl -f SYSLOG_IDENTIFIER=<service_id>

Get logs of the authorization service from the certain date

journalctl --no-pager CONTAINER_NAME=dci_auth_back4_1 --since "YYYY-MM-DD"

YYYYY-MM-DD — date from which logs should be collected