Platform log files
VMmanager saves information about the operation of services in log files. The data from log files can be used to diagnose the platform operation.
Log files list
Main log files
The main log files of VMmanager services are stored in the vm_box container in the /var/log/ directory. Usually the name of the log file is the same as the service name. For example, the log file of the vmwatch service is called vmwatch.log.
File name | Contents |
---|---|
Common services | |
vm_1_writer.log |
|
vm_1_reader.log | API requests executed with GET method |
monitor.log | monitoring service for VMs and cluster nodes |
run_script.log | running scripts |
vmctl.log |
|
vmwatch.log | libvirt events, VM synchronization status |
Operations with VMs | |
check_storage.log | VM storage check |
connect_storages.log | VM storage connection |
disk_resize.log | VM disk resizing operations |
gosockify.log | proxy service for access to VMs via VNC |
host.log | creating and deleting VMs |
host_action.log | starting, stopping, restarting VMs |
host_change_password.log | changing the root password for a VM |
host_firewall.log | firewall configuration |
host_image_create.log | creating VM images |
host_image_delete.log | deleting VM images |
host_migration.log | VM migration |
host_redefine.log | changing VM settings, setting VNC password |
host_snapshot.log | operations with VM snapshots |
host_source_copy.log | downloading and copying of VM images and OS templates; |
image_relocate.log | relocating VM images |
iso.log | downloading and mounting ISO images |
uploader.log | ISO images uploading |
websockify.log | access service for VMs via VNC |
Operations with LXD containers | |
lxdconsole.log | cloning LXD containers |
lxd_init.log | creating LXD containers |
Operations with cluster nodes | |
hawatch.log | hawatch microservice. Read more in Operating principle of high availability clusters |
lxd_node_info.log | collecting information about LXD cluster nodes |
network_configure.log node_network.log | network configuration on the cluster node |
node_create/{node_id}.log | collecting information about adding of the cluster nodes |
node_info.log | collecting information about the cluster node (information about the OS, disk, etc.) |
Platform installation
The platform installation log is saved in the file /opt/ispsystem/vm/install.log.
Various services
Function | Container | File path |
---|---|---|
Licensing | vm_license_1 | container stdout |
Notification service | vm_msgsender_1 | /var/log/msgsender_1_reader.1.log /var/log/msgsender_1_writer.1.log |
Mail notifications about creating, reinstalling the operating system, or changing the VM password | vm_msgsender_1 | /var/log/email.log |
Operation of Telegram bot. Read more in Sending notifications to Telegram | vm_msgsender_1 | /var/log/telegram.log |
Collecting information about cluster node uptime, QEMU and libvirt versions | nodewatch | /var/log/nodewatch.log |
Migration from VMmanager 5 | importer | /var/log/importer.log |
Address space migration | vm_ipmgr_1 | /var/log/ip_ipmgr5_proxy.log |
Synchronization with the LDAP directory | vm_ldap_1 | /var/log/isp_ldap.log |
Execution of query sequences | vm_batch_1 (vm-batch-1) | container stdout |
Log files on the cluster node
Function | File path |
---|---|
Collecting metrics for statistics | /var/log/gomon.log |
ha-agent service. Read more in Operating principle of high availability clusters | /var/log/ha-agent.log |
Working with logs
Copy log file from a container
To copy a log file from the container to the current directory, run the command:
docker cp <container_name>:<path_to_log> ./
<container_name> — container name
<path_to_log> — log file path and name
Example command
docker cp vm_auth_back_1:/var/log/licupdate.log ./
View log files in a container
Enter the required container:
docker exec -it <container_name> sh
BASHComments to the command<container_name> — container name
Example command
docker exec -it vm_box sh
BASHView the log file using standard Linux utilities. For example, cat, tail, less, etc.
Example command
less /var/log/host.log
BASHTo view the logs of a specific task:
- In the platform interface go to the Tasks → select the task → copy the request_id value.
Run the command:
grep <request_id> /var/log/*
CODEComments to the command<request_id> — request_id value
Exit the container:
exit
BASH
Viewing log files via stdout
Separate log files are not created for some services. Logs of these services are written to stdout. To view such logs, run the command:
docker logs <container_name> --tail <lines> --since <period>
<container_name> — container name
<lines> — number of lines
<period> — logging period
Example command: Display the last 100 log lines of vm_box contatiner created in the last 60 minutes
docker logs vm_box --tail 100 --since 60m
Collecting platform logs
To collect all the log files of the platform in one directory:
- Connect to the server with the platform via SSH.
Create the bash script logs_collect.sh with the following content:
#!/bin/bash rm /home/logs -fr DOCKER_CONTAINER_NAMES=`docker ps --format '{{.Names}}'` SERVICES=($DOCKER_CONTAINER_NAMES) cd /home mkdir -p logs cd logs for service in ${SERVICES[@]} do echo -e "----\033[0;31mCopying logs from $service\033[0m----\n" mkdir -p $service docker cp $service:/var/log/. $service/. docker logs $service > $service/${service}_stdout.log 2>&1 done cp -r /opt/ispsystem/*/install.log install.log ARCHIVE_NAME="logs_$(date +'%Y_%m_%d_%H_%M_%S').tar.gz" tar -cvzf /home/$ARCHIVE_NAME -C /home/logs .
BASHRun the script:
bash logs_collect.sh
BASH
The log files will be saved in the /home/logs/ directory, the log archive will be saved in the /home/ directory.
Logging to journald
By default, logging in the journald system service is disabled in the platform. To enable logging:
- Connect to a server with the platform via SSH.
Create the patch file /opt/ispsystem/vm/journald.yaml with the contents:
version: "3.5" services: auth_back4: volumes: - /var/run/systemd/journal/:/var/run/systemd/journal/ ldap: volumes: - /var/run/systemd/journal/:/var/run/systemd/journal/ vm_box: volumes: - /opt/ispsystem/vm/backup:/opt/ispsystem/vm/backup - /opt/ispsystem/license:/opt/ispsystem/license - /opt/ispsystem/vm/socket/:/opt/ispsystem/vm/vmbox/ - /var/run/systemd/journal/:/var/run/systemd/journal/
CODEApply the patch:
vm add-patch -p journald -f /opt/ispsystem/vm/journald.yaml
BASH
Useful tips
Related topics: