Statistics collection logic

To collect, process and visualize statistics, the platform uses the DCImanager 6 equipment management service and third-party software:

• Clickhouse — to collect and store statistical data;
• Grafana — to visualize, monitor and analyze data.

Services for handling statistics are run in docker containers:

• dci_eservice_1 —  DCImanager 6 equipment management service;
• dci_carbon_clickhouse_1 — statistical data aggregator;
• clickhouse_server — statistics storage service — Clickhouse database;
• graphite_clickhouse — service for converting statistical data into graphite format;
• graphite_carbonapi — API service for transferring metrics to Grafana and the platform interface;
• dci_grafana_1 — Grafana;
• dci_grafana_user_1 — Grafana with data on user servers.

Collecting and storing data

Scheme for collecting statistics:

1. DCImanager 6 equipment management service receives data from the device ports every five minutes.
   
2. The Carbone clickhouse service receives the data collected in graphite format, converts it to clickhouse format, and sends it to the Clickhouse server service.
3. The Graphite clickhouse service requests data from the Clickhouse server, converts it into graphite format and sends it to the Graphite carbonapi service.
4. The Graphite carbonapi service sends data to Grafana and the platform interface.

Statistical data are available at the unique metric address. For example, the metric address for traffic is instance.1.server.{server_id}.port.{port_id}.{type}, where: 

• server_id — server identifier;
• port_id — identifier of the port to which the server is connected;
• type — traffic type. This parameter can assume the values: rx — incoming, tx — outgoing.

The statistics data are stored in Graphite for three year.

Data visualization

Grafana is used to build and display the graphs in DCImanager 6 interface. To view the statistical data, go to the device page. For example, to view the server traffic statistics, go to Servers → select the server → Server parameters → Statistics. Read more in Traffic statistics.

Grafana's web interface is available at URL https://domain.com/dci/grafana/.

To enter Grafana's web interface, in the right-hand menu click → Grafana.