MySQL

Metrics, Dashboards, Alerts and more for MySQL Integration in Sysdig Monitor.
MySQL

This integration is enabled by default.

Versions supported: > v5.7

This integration uses a standalone exporter that is available in UBI or scratch base image.

This integration has 47 metrics.

Timeseries generated: 1005 series per instance

List of Alerts

AlertDescriptionFormat
[MySQL] Mysql DownMySQL instance is downPrometheus
[MySQL] Mysql RestartedMySQL has just been restarted, less than one minute agoPrometheus
[MySQL] Mysql Too many Connections (>80%)More than 80% of MySQL connections are in usePrometheus
[MySQL] Mysql High Threads RunningMore than 60% of MySQL connections are in running statePrometheus
[MySQL] Mysql HighOpen FilesMore than 80% of MySQL files openPrometheus
[MySQL] Mysql Slow QueriesMySQL server mysql has some new slow queryPrometheus
[MySQL] Mysql Innodb Log WaitsMySQL innodb log writes stallingPrometheus
[MySQL] Mysql Slave Io Thread Not RunningMySQL Slave IO thread not runningPrometheus
[MySQL] Mysql Slave Sql Thread Not RunningMySQL Slave SQL thread not runningPrometheus
[MySQL] Mysql Slave Replication LagMySQL Slave replication lagPrometheus

List of Dashboards

MySQL

The dashboard provides information on the status, error rate and resource usage of a MySQL instance. MySQL

List of Metrics

Metric name
mysql_global_status_aborted_clients
mysql_global_status_aborted_connects
mysql_global_status_buffer_pool_pages
mysql_global_status_bytes_received
mysql_global_status_bytes_sent
mysql_global_status_commands_total
mysql_global_status_connection_errors_total
mysql_global_status_innodb_buffer_pool_read_requests
mysql_global_status_innodb_buffer_pool_reads
mysql_global_status_innodb_log_waits
mysql_global_status_innodb_mem_adaptive_hash
mysql_global_status_innodb_mem_dictionary
mysql_global_status_innodb_page_size
mysql_global_status_questions
mysql_global_status_select_full_join
mysql_global_status_select_full_range_join
mysql_global_status_select_range_check
mysql_global_status_select_scan
mysql_global_status_slow_queries
mysql_global_status_sort_merge_passes
mysql_global_status_sort_range
mysql_global_status_sort_rows
mysql_global_status_sort_scan
mysql_global_status_table_locks_immediate
mysql_global_status_table_locks_waited
mysql_global_status_table_open_cache_hits
mysql_global_status_table_open_cache_misses
mysql_global_status_threads_cached
mysql_global_status_threads_connected
mysql_global_status_threads_created
mysql_global_status_threads_running
mysql_global_status_uptime
mysql_global_variables_innodb_additional_mem_pool_size
mysql_global_variables_innodb_log_buffer_size
mysql_global_variables_innodb_open_files
mysql_global_variables_key_buffer_size
mysql_global_variables_max_connections
mysql_global_variables_open_files_limit
mysql_global_variables_query_cache_size
mysql_global_variables_thread_cache_size
mysql_global_variables_tokudb_cache_size
mysql_slave_status_master_server_id
mysql_slave_status_seconds_behind_master
mysql_slave_status_slave_io_running
mysql_slave_status_slave_sql_running
mysql_slave_status_sql_delay
mysql_up

Preparing the Integration

Create Credentials for MySQL Exporter

  1. Create the user and password for the exporter in the database:
CREATE USER 'exporter' IDENTIFIED BY 'YOUR-PASSWORD' WITH MAX_USER_CONNECTIONS 3;
GRANT PROCESS, REPLICATION CLIENT, SELECT ON *.* TO 'exporter';

Replace the user name and the password in the SQL sentence for your custom ones.

  1. Create a mysql-exporter.cnf file with the credentials of the exporter:
[client]
user = exporter
password = "YOUR-PASSWORD"
host=YOUR-DB-IP
  1. In your cluster, create the secret with the mysql-exporter.cnf file. This file will be mounted in the exporter to authenticate with the database:
kubectl create secret -n Your-Application-Namespace generic mysql-exporter \
  --from-file=.my.cnf=./mysql-exporter.cnf

Using SSL Authentication

If your database requires SSL authentication, you need to create secrets with the certificates. To do so, create the secret with SSL certificates for the exporter:

kubectl create secret -n Your-Application-Namespace generic mysql-exporter \
  --from-file=.my.cnf=./mysql-exporter.cnf
  --from-file=ca.pem=./certs/ca.pem \
  --from-file=client-key.pem=./certs/client-key.pem \
  --from-file=client-cert.pem=./certs/client-cert.pem

In the mysql-exporter.cnf file, include the following lines to route to the certificates in the exporter:

[client]
user = exporter
password = "YOUR-PASSWORD"
host=YOUR-DB-IP
ssl-ca=/lib/cert/ca.pem
ssl-key=/lib/cert/client-key.pem
ssl-cert=/lib/cert/client-cert.pem

Installing

An automated wizard is present in the Monitoring Integrations in Sysdig Monitor. However, you can also use this Helm chart for expert users: https://github.com/sysdiglabs/integrations-charts/tree/main/charts/mysql-exporter

Monitoring and Troubleshooting MySQL

This document describes important metrics and queries that you can use to monitor and troubleshoot MySQL.

Tracking metrics status

You can track MySQL metrics status with following alerts: Exporter proccess is not serving metrics

# [MySQL] Exporter Process Down
absent(mysql_up{kube_cluster_name=~$cluster,kube_namespace_name=~$namespace,kube_workload_name=~$workload}) > 0

Agent Configuration

This is the default agent job for this integration:

- job_name: mysql-default
  tls_config:
    insecure_skip_verify: true
  kubernetes_sd_configs:
  - role: pod
  relabel_configs:
  - action: keep
    source_labels: [__meta_kubernetes_pod_host_ip]
    regex: __HOSTIPS__
  - action: keep
    source_labels:
    - __meta_kubernetes_pod_annotation_promcat_sysdig_com_integration_type
    regex: "mysql"
  - action: replace
    source_labels: [__address__, __meta_kubernetes_pod_annotation_promcat_sysdig_com_port]
    regex: ([^:]+)(?::\d+)?;(\d+)
    replacement: $1:$2
    target_label: __address__
  - action: replace
    source_labels: [__meta_kubernetes_pod_annotation_promcat_sysdig_com_target_ns]
    target_label: kube_namespace_name
  - action: replace
    source_labels: [__meta_kubernetes_pod_annotation_promcat_sysdig_com_target_workload_type]
    target_label: kube_workload_type
  - action: replace
    source_labels: [__meta_kubernetes_pod_annotation_promcat_sysdig_com_target_workload_name]
    target_label: kube_workload_name
  - action: replace
    replacement: true
    target_label: sysdig_omit_source
  - action: replace
    source_labels: [__meta_kubernetes_pod_uid]
    target_label: sysdig_k8s_pod_uid
  - action: replace
    source_labels: [__meta_kubernetes_pod_container_name]
    target_label: sysdig_k8s_pod_container_name
  metric_relabel_configs:
  - source_labels: [__name__]
    regex: (mysql_global_status_aborted_clients|mysql_global_status_aborted_connects|mysql_global_status_buffer_pool_pages|mysql_global_status_bytes_received|mysql_global_status_bytes_sent|mysql_global_status_commands_total|mysql_global_status_connection_errors_total|mysql_global_status_innodb_buffer_pool_read_requests|mysql_global_status_innodb_buffer_pool_reads|mysql_global_status_innodb_log_waits|mysql_global_status_innodb_mem_adaptive_hash|mysql_global_status_innodb_mem_dictionary|mysql_global_status_innodb_page_size|mysql_global_status_questions|mysql_global_status_select_full_join|mysql_global_status_select_full_range_join|mysql_global_status_select_range_check|mysql_global_status_select_scan|mysql_global_status_slow_queries|mysql_global_status_sort_merge_passes|mysql_global_status_sort_range|mysql_global_status_sort_rows|mysql_global_status_sort_scan|mysql_global_status_table_locks_immediate|mysql_global_status_table_locks_waited|mysql_global_status_table_open_cache_hits|mysql_global_status_table_open_cache_misses|mysql_global_status_threads_cached|mysql_global_status_threads_connected|mysql_global_status_threads_created|mysql_global_status_threads_running|mysql_global_status_uptime|mysql_global_variables_innodb_additional_mem_pool_size|mysql_global_variables_innodb_log_buffer_size|mysql_global_variables_innodb_open_files|mysql_global_variables_key_buffer_size|mysql_global_variables_max_connections|mysql_global_variables_open_files_limit|mysql_global_variables_query_cache_size|mysql_global_variables_thread_cache_size|mysql_global_variables_tokudb_cache_size|mysql_slave_status_master_server_id|mysql_slave_status_seconds_behind_master|mysql_slave_status_slave_io_running|mysql_slave_status_slave_sql_running|mysql_slave_status_sql_delay|mysql_up)
    action: keep