netdata/health/guides/mysql/mysql_galera_cluster_size.md

Understand the alert

This alert monitors the Galera cluster size and checks if there is a discrepancy between the current cluster size and the maximum size in the last 2 minutes. A warning is raised if the current size is larger, and a critical alert is raised if the current size is smaller than the maximum size in the last minute.

Troubleshoot the alert

1. Check the network connectivity:

Galera Cluster relies on persistent network connections. Review your system logs for any connectivity issues or network errors. If you find such issues, work with your network administrator to resolve them.

1. Check the status of MySQL nodes:

You can use the following query to examine the status of all nodes in the Galera cluster:

   SHOW STATUS LIKE 'wsrep_cluster_%';

Look for the wsrep_cluster_size and wsrep_cluster_status values, and analyze if there are any inconsistencies or issues.

1. Review Galera logs:

Inspect the logs of the Galera cluster for any errors, warnings or issues. The log files are usually located in /var/log/mysql or /var/lib/mysql directories.

1. Check node synchronization:

   • Ensure that all nodes are synced by checking the wsrep_local_state_comment status variable. A value of 'Synced' indicates that the node is in sync with the cluster.

     SHOW STATUS LIKE 'wsrep_local_state_comment';
     

   • If any node is not synced, check its logs to find the cause of the issue and resolve it.

2. Restart nodes if necessary:

If you find that a node is not working properly, you can try to restart the MySQL service on the affected node:

   sudo systemctl restart mysql

Keep in mind that restarting a node can cause temporary downtime for applications connecting to that specific node.

1. If the issue persists, consider contacting the Galera Cluster support team for assistance or consult the Galera Cluster documentation for further guidance.

Useful resources

1. Galera Cluster Monitoring
2. Galera Cluster Documentation