netdata/health/guides/elasticsearch/elasticsearch_node_indices_search_time_fetch.md

Understand the alert

This alert is triggered when the Elasticsearch node's average search_time_fetch exceeds the warning or critical thresholds over a 10-minute window. The search_time_fetch measures the time spent fetching data from shards during search operations. If you receive this alert, it means your Elasticsearch search performance is degraded, and fetches are running slowly.

Troubleshoot the alert

1. Check the Elasticsearch cluster health

Run the following command to check the health of your Elasticsearch cluster:

curl -XGET 'http://localhost:9200/_cluster/health?pretty'

Look for the status field in the output, which indicates the overall health of the cluster:

• green: All primary and replica shards are active and allocated.
• yellow: All primary shards are active, but not all replica shards are active.
• red: Some primary shards are not active.

1. Identify slow search queries

Run the following command to gather information on slow search queries:

curl -XGET 'http://localhost:9200/_nodes/stats/indices/search?pretty'

Look for the query, fetch, and take fields in the output, which indicate the time taken by different parts of the search operation.

1. Check Elasticsearch node resources

Ensure the Elasticsearch node has sufficient resources (CPU, memory, disk space, and disk I/O). Use system monitoring tools like top, htop, vmstat, and iostat to analyze the resource usage on the Elasticsearch node.

1. Optimize search queries

If slow search queries are identified in Step 2, consider optimizing them for better performance. Some techniques for optimizing Elasticsearch search performance include using filters, limiting result set size, and disabling expensive operations like sorting and faceting when not needed.

1. Review Elasticsearch configuration

Check your Elasticsearch configuration to ensure it is optimized for search performance. Verify settings such as index refresh intervals, query caches, and field data caches. Consult the Elasticsearch documentation for best practices on configuration settings.

1. Consider horizontal scaling

If your Elasticsearch node is experiencing high search loads regularly, consider adding more nodes to distribute the load evenly across the cluster.

Useful resources

1. Elasticsearch Performance Tuning