Browse Source

Collecting metrics docs section simplification (#17757)

Fotis Voutsas 9 months ago
parent
commit
b5bb2446d2

+ 0 - 35
docs/collecting-metrics/application-metrics.md

@@ -1,35 +0,0 @@
-# Collect Application Metrics with Netdata
-
-Netdata collects per-second metrics from a wide variety of applications running on your systems, including web servers,
-databases, message brokers, email servers, search platforms, and more. These metrics collectors are pre-installed with
-every Netdata Agent and typically require no configuration. Netdata also
-uses [`apps.plugin`](/src/collectors/apps.plugin/README.md) to gather and visualize resource utilization per application
-on Linux systems.
-
-The `apps.plugin` inspects the Linux process tree every second, similar to `top` or `ps fax`, and collects resource
-utilization data for every running process. However, Netdata goes a step further: instead of just displaying raw data,
-it transforms it into easy-to-understand charts. Rather than presenting a long list of processes, Netdata categorizes
-applications into meaningful groups, such as "web servers" or "databases." Each category has its own charts in the
-**Applications** section of your Netdata dashboard. Additionally, there are charts for individual users and user groups
-under the **Users** and **User Groups** sections.
-
-In addition to charts, `apps.plugin` offers the **Processes** [Function](/docs/top-monitoring-netdata-functions.md),
-which visualizes process entries in a table and allows for intuitive exploration of the processes. For more details on
-how the visualization of Functions works, check out the documentation on
-the [Top tab](/docs/dashboards-and-charts/top-tab.md).
-
-Popular application collectors:
-
-| Collector                                                                                  | Description                                                                                                                                         |
-|--------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|
-| [Prometheus](/src/go/collectors/go.d.plugin/modules/prometheus/README.md)                  | Gathers metrics from one or more Prometheus endpoints that use the OpenMetrics exposition format. Auto-detects more than 600 endpoints.             |
-| [Web server logs (Apache, NGINX)](/src/go/collectors/go.d.plugin/modules/weblog/README.md) | Tails access logs and provides very detailed web server performance statistics. This module is able to parse 200k+ rows in less than half a second. |
-| [MySQL](/src/go/collectors/go.d.plugin/modules/mysql/README.md)                            | Collects database global, replication, and per-user statistics.                                                                                     |
-| [Redis](/src/go/collectors/go.d.plugin/modules/redis/README.md)                            | Monitors database status by reading the server's response to the `INFO` command.                                                                    |
-| [Apache](/src/go/collectors/go.d.plugin/modules/apache/README.md)                          | Collects Apache web server performance metrics via the `server-status?auto` endpoint.                                                               |
-| [Nginx](/src/go/collectors/go.d.plugin/modules/nginx/README.md)                            | Monitors web server status information by gathering metrics via `ngx_http_stub_status_module`.                                                      |
-| [Postgres](/src/go/collectors/go.d.plugin/modules/postgres/README.md)                      | Collects database health and performance metrics.                                                                                                   |
-| [ElasticSearch](/src/go/collectors/go.d.plugin/modules/elasticsearch/README.md)            | Collects search engine performance and health statistics. Can optionally collect per-index metrics as well.                                         |
-
-Check available [data collection integrations](/src/collectors/COLLECTORS.md#available-data-collection-integrations) for
-a comprehensive view to all the integrations you can use to gather metrics with Netdata.

+ 0 - 43
docs/collecting-metrics/container-metrics.md

@@ -1,43 +0,0 @@
-# Collect Container Metrics with Netdata
-
-Thanks to close integration with Linux cgroups and the virtual files it maintains under `/sys/fs/cgroup`, Netdata can
-monitor health, status, and resource utilization of many different types of Linux containers.
-
-Netdata uses the [cgroups.plugin](/src/collectors/cgroups.plugin/README.md) to poll `/sys/fs/cgroup` and convert the raw data into human-readable metrics and meaningful visualizations. Through cgroups, Netdata is compatible with **all Linux containers**, such as Docker, LXC, LXD, Libvirt, systemd-nspawn, and more. Read more about [Docker-specific monitoring](#collect-docker-metrics) below.
-
-Netdata also has robust **Kubernetes monitoring** support thanks to a [Helmchart](/packaging/installer/methods/kubernetes.md) to automate deployment, collectors for k8s agent services, and robust [service discovery](https://github.com/netdata/agent-service-discovery/#service-discovery) to monitor the services running inside of pods in your k8s cluster. Read more about [Kubernetes monitoring](#collect-kubernetes-metrics) below.
-
-A handful of additional collectors gather metrics from container-related services, such as [dockerd](/src/go/collectors/go.d.plugin/modules/docker/README.md) or [Docker Engine](/src/go/collectors/go.d.plugin/modules/docker_engine/README.md). You can find all
-container collectors in our data collection integrations list under the [containers/VMs](/src/collectors/COLLECTORS.md#containers-and-vms) and [Kubernetes](/src/collectors/COLLECTORS.md#kubernetes) sections.
-
-## Collect Docker metrics
-
-Netdata has robust Docker monitoring thanks to the aforementioned [cgroups.plugin](/src/collectors/cgroups.plugin/README.md).
-
-Netdata also collects metrics from applications running inside of Docker containers. For example, if you create a MySQL database container using `docker run --name some-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:tag`, it exposes metrics on port 3306. You can configure the [MySQL collector](/src/go/collectors/go.d.plugin/modules/mysql/README.md) to look at `127.0.0.0:3306` for MySQL metrics:
-
-```yml
-jobs:
-  - name: local
-    dsn: root:my-secret-pw@tcp(127.0.0.1:3306)/
-```
-
-Netdata then collects metrics from the container itself, but also dozens [MySQL-specific metrics](/src/go/collectors/go.d.plugin/modules/mysql/README.md#charts) as well.
-
-### Collect metrics from applications running in Docker containers
-
-You could use this technique to monitor an entire infrastructure of Docker containers. The same [enable and configure](/src/collectors/REFERENCE.md) procedures apply whether an application runs on the host system or inside a container. You may need to configure the target endpoint if it's not the application's default.
-
-Netdata can even [run in a Docker container](/packaging/docker/README.md) itself, and then collect metrics about the host system, its own container with cgroups, and any applications you want to monitor.
-
-See our [application metrics doc](/docs/collecting-metrics/application-metrics.md) for details about Netdata's application metrics collection capabilities.
-
-## Collect Kubernetes metrics
-
-Netdata provides a host of tools and collectors for monitoring the many layers of a Kubernetes cluster. These methods work together to help you troubleshoot performance or availability issues across your k8s infrastructure.
-
-- A [Helm chart](https://github.com/netdata/helmchart), which bootstraps a Netdata Agent pod on every node in your cluster, plus an additional parent pod for storing metrics and managing alert notifications.
-- A [service discovery plugin](https://github.com/netdata/agent-service-discovery), which discovers and creates configuration files for [compatible applications](https://github.com/netdata/helmchart#service-discovery-and-supported-services) and any endpoints covered by our [generic Prometheus collector](/src/go/collectors/go.d.plugin/modules/prometheus/README.md). With these configuration files, Netdata collects metrics from any compatible applications as they run _inside_ a pod. Service discovery happens without manual intervention as pods are created, destroyed, or moved between nodes.
-- A [Kubelet collector](/src/go/collectors/go.d.plugin/modules/k8s_kubelet/README.md), which runs on each node in a k8s cluster to monitor the number of pods/containers, the volume of operations on each container, and more.
-- A [kube-proxy collector](/src/go/collectors/go.d.plugin/modules/k8s_kubeproxy/README.md), which also runs on each node and monitors latency and the volume of HTTP requests to the proxy.
-- A [cgroups collector](/src/collectors/cgroups.plugin/README.md), which collects CPU, memory, and bandwidth metrics for each container running on your k8s cluster.

+ 0 - 13
docs/collecting-metrics/system-metrics.md

@@ -1,13 +0,0 @@
-# Collect System Metrics with Netdata
-
-Netdata collects thousands of metrics directly from the operating systems of physical and virtual machines, IoT/edge devices, and [containers](/docs/collecting-metrics/container-metrics.md) with zero configuration.
-
-To gather system metrics, Netdata uses various plugins, each of which has one or more collectors for very specific metrics exposed by the host. The system metrics Netdata users interact with most for health monitoring and performance troubleshooting are collected and visualized by `proc.plugin`, `cgroups.plugin`, and `ebpf.plugin`.
-
-[**proc.plugin**](/src/collectors/proc.plugin/README.md) gathers metrics from the `/proc` and `/sys` folders in Linux systems, along with a few other endpoints, and is responsible for the bulk of the system metrics collected and visualized by Netdata. It collects CPU, memory, disks, load, networking, mount points, and more with zero configuration. It also allows Netdata to monitor its own resource utilization.
-
-[**cgroups.plugin**](/src/collectors/cgroups.plugin/README.md) collects rich metrics about containers and virtual machines using the virtual files under `/sys/fs/cgroup`. By reading cgroups, Netdata can instantly collect resource utilization metrics for systemd services, all containers (Docker, LXC, LXD, Libvirt, systemd-nspawn), and more. Learn more in the [collecting container metrics](/docs/collecting-metrics/container-metrics.md) doc.
-
-[**ebpf.plugin**](/src/collectors/ebpf.plugin/README.md): Netdata's extended Berkeley Packet Filter (eBPF) collector monitors Linux kernel-level metrics for file descriptors, virtual filesystem IO, and process management. You can use our eBPF collector to analyze how and when a process accesses files, when it makes system calls, whether it leaks memory or creating zombie processes, and more.
-
-While the above plugins and associated collectors are the most important for system metrics, there are many others. You can find all of our data collection integrations [here](/src/collectors/COLLECTORS.md#system-collectors).

+ 1 - 1
docs/developer-and-contributor-corner/lamp-stack.md

@@ -61,7 +61,7 @@ replacing `NODE` with the hostname or IP address of your system.
 
 ## Enable hardware and Linux system monitoring
 
-There's nothing you need to do to enable [system monitoring](/docs/collecting-metrics/system-metrics.md) and Linux monitoring with
+There's nothing you need to do to enable system monitoring and Linux monitoring with
 the Netdata Agent, which autodetects metrics from CPUs, memory, disks, networking devices, and Linux processes like
 systemd without any configuration. If you're using containers, Netdata automatically collects resource utilization
 metrics from each using the [cgroups data collector](/src/collectors/cgroups.plugin/README.md).

+ 3 - 9
docs/netdata-cloud/monitor-your-infrastructure.md

@@ -143,17 +143,11 @@ After you've learned the basics, you should [secure your infrastructure's nodes]
 one of our recommended methods. These security best practices ensure no untrusted parties gain access to the metrics
 collected on any of your nodes.
 
-### Collect metrics from systems and applications
+### Collect metrics from anywhere
 
-Netdata has [300+ pre-installed collectors](/src/collectors/COLLECTORS.md) that gather thousands of metrics with zero
-configuration. Collectors search each of your nodes in default locations and ports to find running applications and
-gather as many metrics as they can without you having to configure them individually.
+Netdata has [300+ pre-installed collectors](/src/collectors/COLLECTORS.md) that gather thousands of metrics with zero configuration. Collectors search each of your nodes in default locations and ports to find running applications and gather as many metrics as they can without you having to configure them individually.
 
-Most collectors work without configuration, should you want more info, you can read more on [how Netdata's metrics collectors work](/src/collectors/README.md) and the [Collectors configuration reference](/src/collectors/REFERENCE.md) documentation.
-
-In addition, find detailed information about which [system](/docs/collecting-metrics/system-metrics.md),
-[container](/docs/collecting-metrics/container-metrics.md), and [application](/docs/collecting-metrics/application-metrics.md) metrics you can
-collect from across your infrastructure with Netdata.
+Check our [comprehensive integrations section](/src/collectors/COLLECTORS.md), to find information about whatever you want to monitor with Netdata
 
 ## Netdata Cloud features
 

+ 11 - 27
src/collectors/COLLECTORS.md

@@ -1,45 +1,29 @@
 # Monitor anything with Netdata
 
 Netdata uses collectors to help you gather metrics from your favorite applications and services and view them in
-real-time, interactive charts. The following list includes collectors for both external services/applications and
-internal system metrics.
+real-time, interactive charts. The following list includes all the integrations where Netdata can gather metrics from.
 
-Learn more
-about [how collectors work](/src/collectors/README.md), and
-then learn how to [enable or
-configure](/src/collectors/REFERENCE.md#enable-and-disable-a-specific-collection-module) any of the below collectors using the same process.
+Learn more about [how collectors work](/src/collectors/README.md), and then learn how to [enable or configure](/src/collectors/REFERENCE.md#enable-and-disable-a-specific-collection-module) a specific collector.
 
-Some collectors have both Go and Python versions as we continue our effort to migrate all collectors to Go. In these
-cases, _Netdata always prioritizes the Go version_, and we highly recommend you use the Go versions for the best
-experience.
-
-If you want to use a Python version of a collector, you need to
-explicitly [disable the Go version](/src/collectors/REFERENCE.md#enable-and-disable-a-specific-collection-module),
-and enable the Python version. Netdata then skips the Go version and attempts to load the Python version and its
-accompanying configuration file.
+> **Note**
+>
+> Some collectors have both Go and Python versions as we continue our effort to migrate all collectors to Go. In these cases, _Netdata always prioritizes the Go version_, and we highly recommend you use the Go versions for the best experience.
 
 ## Add your application to Netdata
 
 If you don't see the app/service you'd like to monitor in this list:
 
-- If your application has a Prometheus endpoint, Netdata can monitor it! Look at our
-  [generic Prometheus collector](/src/go/collectors/go.d.plugin/modules/prometheus/README.md).
+- If your application has a Prometheus endpoint, Netdata can monitor it! Look at our [generic Prometheus collector](/src/go/collectors/go.d.plugin/modules/prometheus/README.md).
 
-- If your application is instrumented to expose [StatsD](https://blog.netdata.cloud/introduction-to-statsd/) metrics,
-  see our [generic StatsD collector](/src/collectors/statsd.plugin/README.md).
+- If your application is instrumented to expose [StatsD](https://blog.netdata.cloud/introduction-to-statsd/) metrics, see our [generic StatsD collector](/src/collectors/statsd.plugin/README.md).
 
-- If you have data in CSV, JSON, XML or other popular formats, you may be able to use our
-  [generic structured data (Pandas) collector](/src/collectors/python.d.plugin/pandas/README.md),
+- If you have data in CSV, JSON, XML or other popular formats, you may be able to use our [generic structured data (Pandas) collector](/src/collectors/python.d.plugin/pandas/README.md),
 
-- Check out our [GitHub issues](https://github.com/netdata/netdata/issues). Use the search bar to look for previous
-  discussions about that collector—we may be looking for assistance from users such as yourself!
+- Check out our [GitHub issues](https://github.com/netdata/netdata/issues). Use the search bar to look for previous discussions about that collector—we may be looking for assistance from users such as yourself!
 
-- If you don't see the collector there, you can make
-  a [feature request](https://github.com/netdata/netdata/issues/new/choose) on GitHub.
+- If you don't see the collector there, you can make a [feature request](https://github.com/netdata/netdata/issues/new/choose) on GitHub.
 
-- If you have basic software development skills, you can add your own plugin
-  in [Go](/src/go/collectors/go.d.plugin/README.md#how-to-develop-a-collector)
-  or [Python](/docs/developer-and-contributor-corner/python-collector.md)
+- If you have basic software development skills, you can add your own plugin in [Go](/src/go/collectors/go.d.plugin/README.md#how-to-develop-a-collector) or [Python](/docs/developer-and-contributor-corner/python-collector.md)
 
 ## Available Data Collection Integrations
 <!-- AUTOGENERATED PART BY integrations/gen_doc_collector_page.py SCRIPT, DO NOT EDIT MANUALLY -->