netdata/docs/netdata-cloud

Netdata Cloud

Netdata Cloud is a service that complements Netdata installations. It is a key part in achieving optimal cost structure for large scale observability.

Technically, Netdata Cloud is a thin control plane that allows the Netdata ecosystem to be a virtually unlimited scalable and flexible observability pipeline. With Netdata Cloud, this observability pipeline can span multiple teams, cloud providers, data centers and services, while remaining a uniform and highly integrated infrastructure, providing real-time and high-fidelity insights.

flowchart TB
    NC("<b>☁️ Netdata Cloud</b>
          access from anywhere,
          horizontal scalability,
          role based access,
          custom dashboards,
          central notifications")
    Users[["<b>✨ Unified Dashboards</b>
            across the infrastructure,
            multi-cloud, hybrid-cloud"]]
    Notifications["<b>🔔 Alert Notifications</b>
                    Slack, e-mail, Mobile App,
                    PagerDuty, and more"]
    Users <--> NC
    NC -->|deduplicated| Notifications
    subgraph On-Prem Infrastructure
        direction TB
        Agents("<b>🌎 Netdata Agents</b>
                Standalone,
                Children, Parents
                (possibly overlapping)")
        TimeSeries[("<b>Time-Series</b>
                    metric samples
                    database")]
        PrivateAgents("<b>🔒 Private
                        Netdata Agents</b>")
        Agents <--> TimeSeries
        Agents ---|stream| PrivateAgents
    end
    NC <-->|secure connection| Agents

Netdata Cloud provides the following features, on top of what the Netdata Agents already provide:

1. Horizontal scalability: Netdata Cloud allows scaling the observability infrastructure horizontally, by adding more independent Netdata Parents and Children. It can aggregate such, otherwise independent, observability islands into one uniform and integrated infrastructure.

Netdata Cloud is a fundamental component for achieving an optimal cost structure and flexibility, in structuring observability the way that is best suited for each case.

1. Role Based Access Control (RBAC): Netdata Cloud has all the mechanisms for user-management and access control. It allows assigning all users a role, segmenting the infrastructure into rooms, and associating Rooms with roles and users.

2. Access from anywhere: Netdata Agents are installed on-prem, and this is where all your data is always stored. Netdata Cloud allows querying all the Netdata Agents (Standalone, Children and Parents) in real-time when dashboards are accessed via Netdata Cloud.

This enables a much simpler access control, eliminating the complexities of setting up VPNs to access observability, and the bandwidth costs for centralizing all metrics to one place.

1. Central dispatch of alert notifications: Netdata Cloud allows controlling the dispatch of alert notifications centrally. By default, all Netdata Agents (Standalone, Children and Parents) send their own notifications. This becomes increasingly complex as the infrastructure grows. So, Netdata Cloud steps in to simplify this process and provide central control of all notifications.

Netdata Cloud also enables the use of the Netdata Mobile App offering mobile push notifications for all users in commercial plans.

1. Custom Dashboards: Netdata Cloud enables the creation, storage and sharing custom dashboards.

Custom dashboards are created directly from the UI, without the need for learning a query language. Netdata Cloud provides all the APIs to the Netdata dashboards to store, browse and retrieve custom dashboards created by all users.

1. Advanced Customization: Netdata Cloud provides all the APIs for the dashboard to have different default settings per space, per Room and per user, allowing administrators and users to customize the Netdata dashboards and charts the way they see fit.

Data Exposed to Netdata Cloud

Netdata Cloud serves as a thin layer on top of Netdata Agents. It does not receive the samples collected, or the logs Netdata Agents maintain.

Netdata's design deliberately avoids centralizing raw metrics and logs. This prevents the common constraints of traditional observability solutions: reduced metric resolution, forced data filtering, and higher costs.

Instead, Netdata Cloud only stores metadata such as:

• Node information and labels
• Metric names, labels, and retention periods
• Active collectors and modules
• Alert configurations and state changes

This information is a small fraction of the total information maintained by Netdata Agents, allowing Netdata Cloud to remain high-resolution, high-fidelity and real-time, while being able to:

• dispatch alerts centrally for all alert transitions.
• know which Netdata Agents to query when users view the dashboards.

Metric samples and logs are transferred via Netdata Cloud to your Web Browser, only when you view them via Netdata Cloud. And even then, Netdata Cloud does not store this information. It only aggregates the responses of multiple Netdata Agents to a single response for your web browser to visualize.

High-Availability

You can subscribe to Netdata Cloud updates at the Netdata Cloud Status page.

Netdata Cloud is a highly available, auto-scalable solution; however, being a monitoring solution, we need to ensure dashboards are accessible during crisis.

Netdata Agents provide the same dashboard Netdata Cloud provides, with the following limitations:

1. Netdata Agents (Children and Parents) dashboards are limited to their databases, while on Netdata Cloud the dashboard presents the entire infrastructure, from all Netdata Agents connected to it.

2. When you are not logged in or the Agent is not connected to Netdata Cloud, certain features of the Netdata Agent dashboard will not be available.

When you are logged in and the Agent is connected to Netdata Cloud, the dashboard has the same functionality as Netdata Cloud.

To ensure dashboard high availability, Netdata Agent dashboards are available by directly accessing them, even when the connectivity between Children and Parents or Netdata Cloud faces issues. This allows the use of the individual Netdata Agents' dashboards during crisis, at different levels of aggregation.

Fidelity and Insights

Netdata Cloud queries Netdata Agents, so it provides exactly the same fidelity and insights Netdata Agents provide. Dashboards have the same resolution, the same number of metrics, exactly the same data.

Performance

The Netdata Agent and Netdata Cloud have similar query performance, but there are additional network latencies involved when the dashboards are viewed via Netdata Cloud.

Accessing Netdata Agents on the same LAN has marginal network latency, and their response time is only affected by the queries. However, accessing the same Netdata Agents via Netdata Cloud has a bigger network round-trip time that looks like this:

1. Your web browser makes a request to Netdata Cloud.
2. Netdata Cloud sends the request to your Netdata Agents. If multiple Netdata Agents are involved, they are queried in parallel.
3. Netdata Cloud receives their responses and aggregates them into a single response.
4. Netdata Cloud replies to your web browser.

If you are sitting on the same LAN as the Netdata Agents, the latency will be two times the round-trip network latency between this LAN and Netdata Cloud.

With multiple Netdata Agents, queries are faster than single-point monitoring solutions. Netdata Cloud distributes each query across multiple Agents, where each Agent processes only a portion of the query. This distributed approach uses your infrastructure's combined processing power, delivering superior performance compared to centralized solutions.

Does Netdata Cloud require Observability Centralization Points?

No. Any or all Netdata Agents can be connected to Netdata Cloud.

We recommend to create observability centralization points, as required for operational efficiency (ephemeral nodes, teams or services isolation, central control of alerts, production systems performance), security policies (internet isolation), or cost optimization (use existing capacities before allocating new ones).

We suggest reviewing the Best Practices for Observability Centralization Points.

When I have Netdata Parents, do I need to connect Netdata Children to Netdata Cloud too?

No, it is unnecessary, but it provides high availability.

When Netdata Parents are connected to Netdata Cloud, all their Netdata Children are available via these Parents.

When multiple Parent nodes store data from the same Child nodes (in clusters or multi-level hierarchies), Netdata Cloud queries each unique node once through a single available Parent.

Netdata Cloud prefers:

• The most distant (from the Child) Parent is available when doing metrics visualization queries (since usually these Parents have been added for this purpose).

• The closest (to the Child) Parent available for Top Monitoring (since top-monitoring provides live data, like the processes running, the list of sockets open, etc.). The streaming protocol of Netdata Parents and Children is able to forward such requests to the right child, via the Parents, to respond with live and accurate data.

Netdata Children may be connected to Netdata Cloud for high-availability, in case the Netdata Parents are unreachable.