enable-streaming.mdx 6.9 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151
  1. ---
  2. title: "Enable streaming between nodes"
  3. description: "With metrics streaming enabled, you can not only replicate metrics data into a second database, but also view dashboards and trigger alarm notifications for multiple nodes in parallel."
  4. type: how-to
  5. custom_edit_url: https://github.com/netdata/netdata/edit/master/docs/metrics-storage-management/enable-streaming.mdx
  6. ---
  7. # Enable streaming between nodes
  8. The simplest streaming configuration is **replication**, in which a child node streams its metrics in real time to a
  9. parent node, and both nodes retain metrics in their own databases.
  10. To configure replication, you need two nodes, each running Netdata. First you'll first enable streaming on your parent
  11. node, then enable streaming on your child node. When you're finished, you'll be able to see the child node's metrics in
  12. the parent node's dashboard, quickly switch between the two dashboards, and be able to serve [alarm
  13. notifications](/docs/monitor/enable-notifications.md) from either or both nodes.
  14. ## Enable streaming on the parent node
  15. First, log onto the node that will act as the parent.
  16. Run `uuidgen` to create a new API key, which is a randomly-generated machine GUID the Netdata Agent uses to identify
  17. itself while initiating a streaming connection. Copy that into a separate text file for later use.
  18. > Find out how to [install `uuidgen`](https://command-not-found.com/uuidgen) on your node if you don't already have it.
  19. Next, open `stream.conf` using [`edit-config`](/docs/configure/nodes.md#use-edit-config-to-edit-configuration-files)
  20. from within the [Netdata config directory](/docs/configure/nodes.md#the-netdata-config-directory).
  21. ```bash
  22. cd /etc/netdata
  23. sudo ./edit-config stream.conf
  24. ```
  25. Scroll down to the section beginning with `[API_KEY]`. Paste the API key you generated earlier between the brackets, so
  26. that it looks like the following:
  27. ```conf
  28. [11111111-2222-3333-4444-555555555555]
  29. ```
  30. Set `enabled` to `yes`, and `default memory mode` to `dbengine`. Leave all the other settings as their defaults. A
  31. simplified version of the configuration, minus the commented lines, looks like the following:
  32. ```conf
  33. [11111111-2222-3333-4444-555555555555]
  34. enabled = yes
  35. default memory mode = dbengine
  36. ```
  37. Save the file and close it, then restart Netdata with `sudo systemctl restart netdata`, or the [appropriate
  38. method](/docs/configure/start-stop-restart.md) for your system.
  39. ## Enable streaming on the child node
  40. Connect to your child node with SSH.
  41. Open `stream.conf` again. Scroll down to the `[stream]` section and set `enabled` to `yes`. Paste the IP address of your
  42. parent node at the end of the `destination` line, and paste the API key generated on the parent node onto the `api key`
  43. line.
  44. Leave all the other settings as their defaults. A simplified version of the configuration, minus the commented lines,
  45. looks like the following:
  46. ```conf
  47. [stream]
  48. enabled = yes
  49. destination = 203.0.113.0
  50. api key = 11111111-2222-3333-4444-555555555555
  51. ```
  52. Save the file and close it, then restart Netdata with `sudo systemctl restart netdata`, or the [appropriate
  53. method](/docs/configure/start-stop-restart.md) for your system.
  54. ## Enable TLS/SSL on streaming (optional)
  55. While encrypting the connection between your parent and child nodes is recommended for security, it's not required to
  56. get started. If you're not interested in encryption, skip ahead to [view streamed
  57. metrics](#view-streamed-metrics-in-netdatas-dashboard).
  58. In this example, we'll use self-signed certificates.
  59. On the **parent** node, use OpenSSL to create the key and certificate, then use `chown` to make the new files readable
  60. by the `netdata` user.
  61. ```bash
  62. sudo openssl req -newkey rsa:2048 -nodes -sha512 -x509 -days 365 -keyout /etc/netdata/ssl/key.pem -out /etc/netdata/ssl/cert.pem
  63. sudo chown netdata:netdata /etc/netdata/ssl/cert.pem /etc/netdata/ssl/key.pem
  64. ```
  65. Next, enforce TLS/SSL on the web server. Open `netdata.conf`, scroll down to the `[web]` section, and look for the `bind
  66. to` setting. Add `^SSL=force` to turn on TLS/SSL. See the [web server
  67. reference](/web/server/README.md#enabling-tls-support) for other TLS/SSL options.
  68. ```conf
  69. [web]
  70. bind to = *=dashboard|registry|badges|management|streaming|netdata.conf^SSL=force
  71. ```
  72. Next, connect to the **child** node and open `stream.conf`. Add `:SSL` to the end of the existing `destination` setting
  73. to connect to the parent using TLS/SSL. Uncomment the `ssl skip certificate verification` line to allow the use of
  74. self-signed certificates.
  75. ```conf
  76. [stream]
  77. enabled = yes
  78. destination = 203.0.113.0:SSL
  79. ssl skip certificate verification = yes
  80. api key = 11111111-2222-3333-4444-555555555555
  81. ```
  82. Restart both the parent and child nodes with `sudo systemctl restart netdata`, or the [appropriate
  83. method](/docs/configure/start-stop-restart.md) for your system, to stream encrypted metrics using TLS/SSL.
  84. ## View streamed metrics in Netdata's dashboard
  85. At this point, the child node is streaming its metrics in real time to its parent. Open the local Agent dashboard for
  86. the parent by navigating to `http://PARENT-NODE:19999` in your browser, replacing `PARENT-NODE` with its IP address or
  87. hostname.
  88. This dashboard shows parent metrics. To see child metrics, open the left-hand sidebar with the hamburger icon
  89. ![Hamburger icon](https://raw.githubusercontent.com/netdata/netdata-ui/master/src/components/icon/assets/hamburger.svg)
  90. in the top panel. Both nodes appear under the **Replicated Nodes** menu. Click on either of the links to switch between
  91. separate parent and child dashboards.
  92. ![Switching between parent and child
  93. dashboards](https://user-images.githubusercontent.com/1153921/110043346-761ec000-7d04-11eb-8e58-77670ba39161.gif)
  94. The child dashboard is also available directly at `http://PARENT-NODE:19999/host/CHILD-HOSTNAME`, which in this example
  95. is `http://203.0.113.0:19999/host/netdata-child`.
  96. ## What's next?
  97. Now that you have a basic streaming setup with replication, you may want to tweak the configuration to eliminate the
  98. child database, disable the child dashboard, or enable SSL on the streaming connection between the parent and child.
  99. See the [streaming reference
  100. doc](/docs/metrics-storage-management/reference-streaming.mdx#examples) for details about
  101. other possible configurations.
  102. When using Netdata's default TSDB (`dbengine`), the parent node maintains separate, parallel databases for itself and
  103. every child node streaming to it. Each instance is sized identically based on the `dbengine multihost disk space`
  104. setting in `netdata.conf`. See our doc on [changing metrics retention](/docs/store/change-metrics-storage.md) for
  105. details.
  106. ### Related information & further reading
  107. - Streaming
  108. - [How Netdata streams metrics](/docs/metrics-storage-management/how-streaming-works.mdx)
  109. - **[Enable streaming between nodes](/docs/metrics-storage-management/enable-streaming.mdx)**
  110. - [Streaming reference](/docs/metrics-storage-management/reference-streaming.mdx)