netdata/integrations/cloud-notifications/metadata.yaml

# yamllint disable rule:line-length
---
- id: 'notify-cloud-mobile-app'
  meta:
    name: 'Netdata Mobile App'
    link: 'https://netdata.cloud'
    categories:
      - notify.cloud
    icon_filename: 'netdata.png'
  keywords:
    - mobile-app
    - phone
    - personal-notifications
  overview:
    notification_description: "From the Netdata Cloud UI, you can manage your user notification settings and enable the configuration to deliver notifications on the Netdata Mobile Application."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites
      - A Netdata Cloud account
      - You need to have the Netdata Mobile Application installed on your [Android](https://play.google.com/store/apps/details?id=cloud.netdata.android&pli=1) or [iOS](https://apps.apple.com/in/app/netdata-mobile/id6474659622) phone.

      ### Netdata Mobile App Configuration
      Steps to login to the Netdata Mobile Application to receive alert and reachability and alert notifications:
      1. Download the Netdata Mobile Application from [Google Play Store](https://play.google.com/store/apps/details?id=cloud.netdata.android&pli=1) or the [iOS App Store](https://apps.apple.com/in/app/netdata-mobile/id6474659622)
      2. Open the App and Choose the Sign In Option
        - Sign In with Email Address: Enter the Email Address of your registered Netdata Cloud Account and Click on the Verification link received by Email on your mobile device.
        - Sign In with QR Code: Scan the QR Code from your `Netdata Cloud` UI under **User Settings** --> **Notifications** --> **Mobile App Notifications** --> **Show QR Code**
      3. Start receiving alert and reachability notifications for your **Space(s)** on a **Paid Subscription plan**

      ### Netdata Configuration Steps
      1. Click on the **User settings** on the bottom left of your screen (your profile icon)
      2. Click on the **Notifications** tab
      3. Enable **Mobile App Notifications** if disabled (Enabled by default)
      4. Use the **Show QR Code** Option to login to your mobile device by scanning the **QR Code**
- id: 'notify-cloud-discord'
  meta:
    name: 'Discord'
    link: 'https://discord.com/'
    categories:
      - notify.cloud
    icon_filename: 'discord.png'
  keywords:
    - discord
    - community
  overview:
    notification_description: "From the Netdata Cloud UI, you can manage your space's notification settings and enable the configuration to deliver notifications on Discord."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites
      - A Netdata Cloud account
      - Access to the Netdata Space as an **Admin**
      - You need to have a Discord server able to receive webhooks integrations.

      ### Discord Server Configuration
      Steps to configure your Discord server to receive [webhook notifications](https://support.discord.com/hc/en-us/articles/228383668) from Netdata:
      1. Go to `Server Settings` --> `Integrations`
      2. **Create Webhook** or **View Webhooks** if you already have some defined
      3. Specify the **Name** and **Channel** on your new webhook
      4. Use Webhook URL to add your notification configuration on Netdata UI

      ### Netdata Configuration Steps
      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **Discord** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        * **Notification settings** are Netdata specific settings
          - Configuration name - you can optionally provide a name for your configuration you can easily refer to it
          - Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration
          - Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only
        * **Integration configuration** are the specific notification integration required settings, which vary by notification method. For Discord:
          - Define the type channel you want to send notifications to: **Text channel** or **Forum channel**
          - Webhook URL - URL provided on Discord for the channel you want to receive your notifications.
          - Thread name - if the Discord channel is a **Forum channel** you will need to provide the thread name as well

- id: 'notify-cloud-pagerduty'
  meta:
    name: 'PagerDuty'
    link: 'https://www.pagerduty.com/'
    categories:
      - notify.cloud
    icon_filename: 'pagerduty.png'
  keywords:
    - pagerduty
  overview:
    notification_description: "From the Netdata Cloud UI, you can manage your space's notification settings and enable the configuration to deliver notifications on PagerDuty."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites
      - A Netdata Cloud account
      - Access to the Netdata Space as an **Admin**
      - The Netdata Space needs to be on a paid plan
      - You need to have a PagerDuty service to receive events using webhooks.


      ### PagerDuty Server Configuration
      Steps to configure your PagerDuty to receive notifications from Netdata:

      1. Create a service to receive events from your services directory page on PagerDuty
      2. At step 3, select `Events API V2` Integration or **View Webhooks** if you already have some defined
      3. Once the service is created, you will be redirected to its configuration page, where you can copy the **Integration Key** and **Integration URL (Alert Events)** fields to add them to your notification configuration in the Netdata UI.

      ### Netdata Configuration Steps

      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **PagerDuty** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        * **Notification settings** are Netdata specific settings
          - Configuration name - you can optionally provide a name for your configuration you can easily refer to it
          - Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration
          - Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only
        * **Integration configuration** are the specific notification integration required settings, which vary by notification method. For PagerDuty:
          - Integration Key - is a 32 character key provided by PagerDuty to receive events on your service.
          - Integration URL (Alert Events) - is the URL provided by PagerDuty where we will send notifications.

- id: 'notify-cloud-slack'
  meta:
    name: 'Slack'
    link: 'https://slack.com/'
    categories:
      - notify.cloud
    icon_filename: 'slack.png'
  keywords:
    - slack
  overview:
    notification_description: "From the Netdata Cloud UI, you can manage your space's notification settings and enable the configuration to deliver notifications on Slack."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites

      - A Netdata Cloud account
      - Access to the Netdata Space as an **Admin**
      - The Netdata Space needs to be on a paid plan
      - You need to have a Slack app on your workspace to receive the Webhooks.

      ### Slack Server Configuration

      Steps to configure your Slack to receive notifications from Netdata:

      1. Create an app to receive webhook integrations. Check [Create an app](https://api.slack.com/apps?new_app=1) from Slack documentation for further details
      2. Install the app on your workspace
      3. Configure Webhook URLs for your workspace
        - On your app go to **Incoming Webhooks** and click on **activate incoming webhooks**
        - At the bottom of **Webhook URLs for Your Workspace** section you have **Add New Webhook to Workspace**
        - After pressing that specify the channel where you want your notifications to be delivered
        - Once completed copy the Webhook URL that you will need to add to your notification configuration on Netdata UI

      For more details please check Slacks's article [Incoming webhooks for Slack](https://slack.com/help/articles/115005265063-Incoming-webhooks-for-Slack).

      ### Netdata Configuration Steps

      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **Slack** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        * **Notification settings** are Netdata specific settings
          - Configuration name - you can optionally provide a name for your configuration you can easily refer to it
          - Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration
          - Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only
        * **Integration configuration** are the specific notification integration required settings, which vary by notification method. For Slack:
          - Webhook URL - URL provided on Slack for the channel you want to receive your notifications.

- id: 'notify-cloud-opsgenie'
  meta:
    name: 'Opsgenie'
    link: 'https://www.atlassian.com/software/opsgenie'
    categories:
      - notify.cloud
    icon_filename: 'opsgenie.png'
  keywords:
    - opsgenie
    - atlassian
  overview:
    notification_description: "From the Netdata Cloud UI, you can manage your space's notification settings and enable the configuration to deliver notifications on Opsgenie."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites

      - A Netdata Cloud account
      - Access to the Netdata Space as an **Admin**
      - The Netdata Space needs to be on a paid plan
      - You need to have permissions on Opsgenie to add new integrations.

      ### Opsgenie Server Configuration

      Steps to configure your Opsgenie to receive notifications from Netdata:

      1. Go to integrations tab of your team, click **Add integration**
      2. Pick **API** from available integrations. Copy your API Key and press **Save Integration**.
      3. Paste copied API key into the corresponding field in **Integration configuration** section of Opsgenie modal window in Netdata.

      ### Netdata Configuration Steps

      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **Opsgenie** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        * **Notification settings** are Netdata specific settings
          - Configuration name - you can optionally provide a name for your configuration you can easily refer to it
          - Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration
          - Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only
        * **Integration configuration** are the specific notification integration required settings, which vary by notification method. For Opsgenie:
          - API Key - a key provided on Opsgenie for the channel you want to receive your notifications.

- id: 'notify-cloud-mattermost'
  meta:
    name: 'Mattermost'
    link: 'https://mattermost.com/'
    categories:
      - notify.cloud
    icon_filename: 'mattermost.png'
  keywords:
    - mattermost
  overview:
    notification_description: "From the Netdata Cloud UI, you can manage your space's notification settings and enable the configuration to deliver notifications on Mattermost."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites

      - A Netdata Cloud account
      - Access to the Netdata Space as an **Admin**
      - The Netdata Space needs to be on a paid plan
      - You need to have permissions on Mattermost to add new integrations.
      - You need to have a Mattermost app on your workspace to receive the webhooks.

      ### Mattermost Server Configuration

      Steps to configure your Mattermost to receive notifications from Netdata:

      1. In Mattermost, go to Product menu > Integrations > Incoming Webhook
        - If you don’t have the Integrations option, incoming webhooks may not be enabled on your Mattermost server or may be disabled for non-admins. They can be enabled by a System Admin from System Console > Integrations > Integration Management. Once incoming webhooks are enabled, continue with the steps below.
      2. Select Add Incoming Webhook and add a name and description for the webhook. The description can be up to 500 characters
      3. Select the channel to receive webhook payloads, then select Add to create the webhook
      4. You will end up with a webhook endpoint that looks like below:
        `https://your-mattermost-server.com/hooks/xxx-generatedkey-xxx`

        - Treat this endpoint as a secret. Anyone who has it will be able to post messages to your Mattermost instance.

      For more details please check Mattermost's article [Incoming webhooks for Mattermost](https://developers.mattermost.com/integrate/webhooks/incoming/).

      ### Netdata Configuration Steps

      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **Mattermost** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        * **Notification settings** are Netdata specific settings
          - Configuration name - you can optionally provide a name for your configuration you can easily refer to it
          - Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration
          - Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only
        * **Integration configuration** are the specific notification integration required settings, which vary by notification method. For Mattermost:
          - Webhook URL - URL provided on Mattermost for the channel you want to receive your notifications

- id: 'notify-cloud-rocketchat'
  meta:
    name: 'RocketChat'
    link: 'https://www.rocket.chat/'
    categories:
      - notify.cloud
    icon_filename: 'rocketchat.png'
  keywords:
    - rocketchat
  overview:
    notification_description: "From the Netdata Cloud UI, you can manage your space's notification settings and enable the configuration to deliver notifications on RocketChat."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites

      - A Netdata Cloud account
      - Access to the Netdata Space as an **Admin**
      - The Netdata Space needs to be on a paid plan
      - You need to have permissions on Mattermost to add new integrations.
      - You need to have a RocketChat app on your workspace to receive the webhooks.

      ### Mattermost Server Configuration

      Steps to configure your RocketChat to receive notifications from Netdata:

      1. In RocketChat, Navigate to Administration > Workspace > Integrations.
      2. Click **+New** at the top right corner.
      3. For more details about each parameter, check [create-a-new-incoming-webhook](https://docs.rocket.chat/use-rocket.chat/workspace-administration/integrations#create-a-new-incoming-webhook).
      4. After configuring integration, click Save.
      5. You will end up with a webhook endpoint that looks like below:
        `https://your-server.rocket.chat/hooks/YYYYYYYYYYYYYYYYYYYYYYYY/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`
        - Treat this endpoint as a secret. Anyone who has it will be able to post messages to your RocketChat instance.


      For more details please check RocketChat's article Incoming webhooks for [RocketChat](https://docs.rocket.chat/use-rocket.chat/workspace-administration/integrations/).

      ### Netdata Configuration Steps

      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **RocketChat** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        * **Notification settings** are Netdata specific settings
          - Configuration name - you can optionally provide a name for your configuration you can easily refer to it
          - Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration
          - Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only
        * **Integration configuration** are the specific notification integration required settings, which vary by notification method. For RocketChat:
          - Webhook URL - URL provided on RocketChat for the channel you want to receive your notifications.

- id: 'notify-cloud-awssns'
  meta:
    name: 'Amazon SNS'
    link: 'https://aws.amazon.com/sns/'
    categories:
      - notify.cloud
    icon_filename: 'awssns.png'
  keywords:
    - awssns
  overview:
    notification_description: "From the Cloud interface, you can manage your space's notification settings and from these you can add a specific configuration to get notifications delivered on AWS SNS."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites

      To add AWS SNS notification you need:

      - A Netdata Cloud account
      - Access to the space as an **Admin**
      - The Space needs to be on a paid plan
      - Have an AWS account with AWS SNS access, for more details check [how to configure this on AWS SNS](#settings-on-aws-sns)

      ### Steps

      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **AwsSns** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        * **Notification settings** are Netdata specific settings
            - Configuration name - you can optionally provide a name for your configuration  you can easily refer to it
            - Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration
            - Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only
        * **Integration configuration** are the specific notification integration required settings, which vary by notification method. For AWS SNS:
            - Topic ARN - topic provided on AWS SNS (with region) for where to publish your notifications. For more details check [how to configure this on AWS SNS](#settings-on-aws-sns)

      ### Settings on AWS SNS

      To enable the webhook integration on AWS SNS you need:
      1. [Setting up access for Amazon SNS](https://docs.aws.amazon.com/sns/latest/dg/sns-setting-up.html)
      2. Create a topic
          - On AWS SNS management console click on **Create topic**
          - On the **Details** section, the standard type and provide the topic name
          - On the **Access policy** section, change the **Publishers** option to **Only the specified AWS accounts** and provide the Netdata AWS account **(123269920060)** that will be used to publish notifications to the topic being created
          - Finally, click on **Create topic** on the bottom of the page
      3. Now, use the new **Topic ARN** while adding AWS SNS integration on your space.

- id: 'notify-cloud-microsoftteams'
  meta:
    name: 'Microsoft Teams'
    link: 'https://www.microsoft.com/en-us/microsoft-teams'
    categories:
      - notify.cloud
    icon_filename: 'teams.svg'
  keywords:
    - microsoft
    - teams
  overview:
    notification_description: "From the Netdata Cloud UI, you can manage your space's notification settings and enable the configuration to deliver notifications to a Microsoft Teams channel."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites

      To add Microsoft Teams notifications integration to your Netdata Cloud space you will need the following:

      - A Netdata Cloud account.
      - Access to the Netdata Cloud space as an **Admin**.
      - The Space to be on a paid plan.
      - A [Microsoft Teams Essentials subscription](https://www.microsoft.com/en-sg/microsoft-teams/essentials) or higher. Note that this is a **paid** feature.

      ### Settings on Microsoft Teams

      1. **Access the Channel Settings**: Navigate to the desired Microsoft Teams channel and hover over the channel name. Click the three dots (ellipsis) icon that appears.
      2. **Create a New Workflow**: Select "Workflows" from the options, then choose "Post to a channel when a webhook request is received."
      3. **Configure Workflow Details**:
          - Give your workflow a descriptive name, such as "Netdata Alerts."
          - Select the target team and channel to receive notifications.
          - Click "Add workflow."
      4. **Obtain the Webhook URL**:
          - Once the workflow is created, you will receive a unique Workflow Webhook URL.
          - Copy this URL, as it will be required to configure Netdata Cloud.

      ### Settings on Netdata Cloud

      1. Click on the **Space settings** cog (located above your profile icon).
      2. Click on the **Notification** tab.
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen).
      4. On the **Microsoft Teams** card click on **+ Add**.
      5. A modal will be presented to you to enter the required details to enable the configuration:
        * **Notification settings** are Netdata specific settings:
            - Configuration name - you can optionally provide a name for your configuration  you can easily refer to it.
            - Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration.
            - Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only.
        * **Integration configuration** are the specific notification integration required settings, which vary by notification method. For Microsoft Teams:
            - Microsoft Teams Incoming Webhook URL - the _Incoming Webhook URL_ that was generated earlier.


- id: 'notify-cloud-telegram'
  meta:
    name: 'Telegram'
    link: 'https://telegram.org/'
    categories:
      - notify.cloud
    icon_filename: 'telegram.svg'
  keywords:
    - Telegram
  overview:
    notification_description: "From the Cloud interface, you can manage your space's notification settings and from these you can add a specific configuration to get notifications delivered on Telegram."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites

      To add Telegram notification you need:

      - A Netdata Cloud account
      - Access to the space as an **Admin**
      - The Space needs to be on a paid plan
      - The Telegram bot token, chat ID and _optionally_ the topic ID

      ### Steps

      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **Telegram** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        - **Notification settings** are Netdata specific settings
            - Configuration name - you can optionally provide a name for your configuration  you can easily refer to it
            - Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration
            - Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only
        - **Integration configuration** are the specific notification integration required settings, which vary by notification method. For Telegram:
            - Bot Token - the token of your bot
            - Chat ID - the chat id where your bot will deliver messages to
            - Topic ID - the identifier of the chat topic to which your bot will send messages. If omitted or 0, messages will be sent to the General topic. If topics are not supported, messages will be sent to the chat.

      ### Getting the Telegram bot token, chat ID and topic ID

      - Bot token: To create one bot, contact the [@BotFather](https://t.me/BotFather) bot and send the command `/newbot` and follow the instructions. **Start a conversation with your bot or invite it into the group where you want it to send notifications**.
      - To get the chat ID you have two options:
          - Contact the [@myidbot](https://t.me/myidbot) bot and send the `/getid` command to get your personal chat ID, or invite it into a group and use the `/getgroupid` command to get the group chat ID.
          - Alternatively, you can get the chat ID directly from the bot API. Send your bot a command in the chat you want to use, then check `https://api.telegram.org/bot{YourBotToken}/getUpdates`, eg. `https://api.telegram.org/bot111122223:7OpFlFFRzRBbrUUmIjj5HF9Ox2pYJZy5/getUpdates`
      - To get the topic ID, the easiest way is this: Post a message to that topic, then right-click on it and select `Copy Message Link`. Paste it on a scratchpad and notice that it has the following structure `https://t.me/c/XXXXXXXXXX/YY/ZZ`. The topic ID is `YY` (integer).
- id: 'notify-cloud-splunk'
  meta:
    name: 'Splunk'
    link: 'https://splunk.com/'
    categories:
      - notify.cloud
    icon_filename: 'splunk-black.svg'
  keywords:
    - Splunk
  overview:
    notification_description: "From the Cloud interface, you can manage your space's notification settings and from these you can add a specific configuration to get notifications delivered on Splunk."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites

      To add Splunk notification you need:

      - A Netdata Cloud account
      - Access to the space as an **Admin**
      - The Space needs to be on a paid plan
      - URI and token for your Splunk HTTP Event Collector. Refer to the [Splunk documentation](https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector) for detailed instructions.

      ### Steps

      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **Splunk** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        - **Notification settings** are Netdata specific settings
            - Configuration name - provide a descriptive name for your configuration to easily identify it.
            - Rooms - select the nodes or areas of your infrastructure you want to receive notifications about.
            - Notification - choose the type of notifications you want to receive: All Alerts and unreachable, All Alerts, Critical only.
        - **Integration configuration** are the specific notification integration required settings, which vary by notification method. For Splunk:
            - HTTP Event Collector URI - The URI of your HTTP event collector in Splunk
            - HTTP Event Collector Token - the token that Splunk provided to you when you created the HTTP Event Collector

- id: 'notify-cloud-victorops'
  meta:
    name: 'Splunk VictorOps'
    link: 'https://www.splunk.com/en_us/about-splunk/acquisitions/splunk-on-call.html'
    categories:
      - notify.cloud
    icon_filename: 'victorops.svg'
  keywords:
    - VictorOps
    - Splunk
    - On-Call
  overview:
    notification_description: "From the Cloud interface, you can manage your space's notification settings and from there you can add a specific configuration to get notifications delivered on Splunk On-Call/VictorOps."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites

      To add Splunk VictorOps notification (also known as Splunk On-Call) you need:

      - A Netdata Cloud account
      - Access to the space as an **Admin**
      - The Space needs to be on a paid plan
      - Destination URL for your Splunk VictorOps REST Endpoint Integration. Refer to the [VictorOps documentation](https://help.victorops.com/knowledge-base/rest-endpoint-integration-guide) for detailed instructions.

      ### Steps

      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **Splunk VictorOps** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        - **Notification settings** are Netdata specific settings
            - Configuration name - provide a descriptive name for your configuration to easily identify it.
            - Rooms - select the nodes or areas of your infrastructure you want to receive notifications about.
            - Notification - choose the type of notifications you want to receive: All Alerts and unreachable, All Alerts, Critical only.
        - **Integration configuration** are the specific notification integration required settings, which vary by notification method. For Splunk VictorOps:
            - Destination URL - The URL provided by VictorOps of your REST endpoint.

- id: 'notify-cloud-webhook'
  meta:
    name: 'Webhook'
    link: 'https://en.wikipedia.org/wiki/Webhook'
    categories:
      - notify.cloud
    icon_filename: 'webhook.svg'
  keywords:
    - generic webhooks
    - webhooks
  overview:
    notification_description: "From the Netdata Cloud UI, you can manage your space's notification settings and enable the configuration to deliver notifications on a webhook using a predefined schema."
    notification_limitations: ''
  setup:
    description: |
      ### Prerequisites

      - A Netdata Cloud account
      - Access to the Netdata Space as an **Admin**
      - The Netdata Space needs to be on a paid plan
      - You need to have an app that allows you to receive webhooks following a predefined schema.

      ### Netdata Configuration Steps

      1. Click on the **Space settings** cog (located above your profile icon)
      2. Click on the **Notification** tab
      3. Click on the **+ Add configuration** button (near the top-right corner of your screen)
      4. On the **Webhook** card click on **+ Add**
      5. A modal will be presented to you to enter the required details to enable the configuration:
        * **Notification settings** are Netdata specific settings
          - Configuration name - you can optionally provide a name for your configuration you can easily refer to it
          - Rooms - by specifying a list of Rooms you are select to which nodes or areas of your infrastructure you want to be notified using this configuration
          - Notification - you specify which notifications you want to be notified using this configuration: All Alerts and unreachable, All Alerts, Critical only
        * **Integration configuration** are the specific notification integration required settings, which vary by notification method. For Webhook:
          - Webhook URL - webhook URL is the url of the service that Netdata will send notifications to. In order to keep the communication secured, we only accept HTTPS urls.
          - Extra headers - these are optional key-value pairs that you can set to be included in the HTTP requests sent to the webhook URL.
          - Authentication Mechanism - Netdata webhook integration supports 3 different authentication mechanisms.
            * Mutual TLS (recommended) - default authentication mechanism used if no other method is selected.
            * Basic - the client sends a request with an Authorization header that includes a base64-encoded string in the format **username:password**. These will settings will be required inputs.
            * Bearer - the client sends a request with an Authorization header that includes a **bearer token**. This setting will be a required input.


        ### Webhook service

        A webhook integration allows your application to receive real-time alerts from Netdata by sending HTTP requests to a specified URL. In this document, we'll go over the steps to set up a generic webhook integration, including adding headers, and implementing different types of authorization mechanisms.

        #### Netdata webhook integration

        A webhook integration is a way for one service to notify another service about events that occur within it. This is done by sending an HTTP POST request to a specified URL (known as the "webhook URL") when an event occurs.

        Netdata webhook integration service will send alert and reachability notifications to the destination service as soon as they are detected.

        For alert notifications, the content sent to the destination service contains a JSON object with the following properties:

        | field   | type   | description |
        | :--     | :--    | :--         |
        | message | string | A summary message of the alert. |
        | alarm | string | The alarm the notification is about. |
        | info | string | Additional info related with the alert. |
        | chart | string | The chart associated with the alert. |
        | context | string | The chart context. |
        | space | string | The space where the node that raised the alert is assigned. |
        | Rooms | object[object(string,string)] | Object with list of Rooms names and urls where the node belongs to. |
        | family | string | Context family. |
        | class | string | Classification of the alert, e.g. "Error". |
        | severity | string | Alert severity, can be one of "warning", "critical" or "clear". |
        | date | string | Date of the alert in ISO8601 format. |
        | duration | string |  Duration the alert has been raised. |
        | additional_active_critical_alerts | integer | Number of additional critical alerts currently existing on the same node. |
        | additional_active_warning_alerts | integer | Number of additional warning alerts currently existing on the same node. |
        | alarm_url | string | Netdata Cloud URL for this alarm. |

        For reachability notifications, the JSON object will contain the following properties:

        | field        | type   | description |
        | :--          | :--    | :--         |
        | message      | string | A summary message of the reachability alert. |
        | url          | string | Netdata Cloud URL for the host experiencing the reachability alert. |
        | host         | string | the host experiencing the reachability alert. |
        | severity     | string | severity for this notification. If host is reachable, severity will be 'info', if host is unreachable, it will be 'critical'. |
        | status       | object | an object with the status information. |
        | status.reachable  | boolean | true if host is reachable, false otherwise |
        | status.text       | string  | can be 'reachable' or 'unreachable'  |

        #### Extra headers

        When setting up a webhook integration, the user can specify a set of headers to be included in the HTTP requests sent to the webhook URL.

        By default, the following headers will be sent in the HTTP request

        |            **Header**            | **Value**                 |
        |:-------------------------------:|-----------------------------|
        |     Content-Type             | application/json        |

        #### Authentication mechanisms

        Netdata webhook integration supports 3 different authentication mechanisms:

        ##### Mutual TLS authentication (recommended)

        In mutual Transport Layer Security (mTLS) authentication, the client and the server authenticate each other using X.509 certificates. This ensures that the client is connecting to the intended server, and that the server is only accepting connections from authorized clients.

        This is the default authentication mechanism used if no other method is selected.

        To take advantage of mutual TLS, you can configure your server to verify Netdata's client certificate. In order to achieve this, the Netdata client sending the notification supports mutual TLS (mTLS) to identify itself with a client certificate that your server can validate.

        The steps to perform this validation are as follows:

        - Store Netdata CA certificate on a file in your disk. The content of this file should be:

        <details>
          <summary>Netdata CA certificate</summary>

        ```
        -----BEGIN CERTIFICATE-----
        MIIF0jCCA7qgAwIBAgIUDV0rS5jXsyNX33evHEQOwn9fPo0wDQYJKoZIhvcNAQEN
        BQAwgYAxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQH
        Ew1TYW4gRnJhbmNpc2NvMRYwFAYDVQQKEw1OZXRkYXRhLCBJbmMuMRIwEAYDVQQL
        EwlDbG91ZCBTUkUxGDAWBgNVBAMTD05ldGRhdGEgUm9vdCBDQTAeFw0yMzAyMjIx
        MjQzMDBaFw0zMzAyMTkxMjQzMDBaMIGAMQswCQYDVQQGEwJVUzETMBEGA1UECBMK
        Q2FsaWZvcm5pYTEWMBQGA1UEBxMNU2FuIEZyYW5jaXNjbzEWMBQGA1UEChMNTmV0
        ZGF0YSwgSW5jLjESMBAGA1UECxMJQ2xvdWQgU1JFMRgwFgYDVQQDEw9OZXRkYXRh
        IFJvb3QgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCwIg7z3R++
        ppQYYVVoMIDlhWO3qVTMsAQoJYEvVa6fqaImUBLW/k19LUaXgUJPohB7gBp1pkjs
        QfY5dBo8iFr7MDHtyiAFjcQV181sITTMBEJwp77R4slOXCvrreizhTt1gvf4S1zL
        qeHBYWEgH0RLrOAqD0jkOHwewVouO0k3Wf2lEbCq3qRk2HeDvkv0LR7sFC+dDms8
        fDHqb/htqhk+FAJELGRqLeaFq1Z5Eq1/9dk4SIeHgK5pdYqsjpBzOTmocgriw6he
        s7F3dOec1ZZdcBEAxOjbYt4e58JwuR81cWAVMmyot5JNCzYVL9e5Vc5n22qt2dmc
        Tzw2rLOPt9pT5bzbmyhcDuNg2Qj/5DySAQ+VQysx91BJRXyUimqE7DwQyLhpQU72
        jw29lf2RHdCPNmk8J1TNropmpz/aI7rkperPugdOmxzP55i48ECbvDF4Wtazi+l+
        4kx7ieeLfEQgixy4lRUUkrgJlIDOGbw+d2Ag6LtOgwBiBYnDgYpvLucnx5cFupPY
        Cy3VlJ4EKUeQQSsz5kVmvotk9MED4sLx1As8V4e5ViwI5dCsRfKny7BeJ6XNPLnw
        PtMh1hbiqCcDmB1urCqXcMle4sRhKccReYOwkLjLLZ80A+MuJuIEAUUuEPCwywzU
        R7pagYsmvNgmwIIuJtB6mIJBShC7TpJG+wIDAQABo0IwQDAOBgNVHQ8BAf8EBAMC
        AQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQU9IbvOsPSUrpr8H2zSafYVQ9e
        Ft8wDQYJKoZIhvcNAQENBQADggIBABQ08aI31VKZs8jzg+y/QM5cvzXlVhcpkZsY
        1VVBr0roSBw9Pld9SERrEHto8PVXbadRxeEs4sKivJBKubWAooQ6NTvEB9MHuGnZ
        VCU+N035Gq/mhBZgtIs/Zz33jTB2ju3G4Gm9VTZbVqd0OUxFs41Iqvi0HStC3/Io
        rKi7crubmp5f2cNW1HrS++ScbTM+VaKVgQ2Tg5jOjou8wtA+204iYXlFpw9Q0qnP
        qq6ix7TfLLeRVp6mauwPsAJUgHZluz7yuv3r7TBdukU4ZKUmfAGIPSebtB3EzXfH
        7Y326xzv0hEpjvDHLy6+yFfTdBSrKPsMHgc9bsf88dnypNYL8TUiEHlcTgCGU8ts
        ud8sWN2M5FEWbHPNYRVfH3xgY2iOYZzn0i+PVyGryOPuzkRHTxDLPIGEWE5susM4
        X4bnNJyKH1AMkBCErR34CLXtAe2ngJlV/V3D4I8CQFJdQkn9tuznohUU/j80xvPH
        FOcDGQYmh4m2aIJtlNVP6+/92Siugb5y7HfslyRK94+bZBg2D86TcCJWaaZOFUrR
        Y3WniYXsqM5/JI4OOzu7dpjtkJUYvwtg7Qb5jmm8Ilf5rQZJhuvsygzX6+WM079y
        nsjoQAm6OwpTN5362vE9SYu1twz7KdzBlUkDhePEOgQkWfLHBJWwB+PvB1j/cUA3
        5zrbwvQf
        -----END CERTIFICATE-----
        ```
        </details>

        - Enable client certificate validation on the web server that is doing the TLS termination. Below we show you how to perform this configuration in `NGINX` and `Apache`

         **NGINX**

        ```bash
        server {
            listen 443 ssl default_server;

            # ... existing SSL configuration for server authentication ...
            ssl_verify_client on;
            ssl_client_certificate /path/to/Netdata_CA.pem;

            location / {
                if ($ssl_client_s_dn !~ "CN=app.netdata.cloud") {
                    return 403;
                }
               # ... existing location configuration ...
            }
        }
        ```

        **Apache**

        ```bash
        Listen 443
        <VirtualHost *:443>
            # ... existing SSL configuration for server authentication ...
            SSLVerifyClient require
            SSLCACertificateFile "/path/to/Netdata_CA.pem"
        </VirtualHost>
        <Directory /var/www/>
            Require expr "%{SSL_CLIENT_S_DN_CN} == 'app.netdata.cloud'"
            # ... existing directory configuration ...
        </Directory>
        ```

        ##### Basic authentication

        In basic authorization, the client sends a request with an Authorization header that includes a base64-encoded string in the format username:password. The server then uses this information to authenticate the client. If this authentication method is selected, the user can set the user and password that will be used when connecting to the destination service.

        ##### Bearer token authentication

        In bearer token authentication, the client sends a request with an Authorization header that includes a bearer token. The server then uses this token to authenticate the client. Bearer tokens are typically generated by an authentication service, and are passed to the client after a successful authentication. If this method is selected, the user can set the token to be used for connecting to the destination service.

        ##### Challenge secret

        To validate that you have ownership of the web application that will receive the webhook events, we are using a challenge response check mechanism.

        This mechanism works as follows:

        - The challenge secret parameter that you provide is a shared secret between you and Netdata only.
        - On your request for creating a new Webhook integration, we will make a GET request to the url of the webhook, adding a query parameter `crc_token`, consisting of a random string.
        - You will receive this request on your application and it must construct an encrypted response, consisting of a base64-encoded HMAC SHA-256 hash created from the crc_token and the shared secret. The response will be in the format:

        ```json
        {
          "response_token": "sha256=9GKoHJYmcHIkhD+C182QWN79YBd+D+Vkj4snmZrfNi4="
        }
        ```

        - We will compare your application's response with the hash that we will generate using the challenge secret, and if they are the same, the integration creation will succeed.

        We will do this validation everytime you update your integration configuration.

        - Response requirements:
            - A base64 encoded HMAC SHA-256 hash created from the crc_token and the shared secret.
            - Valid response_token and JSON format.
            - Latency less than 5 seconds.
            - 200 HTTP response code.

        **Example response token generation in Python:**

        Here you can see how to define a handler for a Flask application in python 3:

        ```python
        import base64
        import hashlib
        import hmac
        import json

        key ='YOUR_CHALLENGE_SECRET'

        @app.route('/webhooks/netdata')
        def webhook_challenge():
          token = request.args.get('crc_token').encode('ascii')

          # creates HMAC SHA-256 hash from incomming token and your consumer secret
          sha256_hash_digest = hmac.new(key.encode(),
                                        msg=token,
                                        digestmod=hashlib.sha256).digest()

          # construct response data with base64 encoded hash
          response = {
            'response_token': 'sha256=' + base64.b64encode(sha256_hash_digest).decode('ascii')
          }

          # returns properly formatted json response
          return json.dumps(response)
        ```