Custom Events

Sysdig Monitor can receive and process Custom Events that you defined, including code deploys, auto-scaling activities, and business-level operations. You can visualize these Custom Events in the Event feed or as an Event Overlay. Additionally, you can configure Event Alerts based on Custom Events. The following sections describe the different ways you can leverage Custom Events in Sysdig Monitor.

Custom Event visibility is restricted to the same team from which the custom event was sent.

The Sysdig Monitor Slackbot named Sysdigbot allows you to post Custom Events directly to the Sysdig Cloud through Slack messages.

Use Cases

Custom events can give visibility in the following use cases.

  • CI/CD pipelines can include build information in custom events. When visualized using the event overlay, users can understand important metrics before and after a deploy.
  • Terraform can be configured to post a custom event upon running terraform apply, allowing you to trace infrastructure changes alongside metrics.
  • Custom Events can be included in tasks or jobs in order to understand when and where a failure has occurred.
  • Custom Events can be included in a systemd unit file to track how often a service has restarted.
  • Use a custom controller or an operator in Kubernetes to listen for certain kubernetes events and send a custom event to Sysdig Monitor

Create a Custom Event

Send Custom Events with Python

The SDCClient can be used to send arbitrary custom events to Sysdig Monitor. This client acts as a wrapper around the Sysdig Monitor Rest API, exposing most of the REST API functions through the Python interface.

from sdcclient import SdMonitorClient

# Replace sdc_url with region URL
sdclient = SdMonitorClient(token="<sysdig_api_token>", sdc_url="https://app.sysdigcloud.com")

# Send custom event
print(sdcclient.post_event('sample_custom_event'))

# Send custom event with description, severity, scope, and tags
print(sdclient.post_event(name='Payment Service Restart', description="Payment Service has been restarted", severity="LOW", tags={"service":"payment"}, event_filter='component = "frontend" and application = "loadbalancer"'))

Post a Custom Event for each CI/CD deploy with curl

Alternatively, users can post custom events directly with the Sysdig Monitor API using a curl request.

#!/bin/bash
SDC_ACCESS_TOKEN='626abc7-YOUR-TOKEN-HERE-3a3ghj432'
ENDPOINT='app.sysdigcloud.com'

curl -X POST -s https://$ENDPOINT/api/v2/events \
-H 'Content-Type: application/json; charset=UTF-8' \
-H 'Accept: application/json, text/javascript, */*; q=0.01' -H "Authorization: Bearer ${SDC_ACCESS_TOKEN}" \
--data-binary '{
  "event": {
    "name": "Jenkins - start wordpress deploy",
    "description": "deploy",
    "scope": "host.hostName = \"ip-10-1-1-1\" and build = \"89\"",
    "scopeLabels": {},
    "severity": "MEDIUM",
    "source": "jenkins",
    "tags": {
      "source": "jenkins"
    }
  }
}'

See also Enable/Disable Event Data.

Send a Custom Event when running Terraform Apply

Use custom events to track the creation of resources in Terraform. The creation of this elastic IP address will post a custom event to Sysdig Monitor that includes the dynamic public IP address.

resource "aws_eip" "my_elastic_ip_address" {
  vpc      = true

  provisioner "local-exec" {
    command = <<EOF
curl -X POST -s https://$ENDPOINT/api/v2/events \
-H 'Content-Type: application/json; charset=UTF-8' \
-H 'Accept: application/json, text/javascript, */*; q=0.01' -H "Authorization: Bearer $SDC_ACCESS_TOKEN" \
--data-binary '{
  "event": {
    "name": "elastic IP address provisioned with Terraform",
    "description": "elastic_ip:${self.public_ip} provisioned",
    "scope": "aws_account = \"1234-5678-9012\" and region = \"us-east-1\"",
    "scopeLabels": {},
    "severity": "MEDIUM",
    "source": "terraform",
    "tags": {
      "source": "terraform"
    }
  }
}'
EOF
    environment = {
      ENDPOINT = "app.sysdigcloud.com"
      SDC_ACCESS_TOKEN= "626abc7-YOUR-TOKEN-HERE-3a3ghj432"
     }
}
}

Embed Custom Events into SystemD to monitor when a service restarts

SystemD unit files can post a custom event to Sysdig Monitor whenever a service restarts.

[Unit]
Description=Program Foo
After=network.target

[Service]
ExecStart=/path/to/programfoo
Restart=always
RestartSec=3
ExecStartPost=/bin/bash -c 'curl -X POST -s https://app.sysdigcloud.com/api/v2/events \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer 626abc7-YOUR-TOKEN-HERE-3a3ghj432" \
    --data-binary '{
    "event": {
      "name": "Program Foo was Restarted"
    }
    }'
    '
[Install]
WantedBy=multi-user.target