This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

    Configure Default Integrations

    Each Monitoring Integration holds a specific job that scrapes its metrics and sends them to Sysdig Monitor. To optimize metrics scraping for building dashboards and alerts in Sysdig Monitor, Sysdig offers default jobs for these integrations. Periodically, the Sysdig agent connects with Sysdig Monitor and retrieves the default jobs and make the Monitoring Integrations available for use. See the list of the available integrations and corresponding jobs.

    You can find all the jobs in the /opt/draios/etc/promscrape.yaml file in the sysdig-agent container in your cluster.

    Supported Monitoring Integrations

    IntegrationOut of the BoxEnabled by defaultJob name in config file
    Apacheapache-exporter-default, apache-grok-default
    Cephceph-default
    ElasticSearchelasticsearch-default
    Harborharbor-exporter-default, harbor-core-default, harbor-registry-default, harbor-jobservice-default
    Kubernetes Control Planekube-dns-default, kube-controller-manager-default, kube-scheduler-default
    Kubernetes Etcdetcd-default
    Kubeletk8s-kubelet-default
    Kube-Proxykubernetes-kube-proxy-default
    Kubernetes Persistent Volume Claimk8s-pvc-default
    Kedakeda-default
    Memcachedmemcached-default
    MongoDBmongodb-default
    MySQLmysql-default
    Nginxnginx-default
    Nginx Ingressnginx-ingress-default
    Open Policy Agent - Gatekeeperopa-default
    PostgreSQLpostgres-default
    Prometheus Default Jobk8s-pods
    RabbitMQrabbitmq-default
    Redisredis-default
    Sysdig Admission Controllersysdig-admission-controller-default

    Enable and Disable Integrations

    Some integrations are disabled by default, due to the potential high cardinality of their metrics. To enable them, contact our support department. The same applies to disabling integrations by default in all your clusters.

    Customize a Default Job

    The default jobs offered by Sysdig for integrations are optimized to scrape the metrics for building dashboards and alerts in Sysdig Monitor. Instead of processing all the metrics available, you can determine which metrics to include or exclude for your requirements. To do so, you can overwrite the default configuration in the prometheus.yaml file. The prometheus.yaml file is located in the sysdig-agent ConfigMap in the sysdig-agent namespace.

    You can overwrite the default job for a specific integration by adding a new job to the prometheus.yaml file with the same name as the default job that you want to replace. For example, if you want to create a new job for the Apache integration, create a new job with the name apache-default. The jobs defined by the user has precedence over the default ones.

    See Supported Monitoring Integrations for thee complete list of integrations and corresponding job names.

    Use Sysdig Annotations in Exporters

    Sysdig provides a set of Helm charts that helps you configure the exporters for the integrations. For more information on installing Monitor Integrations, see the Monitoring Integrations option in the Sysdig Monitor. Additionally, the Helm charts are publicly available in the Sysdig Helm repository.

    If exporters are already installed in your cluster, you can use the standard Prometheus annotations and the Sysdig agent will automatically scrape them.

    For example, if you use the annotation given below, the incoming metrics will have the information about the pod that generates the metrics.

    spec:
      template:
        metadata:
          annotations:
            prometheus.io/path: /metrics
            prometheus.io/port: '9100'
            prometheus.io/scrape: 'true'
    

    If you use an exporter, the incoming metrics will be associated with the exporter pod, not the application pod. To change this behavior, you can use the Sysdig-provided annotations and configure the exporter with special settings on the agent.

    Annotate the Exporter

    Use the following annotations to configure the exporter:

    spec:
      template:
        metadata:
          annotations:
            promcat.sysdig.com/port: '9187'
            promcat.sysdig.com/target_ns: my-namespace
            promcat.sysdig.com/target_workload_type: deployment
            promcat.sysdig.com/target_workload_name: my-workload
            promcat.sysdig.com/integration_type: my-integration
    
    • port: The port to scrape for metrics on the exporter.
    • target_ns: The namespace of the workload corresponding to the application (not the exporter).
    • target_workload_type: The type of the workload of the application (not the exporter). The possible values are deployment, statefulset, and daemonset.
    • target_workload_name: The name of the workload corresponding to the application (not the exporter).
    • integration_type: The type of the integration. The job created in the Sysdig agent use this value to find the exporter.

    Configure a New Job

    Edit the prometheus.yaml file to configure a new job in Sysdig agent. The file is located in the sysdig-agent ConfigMap in the sysdig-agentnamespace.

    You can use this example template:

    - job_name: my-integration
      tls_config:
        insecure_skip_verify: true
      kubernetes_sd_configs:
        - role: pod
      relabel_configs:
        - action: keep
          source_labels: [__meta_kubernetes_pod_host_ip]
          regex: __HOSTIPS__
        - action: drop
          source_labels: [__meta_kubernetes_pod_annotation_promcat_sysdig_com_omit]
          regex: true
        - action: keep
          source_labels:
            - __meta_kubernetes_pod_annotation_promcat_sysdig_com_integration_type
          regex: 'my-integration' # Use here the integration type that you defined in your annotations
        - action: replace
          source_labels: [__meta_kubernetes_pod_annotation_promcat_sysdig_com_target_ns]
          target_label: kube_namespace_name
        - action: replace
          source_labels: [__meta_kubernetes_pod_annotation_promcat_sysdig_com_target_workload_type]
          target_label: kube_workload_type
        - action: replace
          source_labels: [__meta_kubernetes_pod_annotation_promcat_sysdig_com_target_workload_name]
          target_label: kube_workload_name
        - action: replace
          replacement: true
          target_label: sysdig_omit_source
        - action: replace
          source_labels: [__address__, __meta_kubernetes_pod_annotation_promcat_sysdig_com_port]
          regex: ([^:]+)(?::\d+)?;(\d+)
          replacement: $1:$2
          target_label: __address__
        - action: replace
          source_labels: [__meta_kubernetes_pod_uid]
          target_label: sysdig_k8s_pod_uid
        - action: replace
          source_labels: [__meta_kubernetes_pod_container_name]
          target_label: sysdig_k8s_pod_container_name
    

    Exclude a Deployment from Being Scraped

    If you want the Agent to exclude a deployment from being scraped, you can use the following annotation:

    spec:
      template:
        metadata:
          annotations:
            promcat.sysdig.com/omit: 'true'