This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

  • 1:
    • 2:

      AWS: Integrate AWS Account and CloudWatch Metrics (Optional)

      When the Sysdig agent is installed in an AWS environment, the Sysdig Platform can collect both general metadata and various types of CloudWatch metrics.

      There are three ways to integrate an AWS account into Sysdig:

      • By manually entering an AWS access key and secret key, and manually managing/rotating them as needed

      • By passing a parameter that allows Sysdig to autodetect an AWS ECS role and its permissions, passing an “implicit key” (On-Prem only).

        The implicit option requires no manual key rotation as AWS handles those permissions behind the scenes.

      • Using AWS Role delegation. Role delegation is an alternative to the existing integration methods using the access keys. This method is considered secure as sharing developer access keys with third-parties is not recommended by Amazon.

      The Sysdig Monitor UI includes links to help easily integrate CloudWatch metrics into Sysdig Monitor, as described below.

      After integrating with an AWS account, data will become visible in the Sysdig UI after a 10-15 minute delay.

      Entry Point in the Sysdig UI

      The Sysdig interface prompts you to perform this integration from the administrator’s Settings menu.

      Access from the Settings Menu

      Once an agent has been installed, log in to Sysdig Monitor or Sysdig Secure as administrator to perform integration steps or review/modify existing AWS settings.

      1. Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.

      2. Choose AWS Accounts.

        A page showing manual key integration, with access key and secret key fields displayed.

        NOTE: If there is no AWS integration yet then click on ADD and provide the access key and secret key.

      Integrate AWS Account Manually

      Have your AWS EC2 account details available. Integration begins on the AWS side and is completed in the Sysdig Monitor UI.

      In AWS

      Create an IAM Policy for Sysdig Access

      You could use the existing IAMReadOnly policy instead, but creating a Sysdig-specific policy provides more granular access control, the activity can be easily distinguished in CloudTrail, and it is considered best practice.

      1. In AWS, select IAM and create a policy to be used for Sysdig. (Sample policy name: SysdigMonitorPolicy.)

      2. Using the JSON editor view, copy/paste the Sysdig-specific policy code into the new policy and save it.

      3. You can review the policy in the Visual Editor.

      When reviewing the completed policy in the Visual editor, you should see something like:

      Create an IAM User and Grant Programmatic Access

      Use an existing IAM user, or (best practice) create a specific IAM user for the Sysdig Backend to programmatically access CloudWatch and use its data.

      1. In the IAM Console, add a User.

      2. Select AWS Access Type: Programmatic Access.

      3. Select ‘Attach existing policies directly’, search for and then select the newly created policy (Sample policy name: SysdigMonitorPolicy.)

      4. Select ‘Create User’ option.

      5. Copy and save the resulting access key and secret key (Note: the Secret is only displayed once, so make sure to download the credentials file or store the key securely that you can reference again.)

      In the Sysdig Monitor UI

      Enter the Access and Secret Key

      1. Log in to Sysdig Monitor or Sysdig Secure as the administrator and select Settings.

      2. Select AWS.

      3. Add an account by entering the **User Access Key**and **Secret Key**and clicking Save.

        The Credentials will be listed with a Status of **OK**checked.

      Should an Error occur, double-check the credentials entered. Mis-typing is the most common cause of errors.

      Enable CloudWatch Integration

      1. Navigate to the AWS page in the Sysdig Monitor UI, if you are not already there.

      2. Toggle the **CloudWatch Integration Status**to Enabled.

        Sysdig Monitor will poll the CloudWatch API every five minutes. Note that this incurs additional charges from AWS.

      After integrating with an AWS account, data will become visible in the Sysdig UI after a 10-15 minute delay.

      Refetch Credentials

      If the integrated AWS account changes on the AWS side, an Error will be listed in the Credentials Status on the Settings > AWS page.

      Use the Refetch Now button to re-establish the integration.

      Integrate AWS Account Using the Implicit Key (On-Prem Only)

      If Sysdig is installed in an EC2 instance, you can take advantage of the existing EC2 IAM role of that instance. This can simplify administration, as you do not have to manually rotate public and private keys provided to the Sysdig backend.

      Use Implicit Key

      Prerequisites

      Have your on-premises Sysdig platform installed in an AWS EC2 instance that has a proper IAM role.

      For this option, you cannot use the AWS Integration step in the Welcome Wizard.

      To enable implicit key, you must set the following parameter:

      -Ddraios.providers.aws.implicitProvider=true
      

      Use the parameter either during initial installation, or, if you already entered keys manually, to switch to an implicit key.

      If switching, you must then restart the api, worker, and collector components in the backend.

      In the Settings > AWS page, the former credentials will be overwritten it will show implicit key.

      Enablement steps depend on whether you are using Kubernetes or Replicated as your orchestrator.

      Kubernetes

      1. Edit the config.yaml to add to the following entries (in the Data section of config.yaml):

        sysdigcloud.jvm.api.options:
        sysdigcloud.jvm.worker.options:
        sysdigcloud.jvm.collector.options:
        

      2. If you are switching from manual to implicit keys, you must also restart the API, worker, and collector components.

        See To Make Configuration Changes for details.

      3. Enable Cloudwatch integration in the Sysdig UI.

      Changing the AWS Services that are Polled

      Sysdig is designed to collect metadata for particular AWS services, which are reflected in the IAM policy code.

      The services are:

      • DynamoDB

      • EC2 hosts

      • ECS

      • Elasticache

      • RDS

      • SQS

      When you implement the code and integration steps as described above, it will trigger two types of collection: first the metadata for each service is collected, and then Sysdig will poll for the metrics about the metadata returned. So, if the service is not enabled in your environment, no metadata (and no metrics) are collected about it. If it is enabled, but you do not want to poll metrics, then delete the lines of code related to that service from the IAM policy. This will avoid potential unwanted AWS API requests and potential AWS charges.

      See also AWSin the Metrics Dictionary.

      Security Groups

      If you have an on-premises Sysdig Backend, and have restricted outbound security groups, you may need to allow HTTPS & DNS access in order for the Sysdig Backend components to make connection to the Amazon APIs. As Amazon API endpoints are referenced by name and have a large number of IP’s, this may need to be full 0.0.0.0/0 outbound access for HTTPS & DNS.

      If you need to filter just to Amazon IP ranges, you can use the following as a guide: https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html

      Retrieving CloudWatch Data for Particular AWS Regions

      To enable metrics collection from only certain AWS regions in your environment, it is necessary to open a ticket with Sysdig Support. See Contact Support for details.

      For information on the resulting AWS services visible in Sysdig Monitor, see the AWS-related information in the Metrics Dictionary (also available from within the Sysdig Monitor UI).

      For information on how licensing affects AWS service views, see About AWS Cloudwatch Licensing.

      1 -

      IAM Policy Code to Use

      Best Practice: Create a Sysdig-specific IAM policy to be used for granting programmatic access to Sysdig. Copy/paste the code snippet below into this policy. It enables Sysdig to collect metadata and CloudWatch metrics from the following services, as applicable to your environment:

      • Dynamodb

      • EC2 hosts

      • ECS

      • Elasticache

      • RDS

      • SQS

      If you want to use your own AWS S3 bucket to store Sysdig capture files, you can append those code snippets to this IAM Policy as well. See Storage: Configure AWS Capture File Storage (Optional) for details.

      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Action": [
                      "autoscaling:Describe*",
                      "cloudwatch:Describe*",
                      "cloudwatch:Get*",
                      "cloudwatch:List*",
                      "dynamodb:ListTables",
                                      "dynamodb:Describe*",
                      "ec2:Describe*",
                      "ecs:Describe*",
                      "ecs:List*",
                      "elasticache:DescribeCacheClusters",
                      "elasticache:ListTagsForResource",
                      "elasticloadbalancing:Describe*",
                      "rds:Describe*",
                      "rds:ListTagsForResource",
                      "sqs:ListQueues",
                      "sqs:GetQueueAttributes",
                      "sqs:ReceiveMessage"
                  ],
                  "Effect": "Allow",
                  "Resource": "*"
              }
          ]
      }
      

      See Changing the AWS Services that are Polled for more detail.

      2 -

      Integrate with AWS Role Delegation

      This section describes how to configure Sysdig Monitor to utilize the Amazon Web Service (AWS) AssumeRole functionality and authorize Sysdig Monitor to discover cloud assets, grab CloudWatch metrics from your AWS account, and utilize custom S3 bucket for storing captures. Upon integrating with an AWS role, you can delegate access to AWS resources that are not associated with your Sysdig AWS account.

      Setting up cross-account access through roles eliminates the need to create individual IAM users in each account. In addition, users don’t have to sign out of one account and sign in to another in order to access resources in different AWS accounts.

      Role delegation is an alternative to the existing integration method using the access keys. This method is considered secure as sharing developer access keys with third-parties is not recommended by Amazon.

      Prerequisites and Guidelines

      This topic assumes that you have the following ready and you are familiar with AWS.

      • Sysdig Monitor API Token

      • External ID

      • API endpoint. In this topic, it is referred to as {{host}}

      • Administrator privileges to configure AWS integration

      • API client. Examples in this topic use curl

      • AWS account ID

        • SaaS: The default AWS account ID is 273107874544 (US East region). For other regions, check AWS account IDs .

        • On-Prem: Customer-specific.

      Enable AWS Role Delegation with API

      This section describes how to enable AWS role delegation using an API.

      Instructions for SaaS

      1. Get Your External ID.

      2. Configure Role Delegation.

      3. Get Role ARN.

      4. Add the AWS Account.

      Instructions for On-Prem

      1. Get Your External ID.

      2. Configure Role Delegation.

      3. Get Role ARN.

      4. Add the AWS Account.

      5. Follow Additional Configuration for On-Prem.

      Get Your External ID

      Retrieve your external ID as follows:

      curl -k --request GET \
      
      --url host/api/users/me \
      
      --header 'authorization: Bearer e71d7c0f-501e-47d4-a159-39da8b716f44' | jq '.[] | .customer | .externalId'
      

      An example of External ID from the response will be 04acdd59-4c98-4d11-8ee5-424326248161.

      Configure Role Delegation

      Integrating the Sysdig Platform with Amazon Web Services requires configuring role delegation using AWS IAM.

      1. Create a new role in the AWS IAM Console:

        1. For the role type, select Another AWS account.

        2. (SaaS) Enter the Sysdig account ID for Account ID.

          This means that you are granting read-only access to your AWS data.

        3. Select Require external ID and enter the one you retrieved in the previous step. Leave MFA disabled.

      2. Click Next: Permissions.

      3. Create the following policies:

        • sysdig_cloudwatch: Gives access to the list and describe supported AWS resources and get CloudWatch metrics for them.

        • sysdig_s3: Defines the bucket name where we wish to store the captures

          For more information on policies, see IAM Policy Code to Use.

        For detailed instructions on how to create a policy, see Integrate AWS Account Manually.

        1. If a policy has already been created, search for it on this page and select it, then skip to step. Otherwise, click Create Policy, which opens in a new window.

        2. Click Review policy.

        3. Name the policy and provide an apt description. For example, sysdig_cloudwatch.

        4. Click Create Policy.

          You can now close this window.

      4. In the Create role window, refresh the list of policies and select the policies you just created.

      5. Click Next: Review.

      6. Give the role a name and an apt description. For example, sysdig_role.

      7. Click Create Role.

      Get Role ARN

      1. Select Roles > sysdig-role.

      2. Copy Role ARN.

      Add the AWS Account

      Using the role that you have created, add an AWS account on the Sysdig Monitor side. Use the following API call:

      curl --request POST \
        --url {{host}}/api/providers \
        --header 'authorization: Bearer e71d7c0f-501e-47d4-a159-39da8b716f44' \
        --header 'content-type: application/json' \
        --data '{"name": "aws","credentials": {"role": "<Role_ARN>"},"alias": "role_delegation"}'
      

      Replace <Role_ARN> with the one that you have copied in the previous section.

      The response lists all the providers. An example response is given below:

      {
      
        "provider": {
      
          "id": 7,
          "name": "aws",
          "credentials": {
      
            "id": "role_delegation",
            "role": "arn:aws:iam::485365068658:role/sysdig-access3"
          },
          "tags": [],
          "status": {
      
            "status": "configured",
            "lastUpdate": null,
            "percentage": 0,
            "lastProviderMessages": []
          },
          "alias": "role_delegation"
        }
      }
      

      Verify the role delegation has been created.

      1. Log in to Sysdig Monitor or Sysdig Secure as administrator.

      2. Select Settings > AWS.

        The role that you have been created will be added to the list of AWS Accounts.

      3. Proceed to enable CloudWatch and AWS S3 bucket.

        See AWS: Integrate AWS Account and CloudWatch Metrics (Optional) for more information.

      Additional Configuration for On-Prem

      1. Create an AWS user that will be used to fetch temporary credentials.

      2. Assign a policy to the user to allow AssumeRole. For example:

        {
          "Version": "2012-10-17",
          "Statement": {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::{ACCOUNT-ID}:role/{ROLE_NAME}*"
          }
        }
        
      3. Make the access keys available to users from one of the sources:

        • Environment variables

        • Java system properties

        • Instance profile credentials delivered through the Amazon EC2 metadata service.

          EC2 metadata service is recommended if the installation is on AWS.

      Example: Set Environment Variables on a Kubernetes Installation

      1. Create Secret:

        apiVersion: v1
        kind: Secret
        metadata:
          name: aws-credentials
        type: Opaque
        data:
          aws.accessKey: {{BASE64_ENCODED_ACCESS_KEY_ID}}
          aws.secretKey: {{BASE64_ENCODED_ACCESS_KEY_SECRET}}
        
      2. Expose variables in deployment descriptors (sysdigcloud-collector, sysdigcloud-worker, sysdigcloud-api) and reference values in the newly created secret:

        - name: AWS_ACCESS_KEY_ID
            valueFrom:
            secretKeyRef:
                key: aws.accessKey
                name: aws-credentials
        - name: AWS_SECRET_ACCESS_KEY
            valueFrom:
            secretKeyRef:
                key: aws.secretKey
                name: aws-credentials
        

        Add variables to descriptors on each platform update until new variables are part of the installer.

      Set Up Resource Discovery

      The supported AWS are EC2, RDS, Elastic Load Balancer (ELB), ElastiCache, SQS, DynamoDB, and Application Load Balancer (ALB).

      By default, all the resources are fetched for all regions supported by AWS. You can avoid this by whitelisting regions when creating a provider key via the API. Example body of the provider key request when whitelisting regions:

      {
          "name": "aws",
          "credentials": {
              "role": "arn:aws:iam::676966947806:role/test-assume-role"
          },
          "additionalOptions": "{\"regions\":[\"US_EAST_1\",\"US_EAST_2\"]}"
      }
      

      Enable AWS Role Delegation with UI

      Use the AWS option in the Settings menu to configure AWS role delegation.

      1. Log in to the Sysdig Monitor as an administrator and select Settings.

      2. Click AWS.

        The AWS Account page is displayed.

      3. Click Add Accounts.

        The Identity Authentication page opens to the Role Delegation tab.

      4. Specify the following:

        • Role ARN: The Role ARN associated with the role you have created for role delegation. The ID is available on the summary page of the role on the AWS console. For more information, see Integrate with AWS Role Delegation.

        • AWS External ID: Ensure that AWS External ID is displayed on the page.

      5. Click Save.