Sysdig Documentation

AWS: Integrate AWS Account and CloudWatch Metrics (Optional)

When a Sysdig agent is installed in an AWS environment, the Sysdig backend (either SaaS or On-Premises) can collect both general metadata and various types of CloudWatch metrics.

There are two ways to integrate an AWS account into Sysdig:

  • By manually entering an AWS access key and secret key, and manually managing/rotating them as needed (SaaS or On-Prem), or

  • By passing a parameter that allows Sysdig to autodetect an AWS ECS role and its permissions, passing an "implicit key" (On-Prem only).

    The implicit option requires no manual key rotation as AWS handles those permissions behind the scenes.

The Sysdig Monitor UI includes links to help easily integrate CloudWatch metrics into Sysdig Monitor, as described below.

Two Entry Points in the Sysdig UI

The Sysdig interface prompts you to perform this integration from two different places: the Welcome Wizard or the administrator's Settings menu.

Access from the Welcome Wizard

When first logging in to the Sysdig Monitor UI, the Welcome Wizard includes an option to integrate your AWS account to collect CloudWatch metrics.

Note

If you want to use an implicit key, you should skip this page in the Wizard. See Implicit Key Option for details.

214728711

Access from the Settings Menu

Once an agent has been installed, log in to Sysdig Monitor or Sysdig Secure as administrator to perform integration steps or review/modify existing AWS settings.

  1. Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.

    373575724.png

  2. Choose AWS .

    215810099.jpg

    An AWS page showing manual key integration, with access key and secret key fields displayed.

    NOTE: If there is no AWS integration yet then click on ADD YOUR AWS ACCOUNT and provide the access key and secret key"

Integration Steps (Manual Option)

Have your AWS EC2 account details available. Integration begins on the AWS side and is completed in the Sysdig Monitor UI.

In AWS

Create an IAM Policy for Sysdig Access

Note

You could use the existing IAMReadOnly policy instead, but creating a Sysdig-specific policy provides more granular access control, activity can be easily distinguished in CloudTrail and it is considered best practice.

  1. In AWS, select IAM and create a policy to be used for Sysdig. (Sample policy name: SysdigMonitorPolicy.)

  2. Using the JSON editor view, copy/paste the Sysdig-specific policy code into the new policy and save it.

  3. You can review the policy in the Visual Editor.

When reviewing the completed policy in the Visual editor, you should see something like:

215810088.jpg

Create an IAM User and Grant Programmatic Access

Use an existing IAM user, or (best practice) create a specific IAM user for the Sysdig Backend to programmatically access CloudWatch and use its data.

  1. In the IAM Console, add a User.

  2. Select AWS Access Type: Programmatic Access.

  3. Select 'Attach existing policies directly', search for and then select the newly created policy (Sample policy name: SysdigMonitorPolicy.)

  4. Select 'Create User' option.

  5. Copy and save the resulting access key and secret key (Note: the Secret is only displayed once, so make sure to download the credentials file or store the key securely that you can reference again.)

In the Sysdig Monitor UI

Enter Access Key and Secret Key

  1. Log in to Sysdig Monitor or Sysdig Secure as administrator and select Settings.

  2. Select AWS.

    215810099.jpg
  3. Add an account by entering the User Access Key and Secret Key and clicking Save.

    The Credentials will be listed with a Status of OK checked.

Note

Should an Error occur instead, double-check the credentials entered. Mis-typing is the most common cause of error.

Enable CloudWatch Integration

  1. Navigate to the AWS page in the Sysdig Monitor UI, if you are not already there.

  2. Toggle the CloudWatch Integration Status to Enabled.

    Sysdig Monitor will poll the CloudWatch API every five minutes. Note that this incurs additional charges from AWS.

Refetch Credentials

If the integrated AWS account changes on the AWS side, an Error will be listed in the Credentials Status on the Settings > AWS page.

Use the Refetch Now button to re-establish the integration.

Integration Steps (Implicit Key Option, On-Prem Only)

If Sysdig is installed in an EC2 instance, you can take advantage of the existing EC2 IAM role of that instance. This can simplify administration, as you do not have to manually rotate public and private keys provided to the Sysdig backend.

Use Implicit Key

Prerequisites

Have your on-premises Sysdig platform installed in an AWS EC2 instance that has a proper IAM role.

Note

For this option, you cannot use the AWS Integration step in the Welcome Wizard.

To enable implicit key, you must set the following parameter:

-Ddraios.providers.aws.implicitProvider=true

Note

Use the parameter either during initial installation, or, if you already entered keys manually, to switch to an implicit key.

If switching, you must then restart the api, worker, and collector components in the backend.

In the Settings > AWS page, the former credentials will be overwritten it will show implicit key.

Enablement steps depend on whether you are using Kubernetes or Replicated as your orchestrator.

Kubernetes

  1. Edit the config.yaml to add to the following entries (in the Data section of config.yaml):

    sysdigcloud.jvm.api.options:
    sysdigcloud.jvm.worker.options:
    sysdigcloud.jvm.collector.options: 
    373575718.png
  2. If you are switching from manual to implicit keys, you must also restart the api, worker, and collector components.

    See Making Configuration Changes for details.

  3. Enable Cloudwatch integration in the Sysdig UI.

Replicated

  1. In the Replicated Management Console, enter -Ddraios.providers.aws.implicitProvider=true in Sysdig Cloud application JVM options.

    If you have other settings in that field, separate the entries with a space.

    373575712.png
    See also Replicated Advanced Settings .

  2. If you are switching from manual keys, you must restart the backend components from the Replicated Management Console.

  3. Enable CloudWatch Integration in the Sysdig Monitor UI.

Changing the AWS Services that are Polled

Sysdig is designed to collect metadata for particular AWS services, which are reflected in the IAM policy code.

The services are:

  • DynamoDB

  • EC2 hosts

  • ECS

  • Elasticache

  • RDS

  • SQS

When you implement the code and integration steps as described above, it will trigger two types of collection: first the metadata for each service is collected, and then Sysdig will poll for the metrics about the metadata returned. So, if the service is not enabled in your environment, no metadata (and no metrics) are collected about it. If it is enabled, but you do not want to poll metrics, then delete the lines of code related to that service from the IAM policy. This will avoid potential unwanted AWS API requests and potential AWS charges.

See also AWSin the Metrics Dictionary.

Security Groups

If you have an on-premises Sysdig Backend, and have restricted outbound security groups, you may need to allow HTTPS & DNS access in order for the Sysdig Backend components to make connection to the Amazon APIs. As Amazon API endpoints are referenced by name and have a large number of IP's, this may need to be full 0.0.0.0/0 outbound access for HTTPS & DNS.

If you need to filter just to Amazon IP ranges, you can use the following as a guide: https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html

Retrieving CloudWatch Data for Particular AWS Regions

To enable metrics collection from only certain AWS regions in your environment, it is necessary to open a ticket with Sysdig Support. See Contact Support for details.

Related Information

For information on the resulting AWS services visible in Sysdig Monitor, see the AWS-related information in the Metrics Dictionary (also available from within the Sysdig Monitor UI).

For information on how licensing affects AWS service views, see AWS Services Licensing.