(Preview) Integrate AWS Lambda Telemetry API

Sysdig provides Sysdig Monitor Lambda Extension (Preview) for AWS Lambda Telemetry API to help you consume Lambda metrics, providing great visibility into serverless environments. The extension can subscribe to the Telemetry API, process the telemetry event streams in realtime, and send the metrics to Sysdig Monitor. You can simply add the Sysdig Lambda Extension as a layer to your individual Lambda functions and receive telemetry metric streams for each Lambda function.

This feature is an extension to the existing AWS Lambda monitoring capabilities via the AWS CloudWatch Metric Steams.

List of Metrics

Sysdig Lambda Extension collects the following metrics. The metrics are measured in milliseconds.

MetricsDescription
aws_lambda_invocationsThe number of times the function code is invoked. This count includes both successful invocations and invocations that resulted in a function error.
aws_lambda_durationThe amount of time that the function code spends processing an event.
aws_lambda_errorThe number of invocations that result in a function error. These errors include the exceptions that the code throws and the exceptions that the Lambda runtime throws.
aws_lambda_postruntime_extensions_durationThe cumulative amount of time that the runtime spends running code for extensions after the function code has completed.

List of Labels

Sysdig enriches the Lambda metrics by the following labels:

LabelDescription
cloud_provider_account_idThe unique ID associated with your AWS account.
cloud_provider_nameThe source of the metrics. In this case, AWS.
cloud_provider_region_nameThe region associated with your AWS account.
extention_idThe unique identifier of the Sysdig Lambda Extension.
function_nameThe name of the Lambda function you are monitoring.
ingest_sourceThe method by which you are collecting the metrics. In this case, the value is lambda exporter.

Configure the Lambda Function

You need to add the Sysdig Lambda Extension as a layer to your Lambda function, set the Sysdig-specific environment variables, and then run the function.

Publish the Extension

  1. Download Sysdig Monitor Lambda Extension.
  2. Log in to your AWS account.
  3. Publish the extension. You can use either the AWS UI or CLI.

AWS Console

  1. Navigate to Lambda > Layers.

  2. On the Layers page, click Create Layer.

  3. Under Layer configuration, specify the following:

    • Name: Specify a unique name for your Sysdig Lambda Extension.
    • Description: Optionally, give a description that can help you identify the extension.
  4. Select Upload a .zip file.

  5. Upload the Sysdig Lambda Extension zip file.

  6. Optionally, you can enter other configuration information as described in Creating Layers

  7. Click Create.

AWS CLI

  1. Run the following command:

    $ aws lambda publish-layer-version --layer-name "sysdig-monitor-lambda-extension-v1" --region <your-region> --zip-file  "fileb://<path-to-sysdig-monitor-lambda-extension.zip>"
    

    Replace the following:

    • <your-region> with the Amazon region where you are running your AWS Lambda function

    • <path-to-sysdig-monitor-lambda-extension-v1.zip> with the path to the sysdig-monitor-lambda-extension-v1.zip file.

    You should see output similar to the following:

       {
           "Content": {
               "Location": "https://awslambda-us-east-2-layers.s3.us-east-2.amazonaws.com/snapshots/059797578166/sysdig-lambda-extension-....",
               "CodeSha256": "gLJlfhvhm28Xp+21aFf+sthrio8XzjPWHwB+mSbUGs4+",
               "CodeSize": 4202227
           },
           "LayerArn": "arn:aws:lambda:us-east-2:059797578166:layer:sysdig-monitor-lambda-extension-v1",
           "LayerVersionArn": "arn:aws:lambda:us-east-2:059797578166:layer:sysdig-monitor-lambda-extension-v1:3",
           "Description": "",
           "CreatedDate": "2022-10-31T19:12:33.965+0000",
           "Version": 4
       }
    
  2. Copy the ARN value. You need to specify the ARN while adding the extension as a layer to your Lambda function.

    In this example, arn:aws:lambda:us-east-2:059797578166:layer:sysdig-lambda-extension

Add the Layer

This section assumes that you have already created the Lambda function that you want to monitor.

  1. Log in to your AWS account.
  2. On the Lambda function page, select the function you want to monitor.
  3. On the Function overview page, click Add a layer.
  4. Click Specify an ARN and paste the ARN you have copied earlier.
  5. Optionally, verify the specified ARN is correct.
  6. Click Add.

Add Environment Variables

  1. From the Lambda function page, select your Lambda function.

  2. On the function page, click Configuration.

  3. Select Environment variables and specify the following:

    • SYSDIG_API_TOKEN: The Sysdig Monitor API associated with your Sysdig account.
    • SYSDIG_API_TOKEN_ENCRYPTED: If you want the Sysdig API token to be encrypted in transit, set this option to TRUE. See Encrypt Sysdig API Token for more information.
    • SYSDIG_SITE The Collector URL associated with your Sysdig region.
  4. Click Save.

Encrypt Sysdig API Token

If you want the Sysdig API Token to be encrypted prior to sending them to your Lambda function, you can do so by using the Encryption configuration option. Encrypted Sysdig API token will be obscured in the Lambda console and API output, even for the users who have the permission to use the key. In your code, the encrypted value will be retrieved from the environment and will be decrypted by using the AWS KMS API.

  1. On the Functions page of the Lambda console, click your function.

  2. Choose Configuration, then select Environment variables from the left navigation bar.

  3. In the Environment variables section, click Edit.

  4. Expand Encryption configuration.

  5. Under Encryption in transit, select Enable helpers for encryption in transit.

  6. Select Encrypt next to the SYSDIG_API_TOKEN environment variable.

  7. Under AWS KMS key to encrypt in transit, select a customer managed key that you have created.

  8. Copy the Execution role policy in JSON format.

    You need the JSON snippet while setting the permissions.

  9. Select Save.

  10. Set up permissions. Because you are enabling the client-side encryption for securing the Sysdig API token in transit, your function needs permission to call the kms:Decrypt API operation.

    1. From your function page, select Configuration, and then click Permissions.
    2. Click the Role name.
    3. On the role’s page, click Add permissions > Create inline policy.
    4. Click JSON and paste the JSON snippet you have copied earlier.
    5. Click Review Policy

For more information, see Securing environment variables.

Run the Function

On the function page, run your function a few times by clicking Test. When you run it for the first time, you will be asked to specify a name.

Verify the Connection

  1. Log in to Sysdig Monitor.

  2. Open Explore > Metrics Explorer.

  3. Select Entire Infrastructure on the scope tree.

  4. Search for one of the metrics listed above. For example, aws_lambda_invocations.

    If you can view the list of lambda metrics, the connection is live.
  5. You can continue with building dashboards, creating alerts, and exploring with Advisor.

Upgrade Sysdig Monitor Lambda Extension

To upgrade Sysdig Lambda Extension, you need to first download the latest one, create a new version of the layer that you have already created, then attach it to your function.

  1. Create a new version of the layer.

    1. Navigate to Lambda > Layers.
    2. On the Layers page, select your layer, then click Create Version.
    3. Under Layer version configuration, specify the following:
    • Name: Specify a unique name for your Sysdig Lambda Extension.
    • Description: Optionally, give a description that can help you identify the extension.
    1. Select Upload a .zip file.
    2. Upload the new Sysdig Lambda Extension zip file.
    3. Optionally, you can enter other configuration information as described in Creating Layers
    4. Click Create.
  2. Attach the new version to your function.

    1. On the Lambda function page, select your function.
    2. On the bottom of the page under Layer, Click Edit.
    3. On the Edit layers page, select the version you want to use.
    4. Click Save.

Compare AWS Metric Streams and AWS Telemetry APIs

Metric StreamsTelemetry APIs
Ingests metrics into Sysdig from a Firehose connection.Processes metrics from Lambda events that are generated in Lambda execution environments in realtime.
Collects larger volume of metrics every 1 minute.Collects specific Lambda metrics every 10 seconds.
Consuming all the Lambda metrics might incure higher cost.Collecting specific Lambda metrics keeps the cost lower.