AWS Fargate Serverless Agents

Introduction

Check the Overview for an explanation of when and why to use serverless agents in “container-as-a-service” cloud environments.

Architecture

The Sysdig serverless agent provides runtime detection through policy enforcement with Falco. At this time, the serverless agent is available for AWS Fargate on ECS. It is comprised of an orchestrator agent and (potentially multiple) workload agents.

The Sysdig serverless orchestrator agent is a collection point installed on each VPC to collect data from the serverless workload agent(s) and to forward them to the Sysdig backend. It also syncs the Falco runtime policies and rules to the workload agent(s) from the Sysdig backend.
The Sysdig serverless workload agent is installed in each task and requires network access to communicate with the orchestrator agent.
Note that the workload agent is designed to secure your workload. However, at deployment, the default setting prioritizes availability over security, using setting that allow your workload to start even if policies are not in place. If you prefer to prioritize security over availability, you can change these settings by configuring your workload starting policy.

Prerequisites

Before starting the installation, ensure that you have the following:

On AWS Side

A custom Terraform/CloudFormation template containing the Fargate task definitions that you want to instrument through the Sysdig Serverless Agent
Two VPC subnets in different availability zones that can connect with the internet via a NAT gateway or an internet gateway

On Sysdig Side

Sysdig Secure up and running
The endpoint of the Sysdig Collector for your region
From the Sysdig Secure UI, retrieve the following:
- Access Key to install the agent and push the data to the Sysdig platform
- Secure API Token to configure the Sysdig provider.

Known Limitations

Sysdig instruments a target workload by patching its task definition to run the original workload below Sysdig instrumentation. To patch the original task definition, Sysdig instrumentation pulls and analyzes the workload image to get the original entry point and the command, along with other information.

Pulling workload images from Public vs Private registries

If you retrieve your workload image from a private registry, you must explicitly define the entry point and the command in the container definition. If you don’t specify them, the Sysdig instrumentation might not be able to collect such information, and the instrumentation might fail.

Instead, if you pull the workload image from a public registry no additional operations are required.

Referencing a parameterized Image in CloudFormation Workload Template

When instrumenting a workload over CloudFormation you must define the Image inline in your TaskDefinition.

Using a parameterized image in the TaskDefinition instead might prevent the Sysdig instrumentation from retrieving the workload image configuration. That could lead to incorrect workload instrumentation.

The example below shows a valid TaskDefinition.

Resources:
  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      ContainerDefinitions:
        - Name: !Ref MyContainerName
          Image: "MyContainerImage"  # Inline Image
        ...

Install Options

The Sysdig serverless agent can be deployed automatically via Terraform or CloudFormation. Alternatively, you can also use the manual process to complete the instrumentation tasks.

Sysdig recommends using the Terraform deployment method to instrument your Fargate workloads.

Terraform

This option presumes you use Terraform to deploy your workload.

You can deploy the orchestrator agent and install the workload agent by using an automated process which will instrument all your task definitions.

For details, see the following Sysdig Terraform registries:

Deployment Steps

Install the Orchestrator Agent

Set up the AWS Terraform provider:

  provider "aws" {
    region = var.region
  }

Configure the Sysdig orchestrator module and deploy it:

module "fargate-orchestrator-agent" {
  source  = "sysdiglabs/fargate-orchestrator-agent/aws"
  version = "0.1.1"

  vpc_id           = var.vpc_id
  subnets          = [var.subnet_a_id, var.subnet_b_id]

  access_key       = var.access_key

  collector_host   = var.collector_host
  collector_port   = var.collector_port

  name             = "sysdig-orchestrator"
  agent_image      = "quay.io/sysdig/orchestrator-agent:latest"

  # True if the VPC uses an InternetGateway, false otherwise
  assign_public_ip = true

  tags = {
    description    = "Sysdig Serverless Agent Orchestrator"
  }
}

Call this module for each VPC that needs instrumentation.

Install the Instrumented Workload

Set up the Sysdig Terraform provider:

terraform {
  required_providers {
    sysdig = {
      source = "sysdiglabs/sysdig"
      version = ">= 0.5.39"
    }
  }
}

provider "sysdig" {
  sysdig_secure_api_token = var.secure_api_token
}

Pass the orchestrator host, port, and container definitions of your workload to the sysdig_fargate_workload_agent data source:

data "sysdig_fargate_workload_agent" "instrumented" {
  container_definitions = jsonencode([...])

  sysdig_access_key     = var.access_key

  workload_agent_image  = "quay.io/sysdig/workload-agent:latest"

  orchestrator_host     = module.sysdig_orchestrator_agent.orchestrator_host
  orchestrator_port     = module.sysdig_orchestrator_agent.orchestrator_port
}

Note: Ensure that the input container definitions must be in JSON format.

Include the instrumented JSON in your Fargate task definition and deploy your instrumented workload:
```
resource "aws_ecs_task_definition" "fargate_task" {
  ...

  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]

  container_definitions    = "${data.sysdig_fargate_workload_agent.instrumented.output_container_definitions}"
}
```
The Sysdig instrumentation will go over the original task definition to instrument it. The process includes replacing the original entry point and command of the containers.
For the images pulled from private registries, explicitly provide the Entrypoint and Command in the related container definition, otherwise, the instrumentation will not be completed.

(Latest) CloudFormation

This option presumes you use a CFT to deploy your workload.

As of Serverless Agent 3.0, a YAML provided by Sysdig helps you deploy the orchestrator agent and the instrumentation service in a single step. Then, you will install the workload agent using an automated process which will instrument all the Fargate task definitions in your CFT.

For the Orchestrator Agent and Instrumention Service, Sysdig provides the serverless-instumentation.yaml to use as a CloudFormation Template which you can deploy through the AWS console. You need one Orchestrator deployment per VPC in your environment that your organization wants to secure.
For the Workload Agents, you need one Workload Agent per Fargate task definition. For example, if you have ten services and ten task definitions, each needs to be instrumented.

Deployment Steps

Deploy the Sysdig Instrumentation and Orchestration Stack

Deploy the serverless-instrumentation.yaml for each desired VPC using CloudFormation:

Log in to the AWS Console, select CloudFormation, Create a stack with new resources, and specify the serverless-instrumentation.yaml as the Template source.
Specify the stack details to deploy the Orchestrator Agent on the same VPC where your service is running. For standard deployments, provide the parameters highlighted in the figure below.
Click Next, complete the stack creation, and wait for the deployment to complete (usually a few minutes).

Deploy the Workload Agent

Edit Your CFT

Once the Sysdig Instrumentation and Orchestration stack is deployed, the Outputs tab provides the transformation instruction as shown in the figure below. Note that the value here is dependent on what you set it to during the deployment of the Sysdig Instrumentation and Orchestration Stack.

Copy and paste the value of the transformation instruction to the root level of your applications CFT. For Example:

Transform: ["SysdigMacro"]

The Sysdig instrumentation will go over the original task definition to instrument it. The instrumentation process includes replacing the original entry point and command of the container. If you are using an image from a public registry, it can determine these values from the image. If you are using an image from a private registry then you must explicitly provide the Entrypoint and Command in the related container definition; otherwise, the instrumentation will not be completed.

Deploy Your CFT

All the new deployments of your CFT will be instrumented.

When instrumentation is complete, Fargate events should be visible in the Sysdig Secure Events feed.

Upgrade from a Prior Version

Up through version 2.3.0, the installation process deployed two stacks, as described in (Legacy) CloudFormation:

Orchestration stack, deployed via YAML
Instrumentation stack, deployed via command-line installer.

Instead, from version 3.0.0, you will deploy the “Instrumentation & Orchestration” stack only, using the (Latest) CloudFormation installation option.

To upgrade to version 3.0.0:

Deploy the new Instrumentation and Orchestration stack and the Workload Agents, as described in (Latest) CloudFormation. When deploying the Instrumentation and Orchestration stack, assign a unique name to your macro, for example, Transform: MyV2SysdigMacro. You now have two versions of the serverless agent components. When you are ready to switch from the earlier version, proceed with the next step.
Stop all the running tasks and use CloudFormation to delete the earlier stacks.
Clean up the earlier macro using the installer: ./installer-linux-amd64 cfn-macro delete MyEarlierSysdigMacro
Redeploy the workload stack with the updated CFT (Transform: MyV2SysdigMacro).

(Legacy) CloudFormation

Note: This option has been deprecated.

This option presumes you use a CFT to deploy your workload.

Up to Serverless Agent 2.3.0, a YAML and an installer provided by Sysdig lets you deploy the Orchestrator Agent and the instrumentation service, respectively. Then, you will install the Workload Agent using an automated process which will instrument all the Fargate task definitions in your CFT.

The following components of the serverless agent are installed separately.

For the Orchestrator Agent, Sysdig provides the orchestrator-agent.yaml to use as a CloudFormation Template which you can deploy through the AWS Console. You need one orchestrator deployment per VPC in your environment that your organization wants to secure.
For the instrumentation service, Sysdig provides the installer to run to deploy the instrumentation service that will automatically instrument your task definitions.
For the Workload Agents, you need one Workload Agent per Fargate task definition. For example, if you have ten services and ten task definitions, each needs to be instrumented.

Additional Prerequisites

In addition to the prerequisites defined above, you will need the following on the AWS side:

AWS CLI configured and permissions to create and use an S3 bucket.

Deployment Steps

Install the Orchestrator Agent

Get the CFT orchestrator-agent.yaml to deploy the orchestrator agent.
Deploy the orchestrator agent for each desired VPC, using CloudFormation. The steps below are an outline of the important Sysdig-related parts.
1. Log in to the AWS Console, select CloudFormation and Create Stack with new resources and specify theorchestrator-agent.yaml as the Template source.
2. Specify the stack details to deploy the Orchestrator Agent on the same VPC where your service is running.
  Stack name: self-defined
  Sysdig Settings
  - Sysdig Access Key: Use the agent key of your Sysdig platform.
  - Sysdig Collector Host: collector.sysdigcloud.com (default); region-dependent in Sysdig SaaS; custom in Sysdig on-prem.
  - Sysdig Collector Port: 6443 (default), or could be custom for on-prem installations.
  Network Settings
  - VPC Id: Choose your VPC.
  - VPC Gateway: Choose the type of Gateway in your VPC to balance the load of the orchestrator service.
  - Subnet A & B: These depend on the VPC you choose; select from the drop-down menu.
  Advanced Settings
  - Sysdig Agent Tags: Enter a comma-separated list of tags (eg. role:webserver,location:europe) Note: tags will also be created automatically from your infrastructure’s metadata, including AWS, Docker, etc.
  - Sysdig Orchestrator Agent Image:
    quay.io/orchestrator-agent:latest (default)
  - Check Collector SSL Certificate: Default: true. False means no validation will be done on the SSL certificate received from the collector, used for dev purposes only.
  - Sysdig Orchestrator Agent Port: 6667 (default). Port where the orchestrator service will be listening for instrumented task connections.
3. Click Next to start the deployment, and wait for the deployment to complete (usually a few minutes).
4. From the Outputs tab, take note of the OrchestratorHost and OrchestratorPort values.

Install the Workload Agents

Install the Instrumentation Service

Prerequisite: Have the orchestrator agent deployed in the appropriate VPC and have the Orchestrator Host and Port information handy.
Download the appropriate installer for your OS.
These set up Kilt, an open-source library mechanism for injection into Fargate containers.
Create a macro for the serverless worker agents, using the installer. Any service tagged with this macro will have the serverless worker agent(s) added and Fargate data will be collected.
1. Log in to the AWS CLI.
2. Create a CFN macro that applies instrumentation. You will need the outputs from previous task. Example:
```
./installer-linux-amd64 cfn-macro install -r us-east-1 MySysdigMacro $OrchestratorHost $OrchestratorPort
```

Edit Your CFT

Once the instrumentation service is deployed, you can use the transformation instruction to instrument your workloads. Copy and paste the transformation instruction at the root level of your CFT. All new deployments of that template will be instrumented.

The Sysdig instrumentation will go over the original task definition to instrument it. The instrumentation process includes replacing the original entry point and command of the containers.

For images pulled from private registries, explicitly provide the Entrypoint and Command in the related container definition, otherwise, the instrumentation will not be completed.

Deploy Your CFT

All new deployments of your CFT will be instrumented.

When instrumentation is complete, Fargate events should be visible in the Sysdig Secure Events feed.

Upgrade from a Prior Version

In most cases, it is advised to upgrade directly to 3.0.0 +, as described in (Latest) CloudFormation. These instructions are kept for special cases.

To upgrade to version 2.3.0, follow the instructions in (Legacy) CloudFormation:

Install the Orchestrator Agent, note that the OrchestratorHost and OrchestratorPort values will be unique.
Install the instrumentation service, note that you have to assign a unique name to your macro, for example, Transform: MyV2SysdigMacro. At this point you have two versions of the Serverless agent components. When you are ready to switch from the earlier version, proceed with the next step.
Stop all running tasks and use CloudFormation to delete the earlier stack. Redeploy the new stack with the updated CFT.
Clean up the earlier macro using the installer: ./installer-linux-amd64 cfn-macro delete MyEarlierSysdigMacro
Redeploy the workload stack with the updated CFT (Transform: MyV2SysdigMacro).

Manual Instrumentation

In some cases, you may prefer not to use the Terraform or CloudFormation installers and instead to use one of the following:

Manually Instrument a Task Definition
Instrument a Container Image (rare).

Manually Instrument a Task Definition

Install the orchestrator agent via Terraform or CloudFormation, as described above.

Take note of the OrchestratorHost and OrchestratorPort values. Such parameters will be passed to the workload via the environment variables SYSDIG_ORCHESTRATORand SYSDIG_ORCHESTRATOR_PORT respectively.

Then, instrument the task definition to deploy the workload agent manually as follows:

Add a new container to your existing task definition:
- Use sysdigInstrumentation as a name for the container.
- Obtain the image from quay.io/sysdig/workload-agent:latest.
- The entrypoint and command can be left empty.
Edit the containers you want to instrument.
- Mount volumes from SysdigInstrumentation.
- Add the SYS_PTRACE capability to your container. See AWS Documentation for detail if needed.
- Prepend /opt/draios/bin/instrument to the entrypoint of your container. So, if your original entrypoint is ["my", "original", "entrypoint"], it becomes ["/opt/draios/bin/instrument", "my", "original", "entrypoint"].

Terraform Example

Task definition before the manual instrumentation:

resource "aws_ecs_task_definition" "test_app" {
  execution_role_arn       = aws_iam_role.test_app_execution_role.arn
  task_role_arn            = aws_iam_role.test_app_task_role.arn

  cpu                      = "2048"
  memory                   = "8192"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]

  tags                     = {
    application = "TestApp"
  }

  container_definitions    = jsonencode([
    {
      "name" : "app",
      "image" : "my_image",
      "entrypoint" : ["/bin/ping", "google.com"],
    }
  ])
}

Task definition after the manual instrumentation:

resource "aws_ecs_task_definition" "test_app" {
  execution_role_arn       = aws_iam_role.test_app_execution_role.arn
  task_role_arn            = aws_iam_role.test_app_task_role.arn

  cpu                      = "2048"
  memory                   = "8192"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]

  tags                     = {
    application = "TestApp"
  }

  container_definitions    = jsonencode([
    {
      "name" : "app",
      "image" : "my_image",
      "entrypoint" : ["/opt/draios/bin/instrument", "/bin/ping", "google.com"],
      "linuxParameters": {
        "capabilities": {
          "add": [
            "SYS_PTRACE"
          ],
        },
      },
      "environment": [
        {
          "name": "SYSDIG_ORCHESTRATOR",
          "value": "<host orchestrator output, region dependent>"
        },
        {
          "name": "SYSDIG_ORCHESTRATOR_PORT",
          "value": "6667"
        },
        {
          "name": "SYSDIG_ACCESS_KEY",
          "value": ""
        },
        {
          "name": "SYSDIG_COLLECTOR",
          "value": ""
        },
        {
          "name": "SYSDIG_COLLECTOR_PORT",
          "value": ""
        },
        {
          "name": "SYSDIG_LOGGING",
          "value": ""
        },
      ],
      "volumesFrom": [
        {
          "sourceContainer": "SysdigInstrumentation",
          "readOnly": true
        }
      ],
    },
    {
      "name" : "SysdigInstrumentation",
      "image" : "quay.io/sysdig/workload-agent:latest",
    }
  ])
}

CloudFormation Example

Task definition before the manual instrumentation:

TestApp:
  Type: AWS::ECS::TaskDefinition
  Properties:
    NetworkMode: awsvpc
    RequiresCompatibilities:
      - FARGATE
    Cpu: 2048
    Memory: 8GB
    ExecutionRoleArn: !Ref TestAppExecutionRole
    TaskRoleArn: !Ref TestAppTaskRole
    ContainerDefinitions:
      - Name: App
        Image: !Ref TestAppImage
        EntryPoint:
        - /bin/ping
        - google.com
    Tags:
      - Key: application
        Value: TestApp

Task definition after the manual Instrumentation

TestApp:
  Type: AWS::ECS::TaskDefinition
  Properties:
    NetworkMode: awsvpc
    RequiresCompatibilities:
      - FARGATE
    Cpu: 2048
    Memory: 8GB
    ExecutionRoleArn: !Ref TestAppExecutionRole
    TaskRoleArn: !Ref TestAppTaskRole
    ContainerDefinitions:
      - Name: App
        Image: !Ref TestAppImage
        EntryPoint:
##### BEGIN patch entrypoint for manual instrumentation
        - /opt/draios/bin/instrument
##### END patch entrypoint for manual instrumentation
        - /bin/ping
        - google.com
##### BEGIN add properties for manual instrumentation
        LinuxParameters:
          Capabilities:
            Add:
            - SYS_PTRACE
        VolumesFrom:
        - SourceContainer: SysdigInstrumentation
          ReadOnly: true
        Environment:
        - Name: SYSDIG_ORCHESTRATOR
          Value: "<host orchestrator output, region dependent>"
        - Name: SYSDIG_ORCHESTRATOR_PORT
          Value: "6667"
        - Name: SYSDIG_ACCESS_KEY
          Value: ""
        - Name: SYSDIG_COLLECTOR
          Value: ""
        - Name: SYSDIG_COLLECTOR_PORT
          Value: ""
        - Name: SYSDIG_LOGGING
          Value: ""
      - Name: SysdigInstrumentation
        Image: !Ref WorkloadAgentImage
##### END add properties for manual instrumentation
    Tags:
      - Key: application
        Value: TestApp

Instrument a Container Image

Alternatively, you can include the Workload Agent in your container at build time.

NOTE: the architecture of the image must match the CpuArchitecture of your Fargate RuntimePlatform. That is, given that X86_64 is the only architecture we support for a fargate task definition you will need to build your image using an X86_64 system or you can use docker experimental feature for building for different platform Docker buildx.

To instrument your image at build time, you will need to update your dockerfile to copy the required files and you will need to specify with environment variables the orchestrator that you want to use, please check the following table:

Environment variable	Description	Example
SYSDIG_ORCHESTRATOR	Serverless orchestrator arn	orchestrator.elb.us-east-1.amazonaws.com
SYSDIG_ORCHESTRATOR_PORT	Serverless orchestrator port	6667

Example Dockerfile:

Notice we prepend the /opt/draios/bin/instrument command to the original entrypoint of the container:

ARG sysdig_agent_version=latest
FROM quay.io/sysdig/workload-agent:$sysdig_agent_version AS workload-agent

FROM my_original_base

COPY --from=workload-agent /opt/draios /opt/draios

ENTRYPOINT ["/opt/draios/bin/instrument", "my", "original", "entry", "point"]

Advanced Configurations

Configuring Workload Starting Policy

To customize the Sysdig instrumentation workload starting policy see Configure Workload Starting Policy.

Configuring Instrumentation Logging

To customize the Sysdig instrumentation logging see Manage Serverless Agent Logs.

Enable Proxy

To configure the Orchestrator/Workload agents to use a proxy see Enable HTTP proxy for agents.

The configuration can be provided through the environment variables that follow:

ADDITIONAL_CONF for the Orchestrator Agent.
SYSDIG_EXTRA_CONF for the Workload Agent.

Both of the environment variables expect a valid YAML or JSON.

Topics in This Section

Configure Workload Starting Policy

Manage Serverless Agent Logs

Feedback

Was this page helpful?

Glad to hear it! Please tell us how we can improve.

Sorry to hear that. Please tell us how we can improve.