Storage: Configure Options for Capture Files
The Sysdig Capture feature allows you to record detailed system trace data via remote connection from any of your agent-installed hosts. In SaaS installations, by default, this data will be stored on Sysdig's secure Amazon S3 storage location, under a separate partition for your account. In on-premises installations, by default, the data will be stored in the Cassandra database.
This page describes two custom alternatives: using an AWS S3 bucket (available for SaaS and on-prem) and using custom S3 storage (on-prem only).
Configure AWS S3 Storage
To configure this option, use the fields provided Sysdig Settings UI and then append some code to the IAM Policy you created in AWS for Sysdig integration.
Prerequisites
Your AWS account must be integrated with Sysdig, but the CloudWatch feature is not required to be enabled.
See AWS: Integrate AWS Account and CloudWatch Metrics (Optional)
Have your S3 bucket name available.
To use your own AWS S3 bucket to store Sysdig capture files, append the following code snippets to the AWS Identity and Access Management (IAM) page.
{ "Version": "2012-10-17", "Statement": [ { "Action": [ "s3:Put*", "s3:List*", "s3:Delete*", "s3:Get*" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::BUCKET_NAME", "arn:aws:s3:::BUCKET_NAME/*" ] } ] }
In the Sysdig UI
Log in as Administrator to Sysdig Monitor or Sysdig Secure.
From the Selector button in the lower-left navigation, select
Settings >Sysdig Storage.
Enable the
Use a custom S3 bucket
toggle and enter your AWS S3 bucket name.
To Test: Capture a Trace File in Sysdig Monitor UI
When enabled, you will have the option to select between "Sysdig Monitor Storage" or your own storage bucket when configuring a file capture. See Create a Sysdig Capture File.
Configure Custom S3 Endpoint
You can set up a custom Amazon-S3-compatible storage, such as Minio or IBM Cloud Object Storage, for storing Captures in a Sysdig on-premises deployment. The capture storage location can be used for both Sysdig Monitor and Sysdig Secure. This is an API-only functionality and currently, no UI support is available.
You must configure values.yaml
corresponding to your Sysdig installation in order for this configuration to work.
Prerequisites
Your on-premise installation is Installer-based. If you have installed Sysdig Platform manually and you want to configure custom S3 buckets to store your capture files, contact your Sysdig representative.
Ensure that AWS-client compatible credentials used for authentication are present in the environment.
Ensure that the list, get, and put operations are functional on the S3 bucket that you wish to use. Confirm this by using the S3 native tools, for example, as described in AWS CLI for IBM Cloud.
Configure Installer
Configure the following parameters in the values.yaml
file so that collectors, workers, and the API server are aware of the custom endpoint configuration.
sysdig.s3.enabled
Required: true Description: Specifies if storing Sysdig Captures in S3 or S3-compatible storage is enabled or not. Options:true|false Default:false
For example:
sysdig: s3: enabled: true
sysdig.s3.endpoint
Required: true Description: The S3 or S3-compatible endpoint for the bucket. This option is ignored if sysdig.s3.enabled is not configured.
For example:
sysdig: s3: endpoint: <your S3-Compatible custom bucket>
sysdig.s3.bucketName
Required: true Description: The name of the S3 or S3-compatible bucket to be used for captures. This option is ignored if sysdig.s3.enabled is not configured
For example:
sysdig: s3: bucketName: <Name of the S3-compatible bucket to be used for captures>
sysdig.accessKey
Required: true Description: The AWS or AWS-compatible access key to be used by Sysdig components to write captures in the S3 bucket.
For example:
sysdig: accessKey: <AWS-compatible access key>
sysdig.secretKey
Required: true Description: The AWS or AWS-compatible secret key to be used by Sysdig components to write captures in the s3 bucket.
For example:
sysdig: secretKey: <AWS-compatible secret key>
For example, the following AWS CLI command uploads a Sysdig Capture file to a Minio bucket:
aws --profile minio --endpoint http://10.101.140.1:9000 s3 cp <Sysdig Capture filename> s3://test/
In this example, the endpoint is http://10.101.140.1:9000/
and the name of the bucket is test
.
When you finish the S3 configuration, continue with the instructions on on-premise installation by using the Installer.