AWS Elastic Registry Scanner
AWS ECR Single Account
Use the single account AWS Elastic Container Registry (ECR) installation to target one registry per installation.
To onboard an entire organization containing several member accounts, check the AWS ECR Organizational setup
Ensure you update the Sysdig Helm chart before installation. This lets you access the latest features and fixes.
Use one of the following installation methods based on where the Kubernetes cluster is located:
Same AWS Account
If Elastic Kubernetes Service (EKS) and the registry are in the same account, launch the installation in that account with an AWS EKS. No specific credential setup is required, since EKS already grants AmazonEC2ContainerRegistryReadOnly policy to the default nodes-eks-node-group-* role.
$ helm upgrade --install registry-scanner sysdig/registry-scanner --version=1 \
--set config.secureBaseURL=<SYSDIG_SECURE_URL> \
--set config.secureAPIToken=<SYSDIG_SECURE_API_TOKEN> \
--set config.registryType=ecr \
--set config.registryURL=<AWS_REGISTRY_URL> \
--set config.aws.region=<AWS_REGION>
<AWS_REGION>: Registry AWS region<AWS_REGISTRY_URL>: Registry URL, such as123456789012.dkr.ecr.us-east-1.amazonaws.com.
On Red Hat OpenShift Service on AWS (ROSA), the Registry Scanner pod cannot use the worker node role. Follow Red Hat OpenShift Service on AWS (ROSA) instead.
Different Account or Kubernetes Not in AWS
Credentials are required in this case:
$ helm upgrade --install registry-scanner sysdig/registry-scanner --version=1 \
--set config.secureBaseURL=<SYSDIG_SECURE_URL> \
--set config.secureAPIToken=<SYSDIG_SECURE_API_TOKEN> \
--set config.registryType=ecr \
--set config.registryURL=<AWS_REGISTRY_URL> \
--set config.aws.region=<AWS_REGION> \
--set config.aws.accessKeyId=<AWS_ACCESS_KEY_ID> \
--set config.aws.secretAccessKey=<AWS_SECRET_ACCESS_KEY>
<AWS_REGION>: Registry AWS region<AWS_REGISTRY_URL>: Registry URL, such as123456789012.dkr.ecr.us-east-1.amazonaws.com.<AWS_ACCESS_KEY_ID>,<AWS_SECRET_ACCESS_KEY>from an AWS user with the following permissions for the given registry to be scanned:ecr:GetAuthorizationToken ecr:BatchCheckLayerAvailability ecr:GetDownloadUrlForLayer ecr:GetRepositoryPolicy ecr:DescribeRepositories ecr:ListImages ecr:DescribeImages ecr:BatchGetImage ecr:GetLifecyclePolicy ecr:GetLifecyclePolicyPreview ecr:ListTagsForResource ecr:DescribeImageScanFindings
Limitations
- The scanner for single-account will target a single registry; in other words, the registry of one region. To scan several regions, either use the AWS Organizational scanner or set up one scanner per region.
- If the AWS Kubernetes cluster and the registry are in different accounts but under the same organization, use the AWS Organizational setup and allow listing a specific account.
Red Hat OpenShift Service on AWS (ROSA)
On Red Hat OpenShift Service on AWS (ROSA) with hosted control planes (HCP), the Registry Scanner reads ECR with an IAM role bound to its Kubernetes service account. You don’t store AWS access keys in the cluster.
The ROSA worker node role grants read access to ECR, but pods on ROSA cannot use it. The Registry Scanner needs its own IAM role, as described in this section, or access keys.
Prerequisites
- A ROSA with HCP cluster that uses AWS Security Token Service (STS).
- An ECR registry in the AWS account of the cluster. For a registry in another account, see Registry in a Different AWS Account.
- The Registry Scanner Helm chart
1.12.0or later. - Registry Scanner
v0.14.0or later. - The
rosa,aws,oc,helm, andjqcommand-line tools, with permission to create IAM roles and policies in the AWS account.
Find the Cluster Values
A ROSA cluster doesn’t present the same identity values as an EKS cluster. Read them from your cluster:
Get the OIDC issuer of the cluster:
OIDC_ISSUER=$(rosa describe cluster --cluster <CLUSTER_NAME> --output json \ | jq -r .aws.sts.oidc_endpoint_url | sed 's|^https://||') echo "$OIDC_ISSUER"The output looks like
oidc.op1.openshiftapps.com/<OIDC_CONFIG_ID>.Verify that the IAM OIDC provider for that issuer exists in the cluster account:
aws iam get-open-id-connect-provider \ --open-id-connect-provider-arn arn:aws:iam::<AWS_ACCOUNT_ID>:oidc-provider/$OIDC_ISSUER \ --query ClientIDListROSA creates this provider when you create the OIDC configuration of the cluster, so you don’t need to create it as you would on EKS.
Create the IAM Role
Create the trust policy. Replace
<AWS_ACCOUNT_ID>with the ID of the AWS account where you create the role, and<NAMESPACE>with the namespace where you install the Registry Scanner:cat > registry-scanner-trust-policy.json <<EOF { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::<AWS_ACCOUNT_ID>:oidc-provider/${OIDC_ISSUER}" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "${OIDC_ISSUER}:sub": "system:serviceaccount:<NAMESPACE>:registry-scanner", "${OIDC_ISSUER}:aud": "openshift" } } } ] } EOF
ROSA issues service account tokens with the openshift audience. If the trust policy expects sts.amazonaws.com, as on EKS, the Registry Scanner cannot assume the role.
Create the permissions policy. Replace
<AWS_ACCOUNT_ID>with the ID of the AWS account of the registry, and<AWS_REGION>with the registry region:cat > registry-scanner-ecr-policy.json <<EOF { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["ecr:GetAuthorizationToken"], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ecr:DescribeRepositories", "ecr:ListImages", "ecr:DescribeImages", "ecr:BatchGetImage", "ecr:GetDownloadUrlForLayer" ], "Resource": "arn:aws:ecr:<AWS_REGION>:<AWS_ACCOUNT_ID>:repository/*" } ] } EOFecr:GetAuthorizationTokencannot be scoped to a resource. To scan only some repositories, keepecr:DescribeRepositoriesonrepository/*so that the Registry Scanner can discover them, and scope the other actions to those repositories. Repositories outside that scope logAccessDeniederrors forecr:ListImagesduring discovery.Create the role and attach the policy:
aws iam create-role --role-name <ROLE_NAME> \ --assume-role-policy-document file://registry-scanner-trust-policy.json aws iam put-role-policy --role-name <ROLE_NAME> \ --policy-name registry-scanner-ecr-read \ --policy-document file://registry-scanner-ecr-policy.json
Install the Registry Scanner on ROSA
Log in to the cluster with oc login, then install the Registry Scanner in the namespace that you used in the trust policy:
helm upgrade --install registry-scanner sysdig/registry-scanner --version=1 \
--namespace <NAMESPACE> --create-namespace \
--set config.secureBaseURL=<SYSDIG_SECURE_URL> \
--set config.secureAPIToken=<SYSDIG_SECURE_API_TOKEN> \
--set config.registryType=ecr \
--set config.registryURL=<AWS_REGISTRY_URL> \
--set config.aws.region=<AWS_REGION> \
--set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::<AWS_ACCOUNT_ID>:role/<ROLE_NAME>
<AWS_REGISTRY_URL>: Registry URL, such as123456789012.dkr.ecr.us-east-1.amazonaws.com.<AWS_REGION>: Registry AWS region.
When the Helm release is named registry-scanner, the chart names the service account registry-scanner, which matches the sub condition of the trust policy. The ROSA pod identity webhook reads the role from the service account annotation and gives the Registry Scanner pods a web identity token for it.
Set the role through serviceAccount.annotations, as shown. Don’t set config.aws.accessKeyId or config.aws.secretAccessKey.
The Registry Scanner pods run under the default restricted-v2 security context constraint. You don’t need to grant an additional one.
Verify the Installation on ROSA
Start a scan instead of waiting for the schedule:
oc create job --from=cronjob/registry-scanner registry-scanner-manual -n <NAMESPACE>Check the result:
oc logs -n <NAMESPACE> job/registry-scanner-manual | grep -E "credentials provider|Total selected"A successful scan logs
credentials provider type: 'ecr'and a summary such asTotal selected images to be scanned: 3 = Success: 3 + Failed: 0 + Skipped: 0.
The results appear in Sysdig Secure under Vulnerabilities > Registry a few minutes after the scan.
Troubleshoot ROSA
| Symptom | Cause | Resolution |
|---|---|---|
InvalidIdentityToken or AccessDenied for sts:AssumeRoleWithWebIdentity | The trust policy doesn’t match the token | Verify that the federated principal and the condition keys use the cluster OIDC issuer, that aud is openshift, and that sub matches the namespace and the service account name |
using AWS ECR default credentials followed by dial tcp 169.254.169.254:80: connect: connection refused | The service account has no eks.amazonaws.com/role-arn annotation, so the Registry Scanner falls back to the node role, which pods on ROSA cannot reach | Set the annotation through serviceAccount.annotations and run the scan again |
| AWS calls to STS or ECR time out | The VPC routes STS or ECR through interface endpoints whose security group doesn’t allow HTTPS from the worker nodes | Allow inbound TCP 443 from the worker nodes on the endpoint security group |
AccessDenied for ecr:ListImages on some repositories | The permissions policy scopes the read actions to other repositories | Expected for repositories outside the scope. Extend the policy to scan them |
Registry in a Different AWS Account
If the registry is in another AWS account than the cluster, create the IAM role in the registry account, and register the cluster OIDC issuer in that account first:
aws iam create-open-id-connect-provider \
--url "https://${OIDC_ISSUER}" \
--client-id-list openshift
Then follow Create the IAM Role with the registry account ID as <AWS_ACCOUNT_ID>.
To scan the registries of an AWS organization from ROSA, use this role as the service account role in AWS ECR Organizational.
ROSA Limitations
- Validated on ROSA with HCP clusters that use STS. ROSA classic clusters and clusters without STS are not validated.
AWS ECR Organizational
AWS organizational accounts have a management account with multiple member accounts and multiple registries or regions. This option allows scanning of all registries on any member account or the region of an AWS Organization. The Kubernetes cluster that the registry will scan may be inside or outside of AWS.
Process Overview
You will:
- Set up roles and permissions in the AWS console to be used for registry scanning.
- Optionally, validate the setup with a Sysdig test script
- Optionally, add a configuration to the Helm chart to limit the member accounts that will be checked for scanning.
- Install with the Helm chart in one of two ways:
- Option 1: Access Key/Secret Key With an AWS Identity Access Management (IAM) user security credentials, aimed for quick testing or a more simple setup.
- Option 2: Service Account With AWS Kubernetes Service Account, aimed for a more productive scenario.
Set Up Permissions in AWS
In the AWS console, set up authentication roles to allow discoverability.
You will:
- Create one role in the management account (
ORGANIZATION_MANAGEMENT_ROLE_ARN). - A recommended step is to create a role name to be assigned to every targeted member account (
ORGANIZATION_MEMBER_ACCOUNTS_ROLE_NAME). Otherwise, the default AWS member role nameOrganizationAccountAccessRolewill be used, which includesadminprivileges.- Tip: To narrow the list of member accounts that will be checked during scanning, you can set the
aws.AllowListMemberAccountIDsattribute option in the Registry Scanner Helm chart. This eliminates noisy error messages.
- Tip: To narrow the list of member accounts that will be checked during scanning, you can set the
- Optionally, validate the setup using Sysdig’s test script.
Create Roles
- Management role:
ORGANIZATION_MANAGEMENT_ROLE_ARNwill be assumed and used to discover member accounts. - Member role name:
ORGANIZATION_MEMBER_ACCOUNTS_ROLE_NAMEwill be assumed on the member accounts to discover and scan their registries.
Providing Credentials to the Registry Scanner
The registry scanner requires three types of credentials to discover and read Amazon Elastic Container Registry (ECR) images from all member accounts and all regions within an AWS organization.
Kubernetes Job/Pod AWS Credentials
You need to provide either an Access Key/Secret Key or a service account that has the necessary permissions.
Access Key/Secret Key
If you choose to provide an Access Key/Secret Key, use the
<AWS_ACCESS_KEY_ID>,<AWS_SECRET_ACCESS_KEY>format. Then, you need to set the policy permissions required to allow these credentials to impersonate theORGANIZATION_MANAGEMENT_ROLE_ARN.Use the following policy to allow the impersonation:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowAssumeManagementRole", "Effect": "Allow", "Action": "sts:AssumeRole", "Resource": "<ORGANIZATION_MANAGEMENT_ROLE_ARN>" } ] }Service Account
For Amazon Elastic Kubernetes Service (EKS) clusters on AWS, you can configure the necessary permissions at the service account level.
Important points:
Create an IAM OpenID Connect (OIDC) provider for your EKS cluster. Note: If you deployed the cluster with the EKS Terraform module, it was created automatically (check
oidc_provideroutput).In the role associated with the service account (
SERVICE_ACCOUNT_IAM_ROLE_ARN), create a policy to enable assuming the role of the management account.{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Resource": "<ORGANIZATION_MANAGEMENT_ROLE_ARN>", "Sid": "" } ], "Version": "2012-10-17" }In the role associated with the service account (
SERVICE_ACCOUNT_IAM_ROLE_ARN), configure a trust relationship to allow the service account in Kubernetes to assume the role.- Default
SERVICE_ACCOUNT_NAMEwill beregistry-scanner, derived from the helm installation release name.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::<MANAGEMENT_ACCOUNT>:oidc-provider/<OIDC_PROVIDER>" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "<OIDC_PROVIDER_URL>:sub": "system:serviceaccount:<NAMESPACE>:<SERVICE_ACCOUNT_NAME>", "<OIDC_PROVIDER_URL>:aud": "sts.amazonaws.com" } } } ] }- Default
Save the role
<SERVICE_ACCOUNT_IAM_ROLE_ARN>value associated with the Kubernetes service-account and use it during the installation.
Credentials for the Management Role
Provide credentials for the <ORGANIZATION_MANAGEMENT_ROLE_ARN> to discover registries for all member accounts and all regions. You must assign policy permissions and trust relationships to this role.
Assign Policy Permissions
Assign one of the following policy permissions to the management role:
AWSOrganizationsReadOnlyAccess: AWS managed policy whichProvides read-only access to AWS Organizations.Inline policy to allow the management role to assume member account roles:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowManagementRoleToAssumeMember", "Effect": "Allow", "Action": "sts:AssumeRole", "Resource": "arn:aws:iam::*:role/<ORGANIZATION_MEMBER_ACCOUNTS_ROLE_NAME>" } ] }
Assign the Trust Relationship
You must assign the trust relationship to allow this role to be assumed by the principal of the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY / service account configured in the previous step.
- Access Key/Secret Key
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "<AWS_ACCESS_USER_ARN>"
},
"Action": "sts:AssumeRole"
}
]
}
Service Account
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "<SERVICE_ACCOUNT_IAM_ROLE_ARN>" }, "Action": "sts:AssumeRole" } ] }
Credentials for the Member Role Name
You need to provide credentials for the <ORGANIZATION_MEMBER_ACCOUNTS_ROLE_NAME> that is available in all scoped member accounts of the organization with access to discover and read registries within.
This role will be assumed by the ORGANIZATION_MANAGEMENT_ROLE_ARN to impersonate each member account.
By default, the scanner uses the AWS-created administration role OrganizationAccountAccessRole. Using this role will work fine; however, if you need it, you can create a lower-permission role.
(Optional) Least-Privilege-Access Permissions
Create a role in all accounts and assign the following least-privilege-access permissions to it.
The permissions must be assigned to all roles in all accounts that will be scanned.
AmazonEC2ContainerRegistryReadOnly: AWS managed policy thatProvides read-only access to Amazon EC2 Container Registry repositories. Assign the following least-privilege-access permissions to the member role name:Custom permissions to allow region discovery:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "CustomPolicyAllowDescribeRegions", "Effect": "Allow", "Action": "EC2:DescribeRegions", "Resource": "*" } ] }The member role must be assumable by the
ORGANIZATION_MANAGEMENT_ROLE_ARN, so you must assign the following trust relationship to the new role:{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "<ORGANIZATION_MANAGEMENT_ROLE_ARN>" }, "Action": "sts:AssumeRole" } ] }
Also, the ORGANIZATION_MANAGEMENT_ROLE_ARN must be configured to allow assuming the ORGANIZATION_MEMBER_ACCOUNTS_ROLE_NAME you just created in all accounts. You must assign the following policy permission to the management role, which will allow it to assume the member role in all accounts:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowManagementRoleToAssumeMember",
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::*:role/<ORGANIZATION_MEMBER_ACCOUNTS_ROLE_NAME>"
}
]
}
Optionally, if you want to test the role with the validation script of the next section you will need to add the following permissions:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "CustomPolicyAllowDescribeRegions",
"Effect": "Allow",
"Action": "iam:SimulatePrincipalPolicy",
"Resource": "*"
}
]
}
(Optional) Validate Setup with Test Script
Sysdig provides the test script aws-org-test-credentials.sh to check the created roles and permissions, along with validating the discoverables registries throughout the organization.
Launch the script from a shell. Use the AWS authentication credentials for the member account where the Registry Scanner will be deployed.
Note: If you’re running it with some other credentials from the member account, remember that a trust relationship must exist with the Management Account Role. See Assign the Trust Relationship under Credentials for the Management Role.
./aws-org-test-credentials.sh <ORGANIZATION_MANAGEMENT_ROLE_ARN> <ORGANIZATION_MEMBER_ACCOUNTS_ROLE_NAME>
Sample output:
> using *** as member accountID
> Organization accountID: *** with discovered member account IDs
***
-------------------------
> Starting registry discovery and permission checkup
> Account ***
Checking AWS registry permissions
Discovering registries
...
-------------------------
> Discovered Registries
***.dkr.ecr.eu-central-1.amazonaws.com
***.dkr.ecr.eu-west-1.amazonaws.com
...
Install with Helm Chart
Use one of the following installation options depending on the credential format used:
Access Key/Secret Key
Suitable for testing or if your Kubernetes cluster is outside AWS.
<AWS_ACCESS_KEY_ID>, <AWS_SECRET_ACCESS_KEY> from the AWS user
$ helm upgrade --install registry-scanner sysdig/registry-scanner --version=1 \
--set config.secureBaseURL=<SYSDIG_SECURE_URL> \
--set config.secureAPIToken=<SYSDIG_SECURE_API_TOKEN> \
--set config.registryType=ecr \
--set config.aws.accessKeyId=<AWS_ACCESS_KEY_ID> \
--set config.aws.secretAccessKey=<AWS_SECRET_ACCESS_KEY> \
--set config.aws.managementAccountRoleARN=<ORGANIZATION_MANAGEMENT_ROLE_ARN> \
--set config.aws.memberAccountsRoleName=<ORGANIZATION_MEMBER_ACCOUNTS_ROLE_NAME>
Service Account
If you’re using AWS Elastic Kubernetes Service (EKS), you must assign an IAM role (SERVICE_ACCOUNT_IAM_ROLE_ARN) to a Service Account, with which the scanner will run.
$ helm upgrade --install registry-scanner sysdig/registry-scanner --version=1 \
--set config.secureBaseURL=<SYSDIG_SECURE_URL> \
--set config.secureAPIToken=<SYSDIG_SECURE_API_TOKEN> \
--set config.registryType=ecr \
--set serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn=<SERVICE_ACCOUNT_IAM_ROLE_ARN> \
--set config.aws.managementAccountRoleARN=<ORGANIZATION_MANAGEMENT_ROLE_ARN> \
--set config.aws.memberAccountsRoleName=<ORGANIZATION_MEMBER_ACCOUNTS_ROLE_NAME>
Additional Helm Configuration
Limit Registry Accounts to Scan
Optionally, limit the scoped registry accounts using the allowListMemberAccountIDs Helm chart attribute. This approach is preferred for initial testing, when the ORGANIZATION_MEMBER_ACCOUNTS_ROLE_NAME may be provisioned only on select accounts and having the scanner check the rest of the member accounts in the organization would generate noisy error messages.
Set the attributes:
--set config.aws.allowListMemberAccountIDs[0]='<ORG_MEMBER_ACCOUNT_ID_1>' \
--set config.aws.allowListMemberAccountIDs[1]='<ORG_MEMBER_ACCOUNT_ID_2>' \
Limitations
- The scanner for AWS Organizational accounts does not support ECRs hosted in the management account, as per AWS Guidelines best practices for management accounts
- Public registries are not supported.
AWS GovCloud
To install the registry Scanner in GovCloud Accounts (both single and organizational), follow the same instructions as above and
make sure to specify the AWS GovCloud region with --set config.aws.region="us-gov-west-1" and to set
the right AWS GobCloud ARN format.
