Manual Upgrade (2.3.0)

As of August 2020, Sysdig has changed its upgrade procedure.

Sysdig platform on-premise releases are listed here. Each release has a version number and specific Release Notes.

This release has the following significant component changes:

1. Includes the option of securing Elasticsearch and/or Cassandra with username/password authentication and TLS-encrypted data in transit. This prevents both unauthorized access to the clusters and network eavesdropping. The upgrade instructions below incorporate this new capability when using the Sysdig-provided Cassandra and Elasticsearch components.

2. Updates of the Postgres database and Anchore engine (if you are running Sysdig Secure).

3. The following parameter has been renamed in config.yaml: elasticsearch.url to elasticsearch.hostname

   The value of elasticsearch.hostname does not include the protocol (e.g.http://); just use the hostname itself. For example, if you had elasticsearch.url: ``http://sysdigcloud-elasticsearch, it would now be elasticsearch.hostname: sysdigcloud-elasticsearch.

Download the New Version

Download the new version from Sysdig’s GitHub and unzip it.

Note that as of this release, versioning standards have changed from a single build number (e.g. v1929) to semantic versioning (e.g. 2.3.0)

wget https://github.com/draios/sysdigcloud-kubernetes/archive/<version_number>.tar.gz &&  tar xvf <version_number>.tar.gz

Edit New Files to Match Your Customized Files

Sysdig Cloud Files

Update the following files with customizations from your existing environment:

• sysdigcloud/config.yaml: Pull configurations from your sysdigcloud-config configmap to the downloaded sysdigcloud/config.yaml.

  The following variables are mandatory for Sysdig installations:

  api.url
  collector.endpoint
  sysdigcloud.license
  

  The following variables are optional but commonly modified for Sysdig installations:

  cassandra.jvm.options
  elasticsearch.jvm.options
  sysdigcloud.jvm.api.options
  sysdigcloud.jvm.collector.options
  sysdigcloud.jvm.worker.options
  

  If you have modified the previous config.yaml, copy the modified options such as the external endpoints. You must also check deployment YAML files for CPU/memory settings.

• Copy configurations from your existing deployment and update the spec.replicas definition in the following files:

  • sysdigcloud/api-deployment.yaml

  • sysdigcloud/collector-deployment.yaml

  • sysdigcloud/worker-deployment.yaml

• If running Sysdig Secure:

-   `sysdigcloud/anchore-core-config.yaml`

-   `sysdigcloud/anchore-worker-config.yaml`

-   `sysdigcloud/anchore-core-deployment.yaml`

-   `sysdigcloud/anchore-worker-deployment.yaml`

-   `sysdigcloud/scanning-api-deployment.yaml`

-   `sysdigcloud/scanning-alertmgr-deployment.yaml`

Postgres File (if running Sysdig Secure)

Update the following file with customizations from your existing environment:

• Modify the storage class name, spec.volumeClaimTemplates[].spec.storageClassName in the datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml file.

Elasticsearch and Cassandra Files

In version 2.3.0, Elasticsearch and Cassandra yaml configurations have been updated. Update the new files with customizations from your existing environment.

• elasticsearch-statefulset.yaml - For example, your environment may have customized the values for the number of replicas, resource constraints, amount of storage, and the storage class name:

  spec.replicas and spec.template.spec.containers[elasticsearch].env[ELASTICSEARCH_GOSSIP_NODES_NUM].value
  spec.template.spec.containers[].resources
  spec.volumeClaimTemplates[].spec.resources.requests.storage
  spec.volumeClaimTemplates[].spec.storageClassName
  

• cassandra-statefulset.yaml - As with Elasticsearch, your environment may have customized the values for the number of replicas, resource constraints, amount of storage, and the storage class name:

  spec.replicas
  spec.template.spec.containers[].resources
  spec.volumeClaimTemplates[].spec.resources.requests.storage
  spec.volumeClaimTemplates[].spec.storageClassName
  

Apply the Files

Run the kubectl commands to apply the relevant files to your cluster.

Note: if you run into an error replacing the statefulsets, you may need to delete the existing one before applying the new configuration. See the Statefulset Deletion and Creation section below.

Upgrade for Sysdig Monitor

NAMESPACE=sysdigcloud
kubectl -n $NAMESPACE apply -f sysdigcloud/config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/api-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/collector-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/worker-deployment.yaml
kubectl -n $NAMESPACE replace -f datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-statefulset.yaml
kubectl -n $NAMESPACE replace -f datastores/as_kubernetes_pods/manifests/cassandra/cassandra-statefulset.yaml

Upgrade for Sysdig Secure

kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-core-config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-worker-config.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-core-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/anchore-worker-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-alertmgr-service.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-api-deployment.yaml
kubectl -n $NAMESPACE apply -f sysdigcloud/scanning-alertmgr-deployment.yaml
kubectl -n $NAMESPACE apply -f datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml

Statefulset Deletion and Creation

If you are unable to update the existing statefulsets with the commands above, you may need to delete them before applying the new configuration.

# Elasticsearch
kubectl -n $NAMESPACE delete statefulset sysdigcloud-elasticsearch
kubectl -n $NAMESPACE apply -f datastores/as_kubernetes_pods/manifests/elasticsearch/elasticsearch-statefulset.yaml

# Cassandra
kubectl -n $NAMESPACE delete statefulset sysdigcloud-cassandra
kubectl -n $NAMESPACE apply -f datastores/as_kubernetes_pods/manifests/cassandra/cassandra-statefulset.yaml

# Postgres (if running Sysdig Secure)
kubectl -n $NAMESPACE delete statefulset sysdigcloud-postgresql
kubectl -n $NAMESPACE apply -f datastores/as_kubernetes_pods/manifests/postgres/postgres-statefulset.yaml

Replace Existing Statefulset Pods

The replace command above only replaces the Kubernetes configuration, but not the running pods themselves. For the changes to take effect, perform the following steps:

1. For Elasticsearch, run:

   kubectl -n $NAMESPACE delete pod -l role=elasticsearch
   kubectl -n $NAMESPACE delete pod -l role=cassandra
   # If running Sysdig Secure
   kubectl -n $NAMESPACE delete pod -l role=postgresql
   

2. Check that all the new pods come up Ready by running the commands below separately:

   kubectl -n $NAMESPACE get pod -l role=elasticsearch
   kubectl -n $NAMESPACE get pod -l role=cassandra
   # If running Sysdig Secure
   kubectl -n $NAMESPACE get pod -l role=postgresql
   

   This may take a few minutes.