Add Agent Files

Sysdig support may request additional files related to the agents.

After generating the files, add them to the Support ticket you created. Review the file upload tips on that page.

Debug the Agent Start Command or Manifest File

Many agent connection problems are due to transcription errors in the agent start command or manifest files.

This is especially true with truncated access keys and when using the Additional_Conf parameter in a container agent installation. Always cut and paste and then modify the Sysdig example commands or manifest files.

The docker start and native agent run commands are available in the Settings > Agent Installation tab of the user interface.

If you are not an Admin level user you will not see the installation tab, in this case, please request the instructions from the admin.

  • Try running the agent from the command line using the ‘docker start’ or curl command.

    1. Ensure that you have the agent container ID.

      docker ps -a
      
    2. Stop the container.

      docker stop <container_id>
      
    3. Debug the container. To do so, create a directory and copy log files to that directory.

      mkdir test_log
      docker cp sysdig-agent:/opt/draios/logs ./test_log/
      
    4. Once debugging is complete, start the agent container.

      docker start <container_id>
      
  • Send in the command used and initial agent output.

  • Tip: if using docker start to debug the agent live, remove the ‘-d' option so the output will display on the console.

Sysdig Agent Configuration File and Logs

It’s helpful to attach the config file and latest log to your support request if you see metrics not reporting or have agent connection issues. Since the agent logs critical startup information when initializing, restarting the agent and then collecting the logs is desirable. For a container agent you can use: docker restart sysdig-agent For a native Linux service agent use: service dragent restart.

The Sysdig agent reads the user-settings configuration file /opt/draios/etc/dragent.yaml.It generates log entries in /opt/draios/logs/draios.log .

The agent will rotate out the log file when it reaches 10MB in size keeping the 10 most recent log files archived with a date-stamp appended to the filename.

Include the host name that the files came from.

Debug log level available

Docker Commands

To copy the configuration file and most recent log file out of an agent running in a container use these Docker commands:

docker cp sysdig-agent:/opt/draios/logs/draios.log  ./draios.log
docker cp sysdig-agent:/opt/draios/etc/dragent.yaml ./dragent.yaml

Kubernetes Commands

To get the dragent.yaml config information, run the command:

kubectl -n NAMESPACE describe configmap sysdig-agent > configmap.out

To get the log files, run the command:

kubectl -n NAMESPACE cp sysdig-agent-AGENT_ID:/opt/draios/logs/. .

The above command will grab ALL the log files (including the rollovers) and place them in the directory where the command was run.

Kubernetes: Check for Delegated Agents

With agents deployed in a Kubernetes environment, it can be useful to know which agents are “delegated” to poll the Kubernetes API server.

To do this, you must generate agent log files (see above) and search for ‘delegated’.

Logs are stored in /opt/draios/logs/draios.log.

  • For agent versions 0.77.0 or later: you will see log entries stating whether your agent is/is not the delegated node and the exact address of the delegated node.

    Search for the string “Delegated’.

  • For agents before version 0.77.0: issue the command:

    kubectl get nodes
    

    By default, the top two addresses automatically became delegated polling agents.

  • Alternatively, enabling debug mode in agent versions 0.64.0 and beyond will show the node list in any agent logs as well.

Docker Swarm: Find the Docker Swarm Master

When using a Sysdig agent inside a Docker Swarm, metadata (information about inventory, labels, ext..) is collected from the Docker Swarm masters.

Effort should be made to collect the logs from the masters, and more specifically the “leader” when a problem with Docker Swarm inventory is suspected.

To determine which Docker Swarm nodes are masters and leaders:

  1. Run the command:

    docker node ls
    

    Example:

    $ docker node ls
    ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
    789sgbfdbtcdtn9npbnkuz worker1 Ready Active
    76avknaakzbvmwlnh5jr3g * manager1 Ready Active Leader
    
  2. Under the Manager Status column, you can note which node is the “Leader”.