Forwarding to Kafka Topic

Kafka is a distributed system consisting of servers and clients that communicate via a high-performance TCP network protocol. It can be deployed on bare-metal hardware, virtual machines, or containers in on-premise as well as cloud environments.

Events are organized and durably stored in topics. Very simplified, a topic is similar to a folder in a filesystem, and the events are the files in that folder.

Configure Event Forwarder Integration with a Kafka Topic

To forward secure data to Kafka:

  • From the Settings module of the Sysdig Secure UI, navigate to the Events Forwarding tab.

  • Click the Add Integration button.

  • Select Kafka topic from the drop-down menu.

  • Configure the required options:

    Integration Name: Define an integration name.

    Brokers: Kafka server endpoints. A Kafka cluster may provide several brokers; it follows the “hostname: port” (without protocol scheme). You can list several using a comma-separated list.

    Topic: Kafka topic where you want to store the forwarded data

    Partitioner/Balancer: Algorithm that the client uses to multiplex data between the multiple Brokers. For compatibility with the Java client, Murmur2 is used as the default partitioner. Supported algorithms are:

    • Murmur2

    • Round robin

    • Least bytes

    • Hash

    • CRC32

    Compression: Compression standard used for the data. Supported algorithms are:

    • LZ4

    • Snappy

    • Gzip

    • Standard

    Data to send: At present, the integration with Kafka topic supports forwarding Secure Policy Events

    Select whether or not you want to allow insecure connections (i.e. invalid or self-signed certificate on the receiving side).

    Toggle the enable switch as necessary. Remember that you will need to “Test Integration” with the button below before enabling the integration.

  • Click the Save button to save the integration.

Last modified July 17, 2021: Aliases to old site urls (#98) (917a9be2)