Forwarding to Kafka Topic
Kafka is a distributed system consisting of servers and clients that communicate via a high-performance TCP network protocol. It can be deployed on bare-metal hardware, virtual machines, or containers in on-premise as well as cloud environments.
Events are organized and durably stored in topics. Very simplified, a topic is similar to a folder in a filesystem, and the events are the files in that folder.
Configure Event Forwarder Integration with a Kafka Topic
To forward secure data to Kafka:
From the
Settings
module of the Sysdig Secure UI, navigate to theEvents Forwarding
tab.Click the
Add Integration
button.Select
Kafka topic
from the drop-down menu.Configure the required options:
Integration Name: Define an integration name.
Brokers: Kafka server endpoints. A Kafka cluster may provide several brokers; it follows the “hostname: port” (without protocol scheme). You can list several using a comma-separated list.
Topic: Kafka topic where you want to store the forwarded data
Partitioner/Balancer: Algorithm that the client uses to multiplex data between the multiple Brokers. For compatibility with the Java client, Murmur2 is used as the default partitioner. Supported algorithms are:
Murmur2
Round robin
Least bytes
Hash
CRC32
Compression: Compression standard used for the data. Supported algorithms are:
LZ4
Snappy
Gzip
Standard
Data to send: At present, the integration with Kafka topic supports forwarding Secure Policy Events
Select whether or not you want to allow insecure connections (i.e. invalid or self-signed certificate on the receiving side).
Toggle the enable switch as necessary. Remember that you will need to “Test Integration” with the button below before enabling the integration.
Click the
Save
button to save the integration.