Administration
Also available as:
PDF
loading table of contents...
Fastcapa-Kafka Configuration File

The path to the configuration file is specified with the -c command line argument. The file can contain any global or topic-specific, producer-focused configuration values accepted by Librdkafka.

The configuration file is a .ini-like Glib configuration file. The global configuration values should be placed under a [kafka-global] header and topic-specific values should be placed under [kafka-topic].

A minimally viable configuration file would only need to include the Kafka broker to connect to.

[kafka-global]
metadata.broker.list = kafka-broker1:9092, kafka-broker2:9092

The configuration parameters that are important for either basic functioning or performance tuning of Fastcapa include the following.

Global configuration values that should be located under the [kafka-global] header.

NameDescriptionDefault
metadata.broker.listInitial list of brokers as a CSV list of broker host or host:port 
client.idClient identifier. 
queue.buffering.max.messagesMaximum number of messages allowed on the producer queue100000
queue.buffering.max.msMaximum time, in milliseconds, for buffering data on the producer queue1000
message.copy.max.bytesMaximum size for message to be copied to buffer. Messages larger than this will be passed by reference (zero-copy) at the expense of larger iovecs.65535
batch.num.messagesMaximum number of messages batched in one MessageSet10000
statistics.interval.msHow often statistics are emitted; 0 = never0
compression.codecCompression codec to use for compressing message sets; {none, gzip, snappy, lz4 }none

Topic configuration values that should be located under the [kafka-topic] header.

NameDescriptionDefault
compression.codecCompression codec to use for compressing message sets; {none, gzip, snappy, lz4 }none
request.required.acksHow many acknowledgements the leader broker must receive from ISR brokers before responding to the request; { 0 = no ack, 1 = leader ack, -1 = all ISRs }1
message.timeout.msLocal message timeout. This value is only enforced locally and limits the time a produced message waits for successful delivery. A time of 0 is infinite.300000
queue.buffering.max.kbytesMaximum total message size sum allowed on the producer queue