Key is not used for partitionning

[mantzas]

I have added the 'key' to the payload, which is the ID of one of my entities, so that i get the same entity in the same partition. Our topic has 4 partitions. Unfortunately it does not seem to work.
We expect that with the above the message will always go in one of the 4 partitions, but we have seen that the message will be send to all partitions.
Do you have any idea with this is the case?

[lcobucci]

@mantzas can you please check if #218 fulfils your needs?

[mantzas]

@lcobucci is this not already implemented in the library?

[lcobucci]

@mantzas nope, that's why I'm asking you to check that particular branch. I'll be reviewing it further this week to integrate, but will be only available in v3.x

[mantzas]

@lcobucci yes, this is what's needed from our side. We integrated the current code in our codebase to make this work now. We will wait until v3 to get the library then. Thanks man

[demyanovs]

@mantzas @lcobucci
Currently, in order not to change this client and wait for the merge #218, you can get information directly from the broker and proceed as follows:

$broker = \Kafka\Broker::getInstance();
$topic_infos = $broker->getTopics();
$topic_meta = $topic_infos[$topic_title];
$part_nums = array_keys($topic_meta);
if ($key) {
    $part_id = $part_nums[crc32($key) % count($part_nums)];
} else {
    shuffle($part_nums);
    $part_id = $part_nums[0];
}

And then set partId

$producer->send([
    [
        'topic' => $topic_title,
        'value' => $message,
        'partId' => $part_id,
    ],
]);

[mantzas]

@demyanovs done this already based on #218. it's running in production already. thanks.