@RetryableTopic submits the updated state of event on retry topic

[mayur-msb-7]

Hello Kafka Community,

Context:
We're implementing a retry mechanism in our service using Spring Kafka. This involves a combination of blocking retries and non-blocking retries via @RetryableTopic.

Requirement:
During event processing, we modify the event's state to track progress and avoid redundant work on retries. The goal is to resume processing from the point of failure on a retry, not from the beginning. For this purpose, we increment a counter within the event data on each successful processing step.

Observation:
Our tests reveal a discrepancy between blocking retries and event state:

Non-blocking retries (via @RetryableTopic): The event published to the retry topic contains the updated state as expected.

Blocking retries: The event processed by the retry logic retains the original state, not the updated state. This leads to redundant processing on retries.

Question:
We're seeking clarification on this behavior. Is this a known issue, or is there a configuration or approach we've missed to ensure that blocking retries also receive the updated event state?

Expected Outcome:
We aim to achieve consistent behavior where both blocking and non-blocking retries leverage the updated event state on the retry topic, allowing for efficient and non-redundant retries.

Git Repository: https://github.com/mayur-msb-7/kafka-retry

[artembilan]

The non-blocking retry goes to a new Kafka topic, in your case it is user.info.retry.
The blocking one does everything in-memory, therefore you cannot see any updates from the topic and so on.
That stepCount property is something custom in your logic.
I doubt the framework has to do anything with that one.