how to modify topics and make it work in consumer at runtime

[moonService]

if topicPattern not enough to describe the possible changes in the topics, how to modify topics and make it work in consumer at runtime

[garyrussell]

It is not clear what you are asking. Perhaps you are referring to this recent enhancement:

spring-kafka/spring-kafka/src/main/java/org/springframework/kafka/core/KafkaAdmin.java

Lines 146 to 154 in 3d8dcce

	/**
	* Set to true to compare the current topic configuration properties with those in the
	* {@link NewTopic} bean, and update if different.
	* @param modifyTopicConfigs true to check and update configs if necessary.
	* @since 2.8.7
	*/
	public void setModifyTopicConfigs(boolean modifyTopicConfigs) {
	this.modifyTopicConfigs = modifyTopicConfigs;
	}

??

[artembilan]

I guess the request is more about subscribing to new topics at runtime.
I wonder if we can (or if that makes sense at all) to implement some add(remove)Topics() into a ConcurrentMessageListenerContainer and restart it respectively with a subsequent rebalance and so on...

[garyrussell]

You can use a pattern (regex) to specify the topics and they will be picked up dynamically (checks are every 5 minutes by default).

/**
* Create properties for a container that will subscribe to topics matching the
* specified pattern. The framework will create a container that subscribes to all
* topics matching the specified pattern to get dynamically assigned partitions. The
* pattern matching will be performed periodically against topics existing at the time
* of check.
* @param topicPattern the pattern.
* @see org.apache.kafka.clients.CommonClientConfigs#METADATA_MAX_AGE_CONFIG
*/
public ContainerProperties(Pattern topicPattern) {

[moonService]

@artembilan that's i needed

[moonService]

@garyrussell we can't use pattern (regex) to describe all changes what is possible

[garyrussell]

We can investigate it; stopping/starting the container after altering the topics would be the simplest solution, but that would cause a rebalance. If that's acceptable, it's probably not too hard to do.

Changing them at runtime would be much more difficult because it would need to be robust with many test cases.