Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue: https://github.com/apache/pulsar/issues/23555. #4

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open
12 changes: 6 additions & 6 deletions .github/workflows/integration-test.yaml
Original file line number Diff line number Diff line change
@@ -71,22 +71,22 @@ jobs:
distribution: 'adopt'
java-version: 17

- name: install org.apache.pulsar.tests:integration:jar:tests:3.0.7
- name: install org.apache.pulsar.tests:integration:jar:tests:4.0.1
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
cd ~
git clone --depth 50 --single-branch --branch v3.0.7 https://github.com/apache/pulsar
git clone --depth 50 --single-branch --branch v4.0.1 https://github.com/apache/pulsar
cd pulsar
mvn -B -ntp -f tests/pom.xml -pl org.apache.pulsar.tests:tests-parent,org.apache.pulsar.tests:integration install

- name: build apachepulsar/pulsar-test-latest-version:latest
if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
run: |
docker pull apachepulsar/pulsar-all:3.0.7
docker pull apachepulsar/pulsar:3.0.7
docker tag apachepulsar/pulsar-all:3.0.7 apachepulsar/pulsar-all:latest
docker tag apachepulsar/pulsar:3.0.7 apachepulsar/pulsar:latest
cd ~/pulsar
docker pull apachepulsar/pulsar-all:4.0.1
docker pull apachepulsar/pulsar:4.0.1
docker tag apachepulsar/pulsar-all:4.0.1 apachepulsar/pulsar-all:4.0.1-$(git rev-parse --short=7 HEAD)
docker tag apachepulsar/pulsar:4.0.1 apachepulsar/pulsar:4.0.1-$(git rev-parse --short=7 HEAD)
mvn -B -ntp -f tests/docker-images/pom.xml install -pl org.apache.pulsar.tests:latest-version-image -am -Pdocker,-main -DskipTests

- name: run integration tests
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -34,11 +34,11 @@ mvn install
In order to build this repository the linked Pulsar release must be released to Maven Central
other wise you have to build it locally.

For instance if this code depends on Pulsar 3.0.7 you have to build Pulsar 3.0.7 locally
For instance if this code depends on Pulsar 4.0.1 you have to build Pulsar 4.0.1 locally

```
git clone https://github.com/apache/pulsar
git checkout v3.0.7
git checkout v4.0.1
mvn clean install -DskipTests
```

4 changes: 2 additions & 2 deletions examples/kafka-streams/pom.xml
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@
<parent>
<groupId>org.apache.pulsar.examples</groupId>
<artifactId>pulsar-adapters-examples</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
</parent>

<artifactId>kafka-streams</artifactId>
@@ -35,7 +35,7 @@
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
</dependency>

<!-- Apache Kafka dependencies -->
2 changes: 1 addition & 1 deletion examples/pom.xml
Original file line number Diff line number Diff line change
@@ -25,7 +25,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-adapters</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
</parent>

<groupId>org.apache.pulsar.examples</groupId>
2 changes: 1 addition & 1 deletion examples/spark/pom.xml
Original file line number Diff line number Diff line change
@@ -25,7 +25,7 @@
<parent>
<groupId>org.apache.pulsar.examples</groupId>
<artifactId>pulsar-adapters-examples</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
</parent>

<artifactId>spark</artifactId>
16 changes: 11 additions & 5 deletions pom.xml
Original file line number Diff line number Diff line change
@@ -32,7 +32,7 @@
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-adapters</artifactId>

<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>

<name>Pulsar Adapters</name>
<description>A collections of Pulsar adapters.</description>
@@ -76,7 +76,7 @@
</issueManagement>

<properties>
<pulsar.version>3.0.7</pulsar.version>
<pulsar.version>4.0.1</pulsar.version>
<kafka-client.version>2.7.2</kafka-client.version>
<storm.version>2.0.0</storm.version>
<kafka_0_8.version>0.8.1.1</kafka_0_8.version>
@@ -123,7 +123,7 @@
<surefire.version>3.0.0-M3</surefire.version>
<maven-assembly-plugin.version>3.3.0</maven-assembly-plugin.version>
<maven-dependency-plugin.version>3.1.2</maven-dependency-plugin.version>
<maven-shade-plugin>3.4.1</maven-shade-plugin>
<maven-shade-plugin>3.6.0</maven-shade-plugin>
<maven-antrun-plugin.version>3.0.0</maven-antrun-plugin.version>
<nifi-nar-maven-plugin.version>1.2.0</nifi-nar-maven-plugin.version>
<maven-checkstyle-plugin.version>3.1.2</maven-checkstyle-plugin.version>
@@ -296,6 +296,12 @@
<artifactId>powermock-module-testng</artifactId>
<version>${powermock.version}</version>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>3.12.4</version>
<scope>test</scope>
</dependency>

</dependencies>
</dependencyManagement>
@@ -345,8 +351,8 @@
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.annotation</groupId>
<artifactId>javax.annotation-api</artifactId>
<groupId>jakarta.annotation</groupId>
<artifactId>jakarta.annotation-api</artifactId>
<scope>provided</scope>
</dependency>

2 changes: 1 addition & 1 deletion pulsar-client-kafka-compat/pom.xml
Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-adapters</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka-compat</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka-compat</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka-compat</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

7 changes: 1 addition & 6 deletions pulsar-client-kafka-compat/pulsar-client-kafka-tests/pom.xml
Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka-compat</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

@@ -36,11 +36,6 @@

<description>Tests to verify the correct shading configuration for the pulsar-client-kafka wrapper</description>

<properties>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
</properties>

<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka-compat</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

@@ -83,6 +83,7 @@
<dependency>
<groupId>com.beust</groupId>
<artifactId>jcommander</artifactId>
<version>1.82</version>
<scope>test</scope>
</dependency>

Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka-compat</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

2 changes: 1 addition & 1 deletion pulsar-client-kafka-compat/pulsar-client-kafka/pom.xml
Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka-compat</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

Original file line number Diff line number Diff line change
@@ -66,7 +66,7 @@
import org.apache.pulsar.client.kafka.compat.PulsarClientKafkaConfig;
import org.apache.pulsar.client.kafka.compat.PulsarConsumerKafkaConfig;
import org.apache.pulsar.client.kafka.compat.PulsarKafkaSchema;
import org.apache.pulsar.client.util.ConsumerName;
import org.apache.commons.lang3.RandomStringUtils;
import org.apache.pulsar.client.util.MessageIdUtils;
import org.apache.pulsar.common.naming.TopicName;
import org.apache.pulsar.common.util.FutureUtil;
@@ -270,7 +270,7 @@ public void subscribe(Collection<String> topics, ConsumerRebalanceListener callb
consumerBuilder.subscriptionName(groupId);
if (numberOfPartitions > 1) {
// Subscribe to each partition
consumerBuilder.consumerName(ConsumerName.generateRandomName());
consumerBuilder.consumerName(RandomStringUtils.randomAlphanumeric(5));
for (int i = 0; i < numberOfPartitions; i++) {
String partitionName = TopicName.get(topic).getPartition(i).toString();
CompletableFuture<org.apache.pulsar.client.api.Consumer<byte[]>> future = consumerBuilder.clone()
2 changes: 1 addition & 1 deletion pulsar-client-kafka-compat/pulsar-client-kafka_0_8/pom.xml
Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka-compat</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

6 changes: 5 additions & 1 deletion pulsar-client-kafka-compat/pulsar-client-kafka_0_9/pom.xml
Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka-compat</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

@@ -108,6 +108,10 @@
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
</dependency>
</dependencies>

</project>
Original file line number Diff line number Diff line change
@@ -60,7 +60,7 @@
import org.apache.pulsar.client.kafka.compat.PulsarClientKafkaConfig;
import org.apache.pulsar.client.kafka.compat.PulsarConsumerKafkaConfig;
import org.apache.pulsar.client.kafka.compat.PulsarKafkaSchema;
import org.apache.pulsar.client.util.ConsumerName;
import org.apache.commons.lang3.RandomStringUtils;
import org.apache.pulsar.client.util.MessageIdUtils;
import org.apache.pulsar.common.naming.TopicName;
import org.apache.pulsar.common.util.FutureUtil;
@@ -221,7 +221,7 @@ public void subscribe(List<String> topics, ConsumerRebalanceListener callback) {
consumerBuilder.subscriptionName(groupId);
if (numberOfPartitions > 1) {
// Subscribe to each partition
consumerBuilder.consumerName(ConsumerName.generateRandomName());
consumerBuilder.consumerName(RandomStringUtils.randomAlphanumeric(5));
for (int i = 0; i < numberOfPartitions; i++) {
String partitionName = TopicName.get(topic).getPartition(i).toString();
CompletableFuture<org.apache.pulsar.client.api.Consumer<byte[]>> future = consumerBuilder.clone()
2 changes: 1 addition & 1 deletion pulsar-log4j2-appender/pom.xml
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-adapters</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

2 changes: 1 addition & 1 deletion pulsar-spark/pom.xml
Original file line number Diff line number Diff line change
@@ -26,7 +26,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-adapters</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

2 changes: 1 addition & 1 deletion pulsar-storm/pom.xml
Original file line number Diff line number Diff line change
@@ -26,7 +26,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-adapters</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>

2 changes: 1 addition & 1 deletion tests/pom.xml
Original file line number Diff line number Diff line change
@@ -26,7 +26,7 @@
<parent>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-adapters</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
</parent>
<groupId>org.apache.pulsar.tests</groupId>
<artifactId>adapters-tests-parent</artifactId>
2 changes: 1 addition & 1 deletion tests/pulsar-kafka-compat-client-test/pom.xml
Original file line number Diff line number Diff line change
@@ -25,7 +25,7 @@
<parent>
<groupId>org.apache.pulsar.tests</groupId>
<artifactId>adapters-tests-parent</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
</parent>

<artifactId>pulsar-kafka-compat-client-test</artifactId>
4 changes: 2 additions & 2 deletions tests/pulsar-spark-test/pom.xml
Original file line number Diff line number Diff line change
@@ -26,7 +26,7 @@
<parent>
<groupId>org.apache.pulsar.tests</groupId>
<artifactId>adapters-tests-parent</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
</parent>

<artifactId>pulsar-spark-test</artifactId>
@@ -41,7 +41,7 @@
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-spark</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<scope>test</scope>
</dependency>

4 changes: 2 additions & 2 deletions tests/pulsar-storm-test/pom.xml
Original file line number Diff line number Diff line change
@@ -26,7 +26,7 @@
<parent>
<groupId>org.apache.pulsar.tests</groupId>
<artifactId>adapters-tests-parent</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
</parent>

<artifactId>pulsar-storm-test</artifactId>
@@ -43,7 +43,7 @@
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-storm</artifactId>
<version>3.0.7</version>
<version>4.0.0-SNAPSHOT</version>
<scope>test</scope>
<exclusions>
<exclusion>