[FLINK-36488] [TABLE SQL/API] Remove deprecated methods StreamTableEnvironment.toAppendStream from flink-table-api-java-bridge module #25528

sn-12-3 · 2024-10-16T05:32:39Z

JIRA issue

FLINK-36488

What is the purpose of the change

Remove deprecated method from StreamTableEnvironment.toAppendStream . This links to parent issue FLINK-36476

Brief change log

Remove deprecated methods StreamTableEnvironment.toAppendStream from flink-table-api-java-bridge module.

Verifying this change

Use mvn spotless:apply to verify.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? not applicable

flinkbot · 2024-10-16T05:36:41Z

CI report:

ed78575 UNKNOWN
2831f98 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

…vironment.toAppendStream from flink-table-api-java-bridge module

davidradl · 2024-12-05T11:35:47Z

@sn-12-3 do you mean to have this PR in draft ?

sn-12-3 · 2024-12-05T11:38:31Z

Yes @davidradl , This PR CI is failing due to a python failure which still needs investigation.

xuyangzhong · 2025-01-02T05:47:32Z

Hi, @sn-12-3 Is there any new progress?

… method

…est dependent on the method

sn-12-3 · 2025-01-13T15:44:00Z

@flinkbot run azure

sn-12-3 · 2025-01-14T04:35:48Z

Hi @xuyangzhong , This PR is ready for review now. I have removed the toAppendStream method from Scala and few tests which had references. Kindly let me know if any further changes is needed.

xuyangzhong

Thanks for updating this pr! I just left some comments.
BTW, it seems that the cleanup hasn't been completely done. When I search for toAppendStream and to_append_stream in the IDE, I still find many references that need to be removed, such as in doc/content/docks/dev/table/data_stream_api.md, table_environment.py, etc.

xuyangzhong · 2025-01-14T11:32:03Z

flink-python/pyflink/datastream/tests/test_stream_execution_environment.py

@@ -260,57 +260,6 @@ def test_execute_async(self):
        execution_result = job_client.get_job_execution_result().result()
        self.assertEqual(str(job_id), str(execution_result.get_job_id()))

-    def test_add_python_file(self):


I don't think we can directly remove this test, as it seems to verify other functionalities.

Used to_data_stream instead of to_append_stream.

xuyangzhong · 2025-01-14T11:49:55Z

...src/test/scala/org/apache/flink/table/planner/runtime/stream/sql/WindowAggregateITCase.scala

@@ -1290,47 +1290,6 @@ class WindowAggregateITCase(
      .isEqualTo(expected.sorted.mkString("\n"))
  }

-  @TestTemplate
-  def testDistinctAggWithMergeOnEventTimeSessionWindow(): Unit = {


Used toDataStream instead of toAppendStream.

xuyangzhong · 2025-01-14T11:52:02Z

...java-bridge/src/main/java/org/apache/flink/table/api/bridge/java/StreamTableEnvironment.java

-    <T> DataStream<T> toAppendStream(Table table, TypeInformation<T> typeInfo);
-
-    /**
-     * Converts the given {@link Table} into a {@link DataStream} of add and retract messages. The


You removed the comment for toRetractStream but kept the comment for toAppendStream.

My bad. Apologies, corrected now.

…eview comments.

sn-12-3 · 2025-01-15T08:26:56Z

@flinkbot run azure

davidradl · 2025-01-15T13:55:00Z

flink-python/pyflink/table/table_environment.py

+        Converts the given Table into a DataStream.
+
+        Since the DataStream API does not support changelog processing natively, this method
+        assumes append-only/insert-only semantics during the table-to-stream conversion. The records


nit: the documentation only refers to insert-only. Can we remove append-only or is this indicating another semantic?

Maybe involve parts of the original comment "Converts the given Table into a DataStream of a specified type. Since the DataStream API does not support changelog processing natively, the table must only have insert (append) changes. If the Table is also modified by update or delete changes, the conversion will fail."

It seems to indicate another semantic fit for insert-only operation. Elsewhere in the codebase, the same documentation is maintained.

After the changes based on the review comments, this is outdated as no new function has been added in this file. Only to_append_stream relevant function has been removed.

davidradl · 2025-01-15T13:59:32Z

flink-python/pyflink/table/tests/test_table_environment_api.py

+                                                                          Types.STRING()]))
+        test_sink = DataStreamTestSinkFunction()
+        ds.add_sink(test_sink)
+        self.env.execute("test_to_data_stream")


can we test deletes and updates error cases?

I'm afraid I understand, this is adding the sink operator to the dataStream function, there is no functions related to delete or update.

After the changes based on the review comments, this is outdated as no new test has been added in this file. Only toAppendStream relevant test case has been removed.

sn-12-3 · 2025-01-18T11:59:12Z

@flinkbot run azure

sn-12-3 · 2025-01-18T17:11:22Z

@xuyangzhong , @davidradl , Thanks for the review. Incorporated the review comments; PR is ready for re-review.

sn-12-3 · 2025-01-27T14:26:20Z

@xuyangzhong , @davidradl , Gentle reminder.

sn-12-3 · 2025-02-20T17:03:17Z

@flinkbot run azure

sn-12-3 changed the title ~~[FLINK-36488] [TABLE SQL/API] Remove deprecated methods StreamTableEn…~~ [FLINK-36488] [TABLE SQL/API] Remove deprecated methods StreamTableEnvironment.toAppendStream from flink-table-api-java-bridge module Oct 16, 2024

flinkbot added the component=TableSQL/API label Oct 16, 2024

[FLINK-36488] [TABLE SQL/API] Remove deprecated methods StreamTableEn…

4a70d1f

…vironment.toAppendStream from flink-table-api-java-bridge module

sn-12-3 force-pushed the 36488-remove-deprecated-methods branch from b14a9b3 to ee7e6b9 Compare October 17, 2024 06:46

sn-12-3 force-pushed the 36488-remove-deprecated-methods branch from ee7e6b9 to 4a70d1f Compare November 12, 2024 07:39

Merge branch 'apache:master' into 36488-remove-deprecated-methods

6da38a6

sn-12-3 force-pushed the 36488-remove-deprecated-methods branch from 91b315b to 6da38a6 Compare January 12, 2025 09:49

“shalini-m20” added 2 commits January 12, 2025 21:31

removing python test case having dependency on the removed deprecated…

3676175

… method

removed toAppendStream from Scala StreamTableEnvironment and python t…

8c869c8

…est dependent on the method

sn-12-3 force-pushed the 36488-remove-deprecated-methods branch from f0ac0a3 to 8c869c8 Compare January 13, 2025 15:14

Merge branch 'apache:master' into 36488-remove-deprecated-methods

30eed1d

removed toAppendStream method reference from WindowAggregateITCase

7cfb0ba

sn-12-3 marked this pull request as ready for review January 14, 2025 04:27

xuyangzhong reviewed Jan 14, 2025

View reviewed changes

“shalini-m20” and others added 6 commits January 14, 2025 22:46

cleanup for toAppendStream and to_append_stream from docs and fixed r…

721188e

…eview comments.

Merge branch 'apache:master' into 36488-remove-deprecated-methods

fb0b5d8

fixed review comments

03443a1

Fixed review comments and cleanup for deprecated method

24bc5fd

Merge branch 'apache:master' into 36488-remove-deprecated-methods

ed78575

cleanup for deprecated method

6a56e17

davidradl reviewed Jan 15, 2025

View reviewed changes

Merge branch 'apache:master' into 36488-remove-deprecated-methods

5c5886c

“shalini-m20” and others added 8 commits January 15, 2025 23:01

renamed method due to duplicate method name

cacce48

Merge branch 'apache:master' into 36488-remove-deprecated-methods

780bae6

Merge branch 'apache:master' into 36488-remove-deprecated-methods

6c7a6b4

corrected indentation and doc

19bc007

Merge branch 'apache:master' into 36488-remove-deprecated-methods

62ea99b

corrected test failure

8b39ade

Merge branch 'apache:master' into 36488-remove-deprecated-methods

f6eac34

correcte test failures

c1900ad

Merge branch 'apache:master' into 36488-remove-deprecated-methods

ad21377

sn-12-3 added 3 commits January 28, 2025 21:58

Merge branch 'apache:master' into 36488-remove-deprecated-methods

31b7780

Merge branch 'apache:master' into 36488-remove-deprecated-methods

3250f79

Merge branch 'apache:master' into 36488-remove-deprecated-methods

fb478df

sn-12-3 requested review from xuyangzhong and davidradl February 11, 2025 11:36

sn-12-3 added 3 commits February 13, 2025 12:02

Merge branch 'apache:master' into 36488-remove-deprecated-methods

d13bd85

Merge branch 'apache:master' into 36488-remove-deprecated-methods

e7e0bff

Merge branch 'apache:master' into 36488-remove-deprecated-methods

1a8f489

Merge branch 'apache:master' into 36488-remove-deprecated-methods

2831f98

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FLINK-36488] [TABLE SQL/API] Remove deprecated methods StreamTableEnvironment.toAppendStream from flink-table-api-java-bridge module #25528

[FLINK-36488] [TABLE SQL/API] Remove deprecated methods StreamTableEnvironment.toAppendStream from flink-table-api-java-bridge module #25528

sn-12-3 commented Oct 16, 2024 •

edited

Loading

flinkbot commented Oct 16, 2024 •

edited

Loading

davidradl commented Dec 5, 2024

sn-12-3 commented Dec 5, 2024 •

edited

Loading

xuyangzhong commented Jan 2, 2025

sn-12-3 commented Jan 13, 2025

sn-12-3 commented Jan 14, 2025

xuyangzhong left a comment

xuyangzhong Jan 14, 2025

sn-12-3 Jan 18, 2025 •

edited

Loading

xuyangzhong Jan 14, 2025

sn-12-3 Jan 18, 2025 •

edited

Loading

xuyangzhong Jan 14, 2025

sn-12-3 Jan 18, 2025

sn-12-3 commented Jan 15, 2025

davidradl Jan 15, 2025 •

edited

Loading

sn-12-3 Jan 18, 2025 •

edited

Loading

davidradl Jan 15, 2025

sn-12-3 Jan 18, 2025 •

edited

Loading

sn-12-3 commented Jan 18, 2025

sn-12-3 commented Jan 18, 2025

sn-12-3 commented Jan 27, 2025

sn-12-3 commented Feb 20, 2025

[FLINK-36488] [TABLE SQL/API] Remove deprecated methods StreamTableEnvironment.toAppendStream from flink-table-api-java-bridge module #25528

Are you sure you want to change the base?

[FLINK-36488] [TABLE SQL/API] Remove deprecated methods StreamTableEnvironment.toAppendStream from flink-table-api-java-bridge module #25528

Conversation

sn-12-3 commented Oct 16, 2024 • edited Loading

JIRA issue

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

flinkbot commented Oct 16, 2024 • edited Loading

CI report:

davidradl commented Dec 5, 2024

sn-12-3 commented Dec 5, 2024 • edited Loading

xuyangzhong commented Jan 2, 2025

sn-12-3 commented Jan 13, 2025

sn-12-3 commented Jan 14, 2025

xuyangzhong left a comment

Choose a reason for hiding this comment

xuyangzhong Jan 14, 2025

Choose a reason for hiding this comment

sn-12-3 Jan 18, 2025 • edited Loading

Choose a reason for hiding this comment

xuyangzhong Jan 14, 2025

Choose a reason for hiding this comment

sn-12-3 Jan 18, 2025 • edited Loading

Choose a reason for hiding this comment

xuyangzhong Jan 14, 2025

Choose a reason for hiding this comment

sn-12-3 Jan 18, 2025

Choose a reason for hiding this comment

sn-12-3 commented Jan 15, 2025

davidradl Jan 15, 2025 • edited Loading

Choose a reason for hiding this comment

sn-12-3 Jan 18, 2025 • edited Loading

Choose a reason for hiding this comment

davidradl Jan 15, 2025

Choose a reason for hiding this comment

sn-12-3 Jan 18, 2025 • edited Loading

Choose a reason for hiding this comment

sn-12-3 commented Jan 18, 2025

sn-12-3 commented Jan 18, 2025

sn-12-3 commented Jan 27, 2025

sn-12-3 commented Feb 20, 2025

sn-12-3 commented Oct 16, 2024 •

edited

Loading

flinkbot commented Oct 16, 2024 •

edited

Loading

sn-12-3 commented Dec 5, 2024 •

edited

Loading

sn-12-3 Jan 18, 2025 •

edited

Loading

sn-12-3 Jan 18, 2025 •

edited

Loading

davidradl Jan 15, 2025 •

edited

Loading

sn-12-3 Jan 18, 2025 •

edited

Loading

sn-12-3 Jan 18, 2025 •

edited

Loading