streampipes runtime wrapper supports other third-party stream processing system

[wyyolo]

At present , Streampipes already have wrapper kafka-stream and wrapper flink, supporting third-party stream processing systems such as kafka and Flink to handle incoming events. Currently or in the future, is there any consideration to support other stream processing systems? Here I recommend Function Stream. Function stream is an event-streaming function platform based on Apache Pulsar and WebAssembly. It enables efficient and scalable processing of data streams by leveraging the power of WebAssembly. Function Stream provides seamless integration with Apache Pulsar, allowing users to take full advantage of its robust messaging capabilities. For detailed introduction, please refer to here: https://github.com/FunctionStream/function-stream. Unfortunately, Function Stream is still under rapid development. Do you have any thoughts on this?

[dominikriemer]

Hi @WY-OS thank you for suggesting this feature! And also thanks for pointing to FunctionStream which I didn't know so far.
I had a quick look and like the features FunctionStream provides. On the other hand, writing and maintaining a new wrapper takes some effort.
So what do you (and others) think of the following: I really would like to get a 1.0 version of StreamPipes out in the near future, to indicate the maturity of the project. We can start a discussion on missing critical features and bugs we want to fix for this release, and after 1.0 we can discuss on additional larger features. We could also increase our marketing efforts with the 1.0 release, which could (hopefully) also help us to get more contributors to help with the development of larger efforts (e.g., I'd also like to get the Flink wrapper updated which is in a very experimental state currently).

What do you think? I'd be happy to start a discussion on the roadmap towards 1.0.

BTW, I really appreciate your work on the Go client! :-)

[wyyolo]

Hi,@dominikriemer , thank you very much for your recognition of my work on the Go-client！
This discussion was initiated to get everybody views on this issue. The work of introducing Functionstreams is not very urgent at this time. I am looking forward to introducing FunctionStream in the near future when it is more mature.
As you mentioned, we need to discuss the 1.0 roadmap and key missing features before we can release StreamPipes 1.0. I think this is an important thing, and perhaps it would be appropriate for you to initiate a formal discussion.
Finally, thank you again.

[RobertIndie]

@WY-OS Thanks for bringing this discussion to the StreamPipes Community. At present, the Function Stream is still in its early development phase. I agree that we should wait to integrate it with StreamPipes after the 1.0 release. I'm also keen for everyone to test the novel features of Function Stream.

I'd be happy to start a discussion on the roadmap towards 1.0.

@dominikriemer Would you like to create a new thread to discuss the 1.0?

[luoluoyuyu]

+1 I think integrating Function Stream into streampipes is an even better idea!