Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pushing video frame data to custom stream server #1750

Open
mrtndat opened this issue Mar 15, 2025 · 8 comments
Open

Pushing video frame data to custom stream server #1750

mrtndat opened this issue Mar 15, 2025 · 8 comments

Comments

@mrtndat
Copy link

mrtndat commented Mar 15, 2025

Hi,
I am using your library to push frame data to RTMP server and it works fine.
Now, I need to push that frame to 3rd library which does not expose a rtmp url. It means, I can not use startStream() API.
I want to use RootEncoder to capture camera frame, apply filter, then use that 3rd SDK to push this frame to its stream server.

What I tried.

  1. Accessed the raw camera image using addImageListener. then convert the buffer to NV21 format and pass it to 3rd lib. It is working well, but it is missing the filters, because at this phase, the filter renderer is not applied yet.
  2. Tried to intercept the VideoEncoder.sendBuffer(byteBuffer, bufferInfo) to send this byteBuffer to 3rd lib, but it seems this byteBuffer's format is not recognized. I tried I420, ARGB, NV21 but the 3rd did not recognize it.
  3. Tried to expose the texture id of the Screen Renderer and pass this texture id and eglContext to the 3rd lib. However, I just got crash saying that the eglContext is not set, when using EGL14.eglGetCurrentContext() as eglContext, it results in a green frame.

What I intend to do.

  1. Create a custom Filter to access the texture id and pass this id to 3rd but im not sure if it works because this Filter phase is supposed to do rendering not pushing.

My question is: Is there a callback that notifies a processed frame data, or provide a way to push the processed frame data to external consumer.

Thanks

@pedroSG94
Copy link
Owner

Hello,

In that case I recommend you create your own RtmpStream class. Copy this class:
https://github.com/pedroSG94/RootEncoder/blob/master/library/src/main/java/com/pedro/library/rtmp/RtmpStream.kt
Now, you can remove RtmpClient and use your own library instead of RtmpClient

After that, you can use the class exactly the same way you are currently using RootEncoder library.
This apply to all classes in the library. Just open the source code and replace the client class.

@mrtndat
Copy link
Author

mrtndat commented Mar 15, 2025

Hi @pedroSG94
You mean this method, right.
override fun getVideoDataImp(videoBuffer: ByteBuffer, info: MediaCodec.BufferInfo) {
rtmpClient.sendVideo(videoBuffer, info)
}

Instead of using rtmp client, I will use the 3rd lib to send the video buffer?
May i know what this buffer format is? is it NV21 or I420?

@pedroSG94
Copy link
Owner

Yes, that method.
The buffer is already encoded. H264, H265 or AV1 depend of your codec selected (h264 by default)

@mrtndat
Copy link
Author

mrtndat commented Mar 15, 2025

Thanks, but I have tried it, the 3rd lib only support sending frame in NV21, I420 or ARGB format.

@pedroSG94
Copy link
Owner

Hello,

In that case you can decode the frame. Copy this class (it is not the same in the current release, I did a modification to support this case):
https://github.com/pedroSG94/RootEncoder/blob/master/extra-sources/src/main/java/com/pedro/extrasources/BufferDecoder.kt
Now, you can decode frames using that class and the result should be nv21. Code example:

    val decoder = BufferDecoder(VideoCodec.H264)
    //use the same values than in prepareVideo, width, height, fps and rotation
    decoder.prepare(640, 480, 30, 0)
    //null because we want receive a buffer when decode instead of render a surface.
    //the return indicate the raw color received when decode. check MediaCodecInfo.CodecCapabilities values
    val resultColor = decoder.start(null)
    //call it when needed. null if fail or if you set a surface in the start method
    val rawBuffer = decoder.decode(data)
    //stop it after finish
    decoder.stop()

@mrtndat
Copy link
Author

mrtndat commented Mar 16, 2025

Thanks, but I can not access the BufferDecoder class from that link (it returns 404).

@pedroSG94
Copy link
Owner

I moved the class my bad:
https://github.com/pedroSG94/RootEncoder/blob/master/encoder/src/main/java/com/pedro/encoder/input/decoder/BufferDecoder.kt

@mrtndat
Copy link
Author

mrtndat commented Mar 16, 2025

I will try it and let you know.
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants