FFmpeg module filter documentation / Code cleanup

[jpc0]

Current behaviour

server/src/modules/ffmpeg/producer/av_producer.cpp

Line 273 in b0ecbee

struct Filter

A lot of the filter code exists but it isn't entirely clear why it exists, for instance in this loop we check for whether the disposition has been set to default or is unset and these streams are later merged together using an alphamerge. This indicates to me that this is for forced subtitles however should be also be looking for disposition == AV_DISPOSITION_FORCED ? This could be a subtle bug that has crept in because this should probably be in a separate function or at least commented.

Expected behaviour

A clear and concise description of what the current behaviour is in a comment or in a function name.

Additional Information

I'm happy to work in this but need to know what is actually happening in the codebase since there may be checks put in place in here to catch edge cases but that hasn't been documented. FFmpeg is a pretty obscure library, any documentation as to why something was done goes a long way, specially when what happens in this codebase diverges from FFmpeg example code.

[Julusian]

I know very little about the ffmpeg code we have, it is an area I have been trying to avoid where possible, so I won't be able to provide many answers. Hopefully the git history can provide some insights?

I have a feeling that alphamerge example was done as a way to allow for having a file containing 2 h264 streams in a file, and to use the second as the alpha channel. I guess as an alternative method to the _A method we have.

[jpc0]

I have a feeling that alphamerge example was done as a way to allow for having a file containing 2 h264 streams in a file, and to use the second as the alpha channel. I guess as an alternative method to the _A method we have.

That is also possible. Honestly I think there are so many different uses of casparcg out there I'm not sure without documentation that we wouldn't be able to break someone's use case. I mean my primary use case is media playout to OBS personally and for work it is a glorified NDI->SDI converter...

[jpc0]

https://github.com/CasparCG/help/wiki/FFmpeg-Producer#transitions-between-videos

Transitions Between Videos
You can play two simultaneous video files from disk to the same consumer during any of the built-in transitions such as mix, wipe, push and slide of a user-selectable number of frames. If your video files have alpha channel (key) those will work as well.

Do you know how this would be invoked from AMCP?

I have a feeling that alphamerge example was done as a way to allow for having a file containing 2 h264 streams in a file, and to use the second as the alpha channel. I guess as an alternative method to the _A method we have.

I'm thinking that the code here from above is to implement this.

[jpc0]

#696

So this is a feature where you can create a network stream with a key and a fill channel which will get merged together into a straight alpha video in CasparCG.

I'm guessing this is in use somewhere since it was brought back in 0ceba5a after being removed.

[Julusian]

CasparCG/help/wiki/FFmpeg-Producer#transitions-between-videos

Transitions Between Videos
You can play two simultaneous video files from disk to the same consumer during any of the built-in transitions such as mix, wipe, push and slide of a user-selectable number of frames. If your video files have alpha channel (key) those will work as well.

Do you know how this would be invoked from AMCP?

Those docs are a bit misleading, the transition code is generic and not specific to the ffmpeg producer.
The transition producer is created from amcp at https://github.com/CasparCG/server/blob/master/src/protocol/amcp/AMCPCommandsImpl.cpp#L289-L303
The implementation of these transitions is in https://github.com/CasparCG/server/tree/master/src/core/producer/transition
And has some help https://github.com/CasparCG/server/blob/master/src/core/producer/layer.cpp#L77-L89 to be provided the 'leading_producer' (the one being transition away from)

[jpc0]

Those docs are a bit misleading, the transition code is generic and not specific to the ffmpeg producer.

Makes sense. Thought it might be a little easier to do it somewhere else. Will probably have a think about how to reword that section or just remove it...

[5opr4ni]

https://github.com/CasparCG/help/wiki/FFmpeg-Producer#transitions-between-videos

Transitions Between Videos
You can play two simultaneous video files from disk to the same consumer during any of the built-in transitions such as mix, wipe, push and slide of a user-selectable number of frames. If your video files have alpha channel (key) those will work as well.

Do you know how this would be invoked from AMCP?

I have a feeling that alphamerge example was done as a way to allow for having a file containing 2 h264 streams in a file, and to use the second as the alpha channel. I guess as an alternative method to the _A method we have.

I'm thinking that the code here from above is to implement this.

True! Fill and Key