-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement video filtering with libavfilter #68
Open
KingOfTheBlues
wants to merge
9
commits into
angelcam:master
Choose a base branch
from
KingOfTheBlues:feat/implement-video-filtergraph
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
b76f47b
implement simple video filtergraph with example
KingOfTheBlues a89529c
add feature flag
KingOfTheBlues e7aaa9d
panic on failed init
KingOfTheBlues de7fa68
simplify source and sink init
KingOfTheBlues f9b67e7
deallocate inputs and outputs per example
KingOfTheBlues 81164a4
update builder arg validation
KingOfTheBlues 2d6c329
remove useless check
KingOfTheBlues 92b4f6c
remove assumption that av_buffersrc_add_frame behaves like avcodec_se…
KingOfTheBlues 01aae72
Revert "remove assumption that av_buffersrc_add_frame behaves like av…
KingOfTheBlues File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,159 @@ | ||
use std::{fs::File, time::Duration}; | ||
|
||
use ac_ffmpeg::{ | ||
codec::{ | ||
video::{self, filter::VideoFilter, VideoEncoder, VideoFrameMut}, | ||
CodecParameters, Encoder, Filter, VideoCodecParameters, | ||
}, | ||
format::{ | ||
io::IO, | ||
muxer::{Muxer, OutputFormat}, | ||
}, | ||
time::{TimeBase, Timestamp}, | ||
Error, | ||
}; | ||
use clap::{App, Arg}; | ||
|
||
/// Open a given output file. | ||
fn open_output(path: &str, elementary_streams: &[CodecParameters]) -> Result<Muxer<File>, Error> { | ||
let output_format = OutputFormat::guess_from_file_name(path) | ||
.ok_or_else(|| Error::new(format!("unable to guess output format for file: {}", path)))?; | ||
|
||
let output = File::create(path) | ||
.map_err(|err| Error::new(format!("unable to create output file {}: {}", path, err)))?; | ||
|
||
let io = IO::from_seekable_write_stream(output); | ||
|
||
let mut muxer_builder = Muxer::builder(); | ||
|
||
for codec_parameters in elementary_streams { | ||
muxer_builder.add_stream(codec_parameters)?; | ||
} | ||
|
||
muxer_builder.build(io, output_format) | ||
} | ||
|
||
/// Create h264 encoded black video file of a given length and with a given | ||
/// resolution, with timecode burnt in using the drawtext filter | ||
fn encode_black_video_with_bitc( | ||
output: &str, | ||
width: u32, | ||
height: u32, | ||
duration: Duration, | ||
) -> Result<(), Error> { | ||
// note: it is 1/fps | ||
let time_base = TimeBase::new(1, 25); | ||
|
||
let pixel_format = video::frame::get_pixel_format("yuv420p"); | ||
|
||
// create a black video frame with a given resolution | ||
let frame = VideoFrameMut::black(pixel_format, width as _, height as _) | ||
.with_time_base(time_base) | ||
.freeze(); | ||
|
||
let mut encoder = VideoEncoder::builder("libx264")? | ||
.pixel_format(pixel_format) | ||
.width(width as _) | ||
.height(height as _) | ||
.time_base(time_base) | ||
.build()?; | ||
|
||
let codec_parameters: VideoCodecParameters = encoder.codec_parameters().into(); | ||
|
||
let mut drawtext_filter = VideoFilter::builder() | ||
.input_codec_parameters(&codec_parameters) | ||
.input_time_base(time_base) | ||
.filter_description( | ||
"drawtext=timecode='00\\:00\\:00\\:00':rate=25:fontsize=72:fontcolor=white", | ||
) | ||
.build()?; | ||
|
||
let mut muxer = open_output(output, &[codec_parameters.into()])?; | ||
|
||
let mut frame_idx = 0; | ||
let mut frame_timestamp = Timestamp::new(frame_idx, time_base); | ||
let max_timestamp = Timestamp::from_millis(0) + duration; | ||
|
||
while frame_timestamp < max_timestamp { | ||
let cloned_frame = frame.clone().with_pts(frame_timestamp); | ||
|
||
if let Err(err) = drawtext_filter.try_push(cloned_frame) { | ||
return Err(Error::new(err.to_string())); | ||
} | ||
|
||
while let Some(frame) = drawtext_filter.take()? { | ||
encoder.push(frame)?; | ||
|
||
while let Some(packet) = encoder.take()? { | ||
muxer.push(packet.with_stream_index(0))?; | ||
} | ||
} | ||
|
||
frame_idx += 1; | ||
frame_timestamp = Timestamp::new(frame_idx, time_base); | ||
} | ||
|
||
drawtext_filter.flush()?; | ||
while let Some(frame) = drawtext_filter.take()? { | ||
encoder.push(frame)?; | ||
|
||
while let Some(packet) = encoder.take()? { | ||
muxer.push(packet.with_stream_index(0))?; | ||
} | ||
} | ||
|
||
encoder.flush()?; | ||
|
||
while let Some(packet) = encoder.take()? { | ||
muxer.push(packet.with_stream_index(0))?; | ||
} | ||
|
||
muxer.flush() | ||
} | ||
|
||
fn main() { | ||
let matches = App::new("encoding") | ||
.arg( | ||
Arg::with_name("output") | ||
.required(true) | ||
.takes_value(true) | ||
.value_name("OUTPUT") | ||
.help("Output file"), | ||
) | ||
.arg( | ||
Arg::with_name("width") | ||
.short("w") | ||
.takes_value(true) | ||
.value_name("WIDTH") | ||
.help("width") | ||
.default_value("640"), | ||
) | ||
.arg( | ||
Arg::with_name("height") | ||
.short("h") | ||
.takes_value(true) | ||
.value_name("HEIGHT") | ||
.help("height") | ||
.default_value("480"), | ||
) | ||
.arg( | ||
Arg::with_name("duration") | ||
.short("d") | ||
.takes_value(true) | ||
.value_name("DURATION") | ||
.help("duration in seconds") | ||
.default_value("10"), | ||
) | ||
.get_matches(); | ||
|
||
let output_filename = matches.value_of("output").unwrap(); | ||
let width = matches.value_of("width").unwrap().parse().unwrap(); | ||
let height = matches.value_of("height").unwrap().parse().unwrap(); | ||
let duration = matches.value_of("duration").unwrap().parse().unwrap(); | ||
|
||
let duration = Duration::from_secs_f32(duration); | ||
|
||
if let Err(err) = encode_black_video_with_bitc(output_filename, width, height, duration) { | ||
eprintln!("ERROR: {}", err); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,108 @@ | ||
#include <libavfilter/avfilter.h> | ||
#include <libavformat/avformat.h> | ||
#include <libavfilter/buffersink.h> | ||
#include <libavfilter/buffersrc.h> | ||
#include <libavcodec/avcodec.h> | ||
#include <libavutil/opt.h> | ||
|
||
AVFilterGraph* ffw_filtergraph_new() { | ||
return avfilter_graph_alloc(); | ||
} | ||
|
||
int ffw_filtersource_new(AVFilterContext** filter_ctx, AVFilterGraph* filter_graph, AVCodecParameters* codec_params, int tb_num, int tb_den) { | ||
/* init buffer source: frames from the decoder will be inserted here. */ | ||
char args[512]; | ||
snprintf(args, sizeof(args), | ||
"video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d", | ||
codec_params->width, codec_params->height, codec_params->format, | ||
tb_num, tb_den, | ||
codec_params->sample_aspect_ratio.num, codec_params->sample_aspect_ratio.den); | ||
return avfilter_graph_create_filter(filter_ctx, avfilter_get_by_name("buffer"), "in", args, NULL, filter_graph); | ||
} | ||
|
||
int ffw_filtersink_new(AVFilterContext** filter_ctx, AVFilterGraph* filter_graph) { | ||
/* init buffer sink to terminate the filter chain. */ | ||
return avfilter_graph_create_filter(filter_ctx, avfilter_get_by_name("buffersink"), "out", NULL, NULL, filter_graph); | ||
} | ||
|
||
int ffw_filtergraph_init(AVFilterGraph* filter_graph, | ||
AVFilterContext* buffersrc_ctx, AVFilterContext* buffersink_ctx, | ||
const char* filters_descr) { | ||
int ret = 0; | ||
AVFilterInOut* outputs = avfilter_inout_alloc(); | ||
AVFilterInOut* inputs = avfilter_inout_alloc(); | ||
|
||
/* | ||
* Set the endpoints for the filter graph. The filter_graph will | ||
* be linked to the graph described by filters_descr. | ||
*/ | ||
|
||
/* | ||
* The buffer source output must be connected to the input pad of | ||
* the first filter described by filters_descr; since the first | ||
* filter input label is not specified, it is set to "in" by | ||
* default. | ||
*/ | ||
outputs->name = av_strdup("in"); | ||
outputs->filter_ctx = buffersrc_ctx; | ||
outputs->pad_idx = 0; | ||
outputs->next = NULL; | ||
|
||
/* | ||
* The buffer sink input must be connected to the output pad of | ||
* the last filter described by filters_descr; since the last | ||
* filter output label is not specified, it is set to "out" by | ||
* default. | ||
*/ | ||
inputs->name = av_strdup("out"); | ||
inputs->filter_ctx = buffersink_ctx; | ||
inputs->pad_idx = 0; | ||
inputs->next = NULL; | ||
|
||
if ((ret = avfilter_graph_parse_ptr(filter_graph, filters_descr, &inputs, &outputs, NULL)) < 0) { | ||
goto end; | ||
} | ||
|
||
if ((ret = avfilter_graph_config(filter_graph, NULL)) < 0) { | ||
goto end; | ||
} | ||
|
||
end: | ||
avfilter_inout_free(&inputs); | ||
avfilter_inout_free(&outputs); | ||
|
||
return ret; | ||
} | ||
|
||
int ffw_filtergraph_push_frame(AVFilterContext* context, AVFrame* frame) { | ||
int ret = av_buffersrc_add_frame(context, frame); | ||
|
||
if (ret == 0 || ret == AVERROR_EOF) { | ||
return 1; | ||
} | ||
else if (ret == AVERROR(EAGAIN)) { | ||
return 0; | ||
} | ||
|
||
return ret; | ||
} | ||
|
||
int ffw_filtergraph_take_frame(AVFilterContext* context, AVFrame** out) { | ||
AVFrame* frame = av_frame_alloc(); | ||
int ret = av_buffersink_get_frame(context, frame); | ||
|
||
if (ret == AVERROR_EOF || ret == AVERROR(EAGAIN)) { | ||
return 0; | ||
} | ||
else if (ret < 0) { | ||
return ret; | ||
} | ||
|
||
*out = frame; | ||
|
||
return 1; | ||
} | ||
|
||
void ffw_filtergraph_free(AVFilterGraph* filter_graph) { | ||
avfilter_graph_free(&filter_graph); | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if this is correct. The FFmpeg documentation does not say that
av_buffersrc_add_frame
would behave the same way asavcodec_send_frame
. Is it possible that pushing one frame to a filter input can generate multiple frames at the filter output? Do we also have to consume all output frames before pushing the next input frame (i.e. is it possible thatav_buffersrc_add_frame
would returnAVERROR(EAGAIN)
)?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thought I had addressed this one, actually I reverted the change for now as applying the respective change in the C code yields "all frames must be consumed before flushing" from CodecError