You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/en/docs/bmf/overview/_index.md
+31-11Lines changed: 31 additions & 11 deletions
Original file line number
Diff line number
Diff line change
@@ -6,24 +6,44 @@ weight: 1
6
6
7
7
8
8
# BMF - Cross-platform, multi-language, customizable video processing framework with strong GPU acceleration
9
-
10
9
**BMF (Babit Multimedia Framework)** is a cross-platform, multi-language, customizable multimedia processing framework developed by [**ByteDance**](https://www.bytedance.com/en).
11
-
With over 4 years of testing and improvements, BMF has been tailored to adeptly tackle challenges in our real-world production environments. It is currently widely used in ByteDance's video streaming, live transcoding, cloud editing and mobile pre/post processing scenarios. More than 2 billion videos are processed by the framework every day.
10
+
With over 4 years of testing and improvements, BMF has been tailored to adeptly tackle challenges in our real-world production environments.As a general multimedia processing framework, BabitMF (Babit Multimedia Framework, BMF) can provide a simple and easy-to-use cross-language interface, flexible scheduling and scalability, and dynamically expand, manage and reuse the atomic capabilities of video processing in a modular manner. It builds a high-performance multimedia processing link in the form of graph/pipeline or realize project integration by direct invocation of individual processing capabilities, helping multimedia users to conveniently and efficiently implement projects in the production environment. The usage scenarios of BMF cover video transcoding, video frame extraction, video enhancement, video analysis, video frame insertion, video editing, video conferencing, VR, etc. At present, hundreds of millions of videos are processed by BMF every day. In the process, the functional diversity, ease of use, compatibility, stability and performance of the BMF framework have been fully polished.
The picture above is a general architecture diagram of BMF, including a complete set of ecosystem, which mainly consists of three parts:
15
+
- Application layer: The top layer provides users with multi-language APIs including Python, Go, and C++, which is convenient for users to develop and integrate according to different project requirements.
16
+
- Framework layer: The middle layer includes the framework's scheduling of graph/pipeline, cross-data type and cross-device data transfer Backend, and commonly used cross-device reformat, color space conversion, tensor operator and other SDKs.
17
+
- Module layer: It includes modules with various atomic capabilities, and provides module development mechanisms in multiple languages. Users can implement algorithms/processing in any of Python, Go, and C++ languages according to their own needs.
18
+
19
+
From architecture design to implementation, BMF covers a variety of features to solve many challenges in the new era:
20
+
## Multi-language interface
21
+
22
+
BMF provides a multi-language interface, which is convenient for users to integrate development in different languages according to project needs, and maximizes the ability to be compatible and reuse the entire framework.
23
+
24
+
## Loosely coupled multi-language modules
25
+
This module provides atomic capabilities for video processing. You can choose Python, C++, and go to focus on developing modules. Loose coupling between modules and frameworks, or between modules, makes it possible to decouple algorithms and projects and enable development in different languages. The decoupling of the development language used by modules and the upper-layer application greatly enhances scalability and improves diverse collaboration capabilities.
26
+
27
+
## Powerful scheduling engine
12
28
13
-
Here are some key features of BMF:
29
+
BMF, which provides a configurable parallel scheduler, multiple usage modes, and advanced features of dynamically adjusting Graph, implements a high-performance scheduling mechanism responsible for DAG/Graph/Pipeline operation, and helps users realize it with a small development cost Functional and performance requirements in complex scenarios through various solutions.
14
30
15
-
-**Cross-Platform Support:** Native compatibility with Linux, Windows, and macOS, as well as optimization for both x86 and ARM CPUs.
31
+
## Fully compatible with FFmpeg features and standards
16
32
17
-
-**Easy to use:** BMF provides Python, Go, and C++ APIs, allowing developers the flexibility to code in their favourite languages.
33
+
BMF is fully compatible with FFmpeg's processing capabilities and indicators, such as demuxing, decoding, filter, encoding, and muxing capabilities. The configuration of these processing capabilities and the results consistent with FFmpeg's pts, duration, bitrate, fps and other indicators can be obtained. Very good It satisfies the need to quickly integrate FFmpeg capabilities into projects.
18
34
19
-
-**Customizability:** Developers can enhance the framework's features by adding their own modules independently because of BMF decoupled architecture.
35
+
## Ability to convert data between multiple frameworks and heterogeneous devices
20
36
21
-
-**High performance:** BMF has a powerful scheduler and strong support for heterogeneous acceleration hardware. Moreover, [**NVIDIA**](https://www.nvidia.com/) has been cooperating with us to develop a highly optimized GPU pipeline for video transcoding and AI inference.
37
+
Supports zero-copy interconversion between multiple mainstream data formats, and is greatly compatible with data connection between different modules. For example, it is necessary to use ffmpeg's codec and filter capabilities, as well as OpenCV's transformation processing, and it also needs to use Pytorch to do intelligent analysis in more complex situations; at the same time, it supports heterogeneous cross-device data transmission and conversion capabilities, and asynchronous scheduling capabilities, so that Help users quickly solve the acceleration optimization problems of the production environment.
22
38
23
-
-**Efficient data conversion:** BMF offers seamless data format conversions across popular frameworks (FFmpeg/Numpy/PyTorch/OpenCV/TensorRT), conversion between hardware devices (CPU/GPU), and color space and pixel format conversion.
39
+
### Enhanced Support for NVIDIA GPU
24
40
41
+
NVIDIA GPU has dedicated video encoding and decoding hardware; in addition, it has high bandwidth memory and large scale parallel computing CUDA cores, which are suitable for general purpose video and image processing. Working with NVIDIA engineers, BMF has enhanced support for NVIDIA GPU:
42
+
* NVENC/NVDEC/GPU filters work out-of-box by inheriting abilities from FFmpeg.
43
+
* High performance frame processing is enabled by integration of CV-CUDA and customized CUDA kernels.
44
+
* AI inferencing can be easily integrated into video pipeline using TensorRT.
45
+
* Data moving between CPU and GPU can be done by a simple call.
25
46
26
-
[**BMFLite**](./bmf_lite/README.md) is a client-side cross-platform, lightweight, more efficient client-side multimedia processing framework.
27
-
So far, the BMFLite client-side algorithm is used in apps such as Douyin/Xigua, serving more than one billion users in live streaming/video playing/pictures/cloud games and other scenarios, and processing videos and pictures trillions of times every day.
47
+
In short, all kinds of video and image SDKs from NVIDIA have been integrated into BMF. So the developer can write GPU accelerated video processing pipelines with Python/Go/C++ easily and run them efficiently.
28
48
29
-
Dive deeper into BMF's capabilities on our [website](https://babitmf.github.io/)for more details.
49
+
In the form of open source, BMF fully integrates into the community, continuously expands the ecology, displays various solutions, and expects to provide better help for developers and enterprises in the industry in the hyper video era.
@@ -71,7 +71,7 @@ If you want to have a quick experiment, you can try it on [](https://colab.research.google.com/github/BabitMF/bmf/blob/master/bmf/demo/colorization_python/deoldify_demo_colab.ipynb)
This demo implements the super-resolution inference process of [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) as a BMF module, showcasing a BMF pipeline that combines decoding, super-resolution inference and encoding.
@@ -96,82 +96,9 @@ If you want to have a quick experiment, you can try it on [](https://colab.research.google.com/github/BabitMF/bmf/blob/master/bmf/test/sync_mode/bmf_syncmode_python.ipynb)
133
-
C++:[](https://colab.research.google.com/github/BabitMF/bmf/blob/master/bmf/test/sync_mode/bmf_syncmode_cpp.ipynb)
134
-
Go:[](https://colab.research.google.com/github/BabitMF/bmf/blob/master/bmf/test/sync_mode/bmf_syncmode_go.ipynb)
135
-
-[Create a Module](https://babitmf.github.io/docs/bmf/getting_started_yourself/create_a_module/)
136
-
- customize module with python, C++ and Go. You can try it on [](https://colab.research.google.com/github/BabitMF/bmf/blob/master/bmf/test/customize_module/bmf_customize_demo_latest.ipynb)
137
-
138
-
-[Multiple Features (with examples)](https://babitmf.github.io/docs/bmf/multiple_features/)
-[API in Python](https://babitmf.github.io/docs/bmf/api/api_in_python/)
154
-
-[API in Cpp](https://babitmf.github.io/docs/bmf/api/api_in_cpp/)
155
-
-[API in Go](https://babitmf.github.io/docs/bmf/api/api_in_go/)
156
-
157
-
-[License](#license)
158
-
-[Contributing](#contributing)
159
-
160
-
## License
161
-
The project has an [Apache 2.0 License](https://github.com/BabitMF/bmf/blob/master/LICENSE). Third party components and dependencies remain under their own licenses.
We use GitHub issues to track and resolve problems. If you have any questions, please feel free to join the discussion and work with us to find a solution.
169
-
170
-
## Acknowledgment
171
-
172
-
173
-
The decoder, encoder and filter reference [ffmpeg cmdline tool](http://ffmpeg.org/). They are wrapped as BMF's built-in modules under the LGPL license.
174
-
175
-
The project also draws inspiration from other popular frameworks, such as [ffmpeg-python](https://github.com/kkroening/ffmpeg-python) and [mediapipe](https://github.com/google/mediapipe). Our [website](https://babitmf.github.io/) is using the project from [docsy](https://github.com/google/docsy) based on [hugo](https://github.com/gohugoio/hugo).
176
-
177
-
Here, we'd like to express our sincerest thanks to the developers of the above projects!
0 commit comments