Skip to content

Commit 9335563

Browse files
author
yang-xuetong
committed
Modify as required
1 parent e7fc6fb commit 9335563

File tree

10 files changed

+76
-169
lines changed

10 files changed

+76
-169
lines changed

content/en/docs/bmf/_index.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,3 @@ menu:
77
weight: 1
88
parent: 'Documentation'
99
---
10-

content/en/docs/bmf/overview/_index.md

Lines changed: 31 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -6,24 +6,44 @@ weight: 1
66

77

88
# BMF - Cross-platform, multi-language, customizable video processing framework with strong GPU acceleration
9-
109
**BMF (Babit Multimedia Framework)** is a cross-platform, multi-language, customizable multimedia processing framework developed by [**ByteDance**](https://www.bytedance.com/en).
11-
With over 4 years of testing and improvements, BMF has been tailored to adeptly tackle challenges in our real-world production environments. It is currently widely used in ByteDance's video streaming, live transcoding, cloud editing and mobile pre/post processing scenarios. More than 2 billion videos are processed by the framework every day.
10+
With over 4 years of testing and improvements, BMF has been tailored to adeptly tackle challenges in our real-world production environments.As a general multimedia processing framework, BabitMF (Babit Multimedia Framework, BMF) can provide a simple and easy-to-use cross-language interface, flexible scheduling and scalability, and dynamically expand, manage and reuse the atomic capabilities of video processing in a modular manner. It builds a high-performance multimedia processing link in the form of graph/pipeline or realize project integration by direct invocation of individual processing capabilities, helping multimedia users to conveniently and efficiently implement projects in the production environment. The usage scenarios of BMF cover video transcoding, video frame extraction, video enhancement, video analysis, video frame insertion, video editing, video conferencing, VR, etc. At present, hundreds of millions of videos are processed by BMF every day. In the process, the functional diversity, ease of use, compatibility, stability and performance of the BMF framework have been fully polished.
11+
12+
<img src="/img/docs/bmf-framework.png" style="zoom:100%;" />
13+
14+
The picture above is a general architecture diagram of BMF, including a complete set of ecosystem, which mainly consists of three parts:
15+
- Application layer: The top layer provides users with multi-language APIs including Python, Go, and C++, which is convenient for users to develop and integrate according to different project requirements.
16+
- Framework layer: The middle layer includes the framework's scheduling of graph/pipeline, cross-data type and cross-device data transfer Backend, and commonly used cross-device reformat, color space conversion, tensor operator and other SDKs.
17+
- Module layer: It includes modules with various atomic capabilities, and provides module development mechanisms in multiple languages. Users can implement algorithms/processing in any of Python, Go, and C++ languages according to their own needs.
18+
19+
From architecture design to implementation, BMF covers a variety of features to solve many challenges in the new era:
20+
## Multi-language interface
21+
22+
BMF provides a multi-language interface, which is convenient for users to integrate development in different languages according to project needs, and maximizes the ability to be compatible and reuse the entire framework.
23+
24+
## Loosely coupled multi-language modules
25+
This module provides atomic capabilities for video processing. You can choose Python, C++, and go to focus on developing modules. Loose coupling between modules and frameworks, or between modules, makes it possible to decouple algorithms and projects and enable development in different languages. The decoupling of the development language used by modules and the upper-layer application greatly enhances scalability and improves diverse collaboration capabilities.
26+
27+
## Powerful scheduling engine
1228

13-
Here are some key features of BMF:
29+
BMF, which provides a configurable parallel scheduler, multiple usage modes, and advanced features of dynamically adjusting Graph, implements a high-performance scheduling mechanism responsible for DAG/Graph/Pipeline operation, and helps users realize it with a small development cost Functional and performance requirements in complex scenarios through various solutions.
1430

15-
- **Cross-Platform Support:** Native compatibility with Linux, Windows, and macOS, as well as optimization for both x86 and ARM CPUs.
31+
## Fully compatible with FFmpeg features and standards
1632

17-
- **Easy to use:** BMF provides Python, Go, and C++ APIs, allowing developers the flexibility to code in their favourite languages.
33+
BMF is fully compatible with FFmpeg's processing capabilities and indicators, such as demuxing, decoding, filter, encoding, and muxing capabilities. The configuration of these processing capabilities and the results consistent with FFmpeg's pts, duration, bitrate, fps and other indicators can be obtained. Very good It satisfies the need to quickly integrate FFmpeg capabilities into projects.
1834

19-
- **Customizability:** Developers can enhance the framework's features by adding their own modules independently because of BMF decoupled architecture.
35+
## Ability to convert data between multiple frameworks and heterogeneous devices
2036

21-
- **High performance:** BMF has a powerful scheduler and strong support for heterogeneous acceleration hardware. Moreover, [**NVIDIA**](https://www.nvidia.com/) has been cooperating with us to develop a highly optimized GPU pipeline for video transcoding and AI inference.
37+
Supports zero-copy interconversion between multiple mainstream data formats, and is greatly compatible with data connection between different modules. For example, it is necessary to use ffmpeg's codec and filter capabilities, as well as OpenCV's transformation processing, and it also needs to use Pytorch to do intelligent analysis in more complex situations; at the same time, it supports heterogeneous cross-device data transmission and conversion capabilities, and asynchronous scheduling capabilities, so that Help users quickly solve the acceleration optimization problems of the production environment.
2238

23-
- **Efficient data conversion:** BMF offers seamless data format conversions across popular frameworks (FFmpeg/Numpy/PyTorch/OpenCV/TensorRT), conversion between hardware devices (CPU/GPU), and color space and pixel format conversion.
39+
### Enhanced Support for NVIDIA GPU
2440

41+
NVIDIA GPU has dedicated video encoding and decoding hardware; in addition, it has high bandwidth memory and large scale parallel computing CUDA cores, which are suitable for general purpose video and image processing. Working with NVIDIA engineers, BMF has enhanced support for NVIDIA GPU:
42+
* NVENC/NVDEC/GPU filters work out-of-box by inheriting abilities from FFmpeg.
43+
* High performance frame processing is enabled by integration of CV-CUDA and customized CUDA kernels.
44+
* AI inferencing can be easily integrated into video pipeline using TensorRT.
45+
* Data moving between CPU and GPU can be done by a simple call.
2546

26-
[**BMFLite**](./bmf_lite/README.md) is a client-side cross-platform, lightweight, more efficient client-side multimedia processing framework.
27-
So far, the BMFLite client-side algorithm is used in apps such as Douyin/Xigua, serving more than one billion users in live streaming/video playing/pictures/cloud games and other scenarios, and processing videos and pictures trillions of times every day.
47+
In short, all kinds of video and image SDKs from NVIDIA have been integrated into BMF. So the developer can write GPU accelerated video processing pipelines with Python/Go/C++ easily and run them efficiently.
2848

29-
Dive deeper into BMF's capabilities on our [website](https://babitmf.github.io/) for more details.
49+
In the form of open source, BMF fully integrates into the community, continuously expands the ecology, displays various solutions, and expects to provide better help for developers and enterprises in the industry in the hyper video era.

content/en/docs/bmf/quick_experience/_index.md

Lines changed: 4 additions & 77 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ This demo uses BMF framework to construct a simple broadcast service. The servic
2121

2222
Below is a screen recording demonstrating the operation of broadcaster:
2323

24-
![broadcaster](../../../docs/broadcaster.gif)
24+
![broadcaster](https://github.com/BabitMF/bmf/blob/master/bmf/demo/broadcaster/broadcaster.gif)
2525

2626

2727
### GPU acceleration
@@ -71,7 +71,7 @@ If you want to have a quick experiment, you can try it on [![Open In Colab](http
7171

7272
If you want to have a quick experiment, you can try it on [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/BabitMF/bmf/blob/master/bmf/demo/colorization_python/deoldify_demo_colab.ipynb)
7373

74-
![deoldify](../../../docs/deoldify.gif)
74+
![deoldify](https://github.com/BabitMF/bmf/blob/master/bmf/demo/colorization_python/deoldify.gif)
7575

7676
#### Supper Resolution
7777
This demo implements the super-resolution inference process of [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) as a BMF module, showcasing a BMF pipeline that combines decoding, super-resolution inference and encoding.
@@ -96,82 +96,9 @@ If you want to have a quick experiment, you can try it on [![Open In Colab](http
9696
#### Edge AI models
9797
This case illustrates the procedures of integrating an external algorithm module into the BMFLite framework and management of its execution.
9898

99-
<img src="../../../docs/sd1.gif" alt="sr" width="250" height="360">
99+
<img src="https://github.com/BabitMF/bmf/blob/master/bmf_lite/iOS/data/sd1.gif" alt="sr" width="250" height="360">
100100

101101
#### Real-time denoise
102102
This example implements the denoise algorithm as a BMF module, showcasing a BMF pipeline that combines video capture, noise reduction and rendering.
103103

104-
<img src="../../../docs/denoise.gif" alt="sr" width="250" height="360" >
105-
106-
o find a solution.
107-
108-
## Acknowledgment## Table of Contents
109-
110-
- [About BMF](https://babitmf.github.io/about/)
111-
112-
- [Quick Experience](#quick-experience)
113-
- [Transcode](#transcode)
114-
- [Edit](#edit)
115-
- [Meeting/Broadcaster](#meetingbroadcaster)
116-
- [GPU acceleration](#gpu-acceleration)
117-
- [GPU Video Frame Extraction](#gpu-video-frame-extraction)
118-
- [GPU Video Transcoding and Filtering](#gpu-video-transcoding-and-filtering)
119-
- [AI Inference](#ai-inference)
120-
- [Deoldify](#deoldify)
121-
- [Super Resolution](#super-resolution)
122-
- [Video Quality Score](#video-quality-score)
123-
- [Face Detect With TensorRT](#face-detect-with-tensorrt)
124-
125-
- [Getting Started](https://babitmf.github.io/docs/bmf/getting_started_yourself/)
126-
- [Install](https://babitmf.github.io/docs/bmf/getting_started_yourself/install/)
127-
- [Create a Graph](https://babitmf.github.io/docs/bmf/getting_started_yourself/create_a_graph/)
128-
- one of transcode example with 3 languages
129-
- [Use Module Directly](https://babitmf.github.io/docs/bmf/getting_started_yourself/use_module_directly/)
130-
- sync mode with 3 languages. You can try it on:
131-
132-
Python:[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/BabitMF/bmf/blob/master/bmf/test/sync_mode/bmf_syncmode_python.ipynb)
133-
C++:[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/BabitMF/bmf/blob/master/bmf/test/sync_mode/bmf_syncmode_cpp.ipynb)
134-
Go:[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/BabitMF/bmf/blob/master/bmf/test/sync_mode/bmf_syncmode_go.ipynb)
135-
- [Create a Module](https://babitmf.github.io/docs/bmf/getting_started_yourself/create_a_module/)
136-
- customize module with python, C++ and Go. You can try it on [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/BabitMF/bmf/blob/master/bmf/test/customize_module/bmf_customize_demo_latest.ipynb)
137-
138-
- [Multiple Features (with examples)](https://babitmf.github.io/docs/bmf/multiple_features/)
139-
- [Graph Mode](https://babitmf.github.io/docs/bmf/multiple_features/graph_mode/)
140-
- [Generator Mode](https://babitmf.github.io/docs/bmf/multiple_features/graph_mode/generatemode/)
141-
- [Sync Mode](https://babitmf.github.io/docs/bmf/multiple_features/graph_mode/syncmode/)
142-
- [Server Mode](https://babitmf.github.io/docs/bmf/multiple_features/graph_mode/servermode/)
143-
- [Preload Mode](https://babitmf.github.io/docs/bmf/multiple_features/graph_mode/preloadmode/)
144-
- [Subgraph](https://babitmf.github.io/docs/bmf/multiple_features/graph_mode/subgraphmode/)
145-
- [PushData Mode](https://babitmf.github.io/docs/bmf/multiple_features/graph_mode/pushdatamode/)
146-
- [FFmpeg Fully Compatible](https://babitmf.github.io/docs/bmf/multiple_features/ffmpeg_fully_compatible/)
147-
- [Data Convert Backend](https://babitmf.github.io/docs/bmf/multiple_features/data_backend/)
148-
- [Dynamic Graph](https://babitmf.github.io/docs/bmf/multiple_features/dynamic_graph/)
149-
- [GPU Hardware Acceleration](https://babitmf.github.io/docs/bmf/multiple_features/gpu_hardware_acc/)
150-
- [BMF Tools](https://babitmf.github.io/docs/bmf/multiple_features/tools/)
151-
152-
- [APIs](https://babitmf.github.io/docs/bmf/api/)
153-
- [API in Python](https://babitmf.github.io/docs/bmf/api/api_in_python/)
154-
- [API in Cpp](https://babitmf.github.io/docs/bmf/api/api_in_cpp/)
155-
- [API in Go](https://babitmf.github.io/docs/bmf/api/api_in_go/)
156-
157-
- [License](#license)
158-
- [Contributing](#contributing)
159-
160-
## License
161-
The project has an [Apache 2.0 License](https://github.com/BabitMF/bmf/blob/master/LICENSE). Third party components and dependencies remain under their own licenses.
162-
163-
## Contributing
164-
165-
Contributions are welcomed. Please follow the
166-
[guidelines](https://github.com/BabitMF/bmf/blob/master/CONTRIBUTING.md).
167-
168-
We use GitHub issues to track and resolve problems. If you have any questions, please feel free to join the discussion and work with us to find a solution.
169-
170-
## Acknowledgment
171-
172-
173-
The decoder, encoder and filter reference [ffmpeg cmdline tool](http://ffmpeg.org/). They are wrapped as BMF's built-in modules under the LGPL license.
174-
175-
The project also draws inspiration from other popular frameworks, such as [ffmpeg-python](https://github.com/kkroening/ffmpeg-python) and [mediapipe](https://github.com/google/mediapipe). Our [website](https://babitmf.github.io/) is using the project from [docsy](https://github.com/google/docsy) based on [hugo](https://github.com/gohugoio/hugo).
176-
177-
Here, we'd like to express our sincerest thanks to the developers of the above projects!
104+
<img src="https://github.com/BabitMF/bmf/blob/master/bmf_lite/android/app/src/main/res/effect-data/denoise.gif" alt="sr" width="250" height="360" >

content/en/docs/broadcaster.gif

-2.39 MB
Binary file not shown.

content/en/docs/denoise.gif

-3.6 MB
Binary file not shown.

content/en/docs/deoldify.gif

-456 KB
Binary file not shown.

content/en/docs/sd1.gif

-2.33 MB
Binary file not shown.

0 commit comments

Comments
 (0)