uvgSaliencyCoding Demonstration

Demonstration area for the saliency-guided High Efficiency Video Coding (HEVC) framework, based on Region of Interest (ROI) detection and tracking.

Features

• Real-time performance
• Energy-efficient
• Accurate ROI tracking capabilities
• Optimized bitrates

Installation

Prerequisites

• C/C++ development tools (gcc, make)
• Autotools (autoconf, automake, libtool)
• Python 3.6+
• Git
• ZeroMQ

apt-get install libzmq3-dev

Kvazaar installation

# Initialize and update the Kvazaar submodule
git submodule init
git submodule update

# Apply the required patch
cd kvazaar
git apply ../patches/kvazaar_demo_750ac27c.patch
cd ..

# Compile and install
cd kvazaar
./autogen.sh
./configure
make -j
sudo make install
sudo ldconfig
cd ..

NOTE: For the most up-to-date installation information, please refer to the Kvazaar repository.

uvgComm installation

# Initialize and update the uvgComm submodule
git submodule init
git submodule update

# Apply the required patch
cd uvgComm
git apply ../patches/uvgComm_demo_7b05640e.patch
cd ..

# Compile and install
cd uvgComm
mkdir build
cd build
cmake ..
make -j
cd ../..

NOTE: For the most up-to-date installation information, please refer to the uvgComm repository.

uvgSaliencyCoding installation

# Initialize and update the uvgComm submodule
git submodule init
git submodule update

# Apply the required patch
cd uvgSaliencyCoding
git apply ../patches/uvgSaliencyCoding_demo_fbd516d.patch

# Install Python dependencies
pip install -r requirements.txt

Energy measurement

Important: For granting Intel Energy Permission, run:

sudo chmod -R a+r /sys/class/powercap/intel-rapl

Pretrained Models

The pretrained weights are provided in the uvgSaliencyCoding/weights directory. Download our pretrained weights to quickly get started with ROI detection and tracking:

Model	Type	Trained On	Size
yolov8n-head	Detection	HollywoodHeads	24.5MB
light-tracker-head	Tracking	UVG-ROI dataset	40.8kB
enhanced-tracker-head	Tracking	UVG-ROI dataset	19.4MB

Usage

Launching uvgComm

Before running any encoding operations, you must first launch the uvgComm communication module:

uvgComm/build/uvgComm

Activating the saliency-guided framework

Important: The current implementation requires sequential activation of components. The saliency-guided framework must be initiated prior to establishing any communication sessions.

python uvgSaliencyCoding/uvgComm.py

Citation

If you use this framework in your research, please cite this paper:

@ARTICLE{10820524,
  author={Partanen, Tero and Hoang, Minh and Mercat, Alexandre and Sainio, Joose and Vanne, Jarno},
  journal={IEEE Journal on Emerging and Selected Topics in Circuits and Systems}, 
  title={Energy-Efficient Saliency-Guided Video Coding Framework for Real-Time Applications}, 
  year={2025},
  volume={15},
  number={1},
  pages={44-57},
  keywords={Encoding;Streaming media;Video coding;Image coding;Saliency detection;Energy efficiency;Energy consumption;Visualization;Object tracking;Computational modeling;Saliency-guided encoding;region-of-interest (ROI);ROI tracking;deep learning (DL);motion vector (MV)},
  doi={10.1109/JETCAS.2024.3525339}}