Feature Release June 2021

The main new feature in this release is the RT-BENE PyTorch training & inference code; thanks to @ahmed-alhindawi for this. @lindayuanyuan kindly modified the standalone script to save images if there are multiple people in the scene. @ahmed-alhindawi made sure that the frame_ids in the ROS code are consistent between the gaze and blink estimation. Otherwise just some minor bugfixes.

Bugfix Release August 2020

Changes:

• Fix Python2 usage (69dbf5b)
• Fix PyTorch training split (900c78b)
• Clean inpainting code (ece1e8d)
• Move image assets in separate folder (#76)
• Download models on demand (#75)
• Added PapersWithCode badges

Feature release June 2020

Version 4.0 release of RT-GENE

1. Addition of RT-BENE (blink estimation). We were very lucky to win the best poster award at the ICCV2019 Workshop on Gaze Estimation in the Wild with this paper and now provide training + evaluation + ROS code, as well as the RT-BENE dataset.
2. In addition to our Tensorflow training+inference code, we now also provide a PyTorch version.
   2.1 This comes with a big clean-up of our ROS code which now allows for more modularity.
   2.2 Many thanks to @ahmed-alhindawi for this.
   2.3 It also includes experimental code to train/evaluate on the MPII dataset!
3. The ROS code now exposes more topics for easier read-out of the head pose, blink messages etc.
4. Our ROS code can now work with CPU only if required (although big disclaimer that this is slow)
5. We simplified the setup of the transforms that are required in the ROS code. Only the camera frame is required now in the launch file.
6. We fixed a bug where head rotations in roll-direction lead to incorrect extraction of the eye landmarks.
7. In addition to our academic-only code in this repository, we provide a version for commercial use. Please get in touch if you are interested!

Beta release for June 2020 version

RT-BENE model evaluation (#73)

* Add evaluation for RT-BENE

* Ignore some more models

* Add testing code description to README

Co-authored-by: Kevin Cortacero <[email protected]>

Feature Release 2019

We are very happy to announce version 3.0 of RT-GENE!

This release contains some major improvements over the previous version:

1. New head pose estimator
   1.1. RT-GENE now uses the 3DDFA landmark extractor as opposed to the face-alignment package because of the increased speed (note that we change the code so that Python 2 is still supported for use in ROS) - credits to @ngageorange
   1.2. There was a bug in the head pose estimation which is now fixed as well (see 9cde812 and 9fbbd39).
   1.3. Unstable head poses over time are fixed, too (see #30)
2. We refactored the code to disentangle ROS-independent code from ROS-specific code (#37)
   2.1. This allows us to provide an easy-to-use standalone (non-ROS) version of the code
   2.2. To facilitate this, we provide a helper script to download all required model files (this was done using ROS catkin scripts before)
   2.3. The ROS-specific code now makes use of tf2 rather than tf.
   2.4. The ROS version now properly installs the Python scripts (#45)
3. Inference speed improvements - massive thanks to @ngageorange (see #34)
   3.1. Inference in batches rather than per subject for landmark extraction as well as gaze estimation.
   3.2. Run ensembles in parallel, so that running the 4-model ensemble is now very fast, too.
4. Tensorflow 2.0 compatibility - we now use the tf.keras API instead of using keras directly and make sure that the code is compatible with both Tensorflow 1.14 as well as 2.0 (see #40).
5. We provide helper scripts to run RT-GENE from video files or ROS-bag files (9b9678d).
6. We prettified the README files and fixed the list of dependencies
7. The script's output is now in colour.

Breaking changes compared to RT-GENE v2.1:

These changes are all in the estimate_gaze.launch file; if you do not modify this, the new version should run straightaway. Please make sure to catkin build again if you use ROS after updating to v3.0.

1. rename rgb_frame_id to ros_tf_frame in estimate_gaze.launch file, the transform publisher is now in start_webcam.launch. The frame itself is now called kinect2_ros_frame rather than kinect2_nonrotated_link.
2. interpupillary_distance is now set via dynamic_reconfigure rather than in the estimate_gaze.launch file
3. SequentialTracker is deprecated; to achieve a similar behavior use FaceEncodingTracker with a large face_encoding_threshold (see estimate_gaze.launch file)

Feature release

This was a beta release - please refer to v3.0-final instead.

Bugfix release

This release captures some smaller fixes that should not be breaking user code, before a new major release in the next few days.

Bugfixes include:

• The tracking feature (keeping the same ID for a subject over time) now uses face tracking (FaceEncodingTracker) by default (#32). This leads to more consistent IDs, and allows keeping the same ID if a subject goes out of the view and returns later. The old SequentialTracker has been deprecated and will be removed in one of the next releases.
• The speed of FaceEncodingTracker has been improved considerably by using several threads (#36).
• Some latency problems between the different ROS nodes have been resolved (2f15196).
• The head pose estimation sometimes "goes crazy" (becomes unstable), and the likelihood of this occuring was reduced in #30.

Big shout to @ngageorange for lots and lots of input for this release.

Bugfix and feature release

• Added option to run different models (for face extraction, gaze estimation etc.) on different GPUs or the CPU
• Fixed visualisation of headpose
• Encourage using conda in readme
• Better tracking algorithm to keep same ID over time, thanks to @twarz and @ngageorange
• Python 3 compatibility (thanks @ngageorange)
• Some smaller fixes

Public release

This is the first release after making RT-GENE available to the public under the CC BY-NC-SA 4.0 license.

The One That Actually Works

This is the first release candidate following a flurry of improvements to the head-pose estimation network to make rt_gene more accurate and reliable.