Improvement : External tracker concept introduction

[BearrGod]

Hello everyone. I've been poking around the openvin's codebase and i've done some diagnostics at runtime through the debug mode. And i've realized that the feature tracking is by far the most time consuming part of the calculations ,which is fairly normal, especially when you consider high resolution images. I was wondering if it wouldn't be interesting to add an external tracker concept, especially for gpu and vpu enabled devices. And then instead of a full image we could provide a vector of tracked points directly as input. We could extend the CameraData class like this :

struct  TrackedPoint
{
  uint32_t 	age = 0 ;  //Number of frame we tracked for
  float 	harrisScore = 0.f ;  // Score of the tracked point
  uint32_t 	id = 0 ;  // Persistent id to track the point
  cv::Point2i 	position ;  // Position in image of the tracked point
 float 	trackingError = 0.f ;  // tracking error
};


/**
 * @brief Struct for a collection of camera measurements.
 *
 * For each image we have a camera id and timestamp that it occured at.
 * If there are multiple cameras we will treat it as pair-wise stereo tracking.
 */
struct CameraData {

  /// Timestamp of the reading
  double timestamp;

  /// Camera ids for each of the images collected
  std::vector<int> sensor_ids;

  /// Raw image we have collected for each camera
  std::vector<cv::Mat> images;

  /// Image sizes in case we decide to only work with trackers and not full images. 
  std::vector<cv::Size> image_sizes;

  /// Eventual tracking points revieved. These are tracked point from last frames to this frame. 
  std::vector<std::vector<TrackedPoint>> tracked_points ; 

  /// Tracking masks for each camera we have
  std::vector<cv::Mat> masks;

  /// Sort function to allow for using of STL containers
  bool operator<(const CameraData &other) const {
    if (timestamp == other.timestamp) {
      int id = *std::min_element(sensor_ids.begin(), sensor_ids.end());
      int id_other = *std::min_element(other.sensor_ids.begin(), other.sensor_ids.end());
      return id < id_other;
    } else {
      return timestamp < other.timestamp;
    }
  }
};

I was wondering how to create a TrackExternal class which obey to the TrackBase schemas to be able to do this.
Thanks a lot.

[goldbattle]

Yes, I think if you extend the TrackBase and this external tracker handles both detection and match, then this can be done.
https://docs.openvins.com/classov__core_1_1TrackBase.html