Improve point trajectory estimation by aggregating across sources

[sfmig]

The simplest case may be centroid tracking, for example obtained with idtracker and SLEAP independently.

The question is: can we combine these two (or more) sources of information to produce a more accurate trajectory for the centroid?

This may be more relevant for a multi-animal case in which there are id swaps, but the two methods to get the data may fail in different cases.

Some options could be: to use the most reliable one to correct the other (for a given definition of reliable), take some kind of consensus, consider sensor fusion approaches like Kalman filter, or taking the rolling median filter using both sources of data. Another approach could be taking the mean of both sources (so somewhat related to this #271).

Some sample data that we could use for this is the EPM:

• DLC_single-mouse_EPM.predictions.csv (or the equiv .h5 file) as the "more reliable" source, and
• SLEAP_single-mouse_EPM.analysis.h5 (or the equivalent .slp file), which is generated using a model trained on less data (should be less reliable).
• They differ in the number of keypoints so we could using the intersection set, or the centroid for this analysis.

[roaldarbol]

For reference, trex can output "tracklets" or each individual: Small egocentric videos of each individual (centered on their centroid) which can then be tracked using SLEAP or DLC. Haven't tried it myself though.

[sfmig]

thanks @roaldarbol for pointing this out, idtracker can do this too (see the tip box here)

[roaldarbol]

Ah, so cool, didn't know they did it too!!!