Possible feature request: unsupervised individual recognition

[aperkes]

Hi SLEAPers,
SLEAP is doing an amazing job tracking my fish, but for an upcoming experiment I will need to also have individual IDs. I'm considering adapting the methods used for idTracker & Trex and build a little classifier to identify individuals, training it from the inferred tracks. I only need to distinguish ~5 fish, which will likely have identifying tattoos, so in theory it shouldn't be that challenging to learn, but before getting into it, I thought I should check if this is something you are already thinking about and/or have good solutions for.

Thanks for your work!

[talmo]

Hi @aperkes,

Thanks for reaching out! Yes, that is something we're actively working on actually. There's a lot of details that make it a bit tricky, but we're hoping to have something that we can roll into SLEAP within the next few months.

Currently, we have experimental support for supervised identity tracking, but it's not quite the same as what idTracker can get you. I know that other SLEAP users have taken the exact approach you described of combining SLEAP with idTracker to resolve ID switches, but we don't have a good way to integrate it for the general case super seamlessly just yet.

If you have identifying tattoos that you can reliably identify during labeling time, you can definitely use the supervised identity tracker for now. If you'd like to reach out to [email protected], we can set up a time to chat or I can send you some instructions on how to get started.

Cheers,

Talmo

[talmo]

Moving this to the discussions, but other readers feel free to pitch in if you have ideas or specific needs.

Unsupervised ID recognition is an important research goal for multi-object tracking and we will be developing approaches that integrate it in the future.