Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve merging and iterative merging #3487

Open
wants to merge 90 commits into
base: main
Choose a base branch
from

Conversation

yger
Copy link
Collaborator

@yger yger commented Oct 18, 2024

This branch depends on #3435 , and adds the possibility to benchmark only meta merging, i.e. the final steps of a sorting procedure. Such step is crucial in the context of drift-corrected recordings. Note that it will replace #2934 that will be closed, since now the code is easier to read, cleaner, and more in line with @samuelgarcia point of view. It can still be discussed however, to know if we should do a proper merging step for the sortingcomponents. I think we should, because several options could be tried and benchmark.

Functions to split by times and/or amplitudes have been rewritten without any dependencies, in a separate file as requested in #2934 (splitting_tools.py in generation)

merging_methods = ["circus", "auto_merges"]


def create_sorting_analyzer_with_templates(sorting, recording, templates, remove_empty=True):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is a need we should have this function somewhere else

return sa


def merging_circus(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move this to sc2

)


def merge_spikes(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use only the
merged_analyzer = auto_merge(analyzer, ...)
which I have push in #3435 because it do the same.
The final sorting is accessed by merged_analyzer.sorting.

@yger
Copy link
Collaborator Author

yger commented Oct 30, 2024

This one is also almost good after the discussion we had yesterday @samuelgarcia @alejoe91. Now the auto_merge_units() function can handle lists of presets, and the benchmark is also there and working. Some questions:

  • are all default params optimal for extension (similarity uses cosine, no lag for example)
  • should we add to auto_merge_units the notion of "strategies", i.e. for example if someone has a good list of presets and want to use it?

pre-commit-ci bot and others added 28 commits October 30, 2024 08:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
curation Related to curation module sortingcomponents Related to sortingcomponents module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants