Merge pull request #86 from funkelab/refactor_for_editing

Add editing and refactor entire code base

docs/source/editing.rst

@@ -0,0 +1,73 @@
+Editing Tracks
+==============
+
+Usage Overview
+**************
+Predicted tracks can be corrected by deleting, adding, or modifying nodes and/or edges, using the buttons in the 'Edit Tracks' widget or their corresponding keyboard shortcuts, or by editing the napari Points and Segmentation layers directly.
+
+Editing nodes
+*************
+
+.. _delete-node:
+
+Deleting nodes
+---------------
+Nodes can be deleted by selecting one or multiple nodes and clicking the 'Delete' button in the Edit Tracks menu or pressing ``D`` or ``Delete`` on the keyboard.
+Deletion of a node results in its removal from the tree view and removal of its corresponding point and segmentation label in the Points and Segmentation napari layers.
+If the node was connected to a predecessor and a successor, a new skip edge will be formed between the predecessor and successor nodes, leaving the remaining track intact.
+If one of the two children of a dividing node is deleted, the nodes of the remaining sibling are relabeled to match the track ID of the parent.
+
+.. figure:: images/delete_node.png
+   :width: 600px
+   :align: center
+
+   Examples of deleting nodes. Deletion of a node in a linear track will create a skip edge (top), whereas deletion of one of the two children (green) of a dividing node will relabel the nodes of the remaining sibling (yellow), which is now part of the parent track (grey) (bottom).
+
+.. _add-node:
+
+Adding nodes
+-------------
+New nodes can be added in two ways:
+    - by adding a new point in the Points layer. A new, non-connected endpoint node will be created at the clicked position. This node does not have a segmentation (yet), but this can be added by the user if needed.
+    - by painting with a (new) label in the Segmentation layer. A new node will be created based on the painted pixels. If the label already exists in the Segmentation layer, it will automatically be connected to its predecessor and/or successor via (skip) edges. If the label does not yet exist, a new endpoint node is created.
+
+.. figure:: images/add_node.png
+   :width: 600px
+   :align: center
+
+   Add a new node by painting. Press ``M`` on the segmentation layer to select a new label color for painting.
+
+Updating nodes
+---------------
+Node attributes (e.g. size, position) can be updated in two ways:
+    - If the source layer for tracking was a Points layer and no segmentation was provided, the points can be repositioned by clicking the 'Select points' button in the Points layer menu and clicking and dragging points to their new location.
+    - If a segmentation layer was provided, node position is determined by the centroid location of each label. Therefore, nodes cannot be repositioned by moving their corresponding points in the Points layer. Instead, nodes can be updated by painting and/or erasing their labels in the Segmentation layer, which will automatically update their position and size properties.
+
+Editing edges
+*************
+
+Deleting edges
+----------------
+Edges can be deleted by selecting two connected nodes and clicking the 'Break' button in the Edit Tracks menu or by pressing ``B`` on the keyboard. In a linear track, deleting an edge will split the track in two fragments. The first fragment will retain the track ID of the source node,
+while a new track ID is assigned to the fragment of the target node.
+
+.. figure:: images/break_edge.png
+   :width: 600px
+   :align: center
+
+   Examples of breaking edges. In a linear track, breaking an edge will relabel the fragment of the target node (top). If an edge between a dividing node and one of its children is deleted, both fragments maintain their track ID but the remaining sibling (magenta) is relabeled since it is now part of the same track as the parent (cyan) (bottom).
+
+Adding an edge
+--------------
+New edges can be added by selecting two non-horizontal nodes and clicking on the 'Add' button in the Edit Tracks menu or by pressing ``A`` on the keyboard. If the target node already has an incoming edge, the user will be prompted with the question whether the existing edge can be broken,
+since a node in a lineage tree cannot have two incoming edges. Note that new edges are also added automatically in certain cases when a node is being added or removed (see :ref:`delete-node` and :ref:`add-node`).
+
+.. figure:: images/add_edge.png
+   :width: 600px
+   :align: center
+
+   Adding an edge between two nodes, creating a new division point.
+
+Undoing and redoing actions
+***************************
+All types of actions described above are appended to the Action History, and can be undone or redone. To undo, click 'Undo' in the Edit Tracks menu or by pressing ``Z``. Similarly, to redo an action, press 'Redo' in the Edit Tracks menu or ``R``.

docs/source/getting_started.rst

@@ -12,7 +12,8 @@ This plugin depends on ``ilpy``, which must be installed via conda.::
 
 If this is successful, you can then run ``napari`` from your command line, and
 the motile plugin should be visible in the ``Plugins`` drop down menu.
-Clicking this should open the motile widget on the right of the viewer.
+Clicking the Main Motile Widget should open the menu widget on the right of the viewer,
+and a lineage tree view in the bottom of the viewer.
 
 Tutorial video
 **************
@@ -30,103 +31,57 @@ The motile plugin does not perform detection: you must provide a Labels layer or
 containing the objects you want to track.
 The Labels layer must have time as the
 first dimension followed by the spatial dimensions (no channels).
-You can load an example Labels layer of a 2D HeLa dataset from the `Cell Tracking Challenge`_
-in ``Files`` -> ``Sample Data`` -> ``Fluo-N2DL-HeLa (Motile)``.
 The Points layer must have the point locations with time as the first number,
 followed by the spatial dimensions. While source images are
 nice to qualitatively evaluate results, they are not necessary to run tracking.
 
+There are two example datasets provided in ``File`` -> ``Open Sample`` -> ``Motile``.
+A 2D HeLa dataset from the `Cell Tracking Challenge`_ is provided, both in full and a cropped subset for testing features on smaller data, and has both a Labels layer and Points layer.
+There is also a 3D dataset of images and segmentations of a membrane-labeled developing early mouse embryo (4-26 cells)
+from `Fabrèges et al (2024)`_, automatically downloaded from `zenodo`_.
+
 In the future, we could also support Shapes layers as input (for example,
 for bounding box tracking) - please react to
 `Issue #48`_ if this is important to your use case, and give feedback on what type
 of shape linking you want.
 
+Plugin widgets
+**************
+You can open the Main Motile Widget via ``Plugins`` -> ``Motile`` -> ``Motile Main Widget``.
+This will open a tab widget containing a ``Motile``, ``Edit Tracks``, and ``Results List`` widget, as well
+as an empty ``Lineage View`` widget at the bottom of the screen. Alternatively, you can open the Tree View
+and the Menus widgets individually under ``Plugins`` -> ``Motile`` and dock them manually.
+
 Running tracking
 ****************
-The motile plugin by default opens to the ``Run Editor`` view. In this view,
+The ``Motile`` tab by default opens to the ``Run Editor`` view. In this view,
 you can pick a name for your run, select an input layer, set
 hyperparameters, and start a motile run. Hovering over the title of each
 element in the widget will make a tooltip appear describing the purpose
-of the element. All hyperparameters are explained in more detail below.
+of the element. All hyperparameters are explained in the :doc:`tracking with motile <motile>` docs page.
 When you are ready, click the ``Run Tracking`` button to start tracking.
 
-Hyperparameters
----------------
-The hyperparameters set up constraints on the optimization problem.
-These constraints will never be violated by the solution.
-
-- ``Max Move Distance`` - The maximum distance an object center can move between time frames, in pixels. This should be an upper bound, as nothing further will be connected.
-- ``Max Children`` - The maximum number of objects in the next time frames that an object can be linked to. Set this to 1 for tracking problems without divisions.
-
-Constant Costs
---------------
-The constant costs contribute to the objective function that the optimization
-problem will minimize. They are the same for all possible links in the
-tracking problem, and should be set based on general properties of the problem.
-Unchecking the checkbox will not include the cost in the objective at all,
-while changing the weight values will change how much each cost contributes
-to the objective.
-
-- ``Edge Selection`` - A cost for linking any two objects between time frames. If you do not have many false positive detections, this value should be quite negative to encourage selecting as many linking edges as possible. If none of your costs are negative, the objective function will be minimized by selecting nothing (which has cost 0), so this cost can control generally how many edges/links are selected.
-- ``Appear`` - A cost for starting a new track. Assuming tracks should be long and continuous, the appear cost should be positive. A high appear cost encourages continuing tracks where possible, but can lead to not selecting short tracks at all.
-- ``Division`` - A cost for dividing, where a higher division cost will lead to fewer divisions. If your task does not have divisions, this cost does not matter - you can un-check it for clarity, but including it will do nothing.
-
-Attribute weights
------------------
-The attribute-based costs also contribute to the objective function that
-the optimizer will minimize. Every possible link in the tracking problem
-will have a different cost based on the specific attributes (distance or IoU)
-of that pair of detections. Unchecking the checkbox will not include the
-feature in the objective. The user-provided weight values will be multiplied by
-the attribute values to generate the cost for linking a specific pair of
-detections.
-
-- ``Distance`` - Use the distance between objects as a feature for linking. The provided weight will be multiplied by the distance between objects to get the cost for linking the two objects. This weight should usually be positive, so that higher distances are more costly.
-- ``IoU`` - Use the Intersection over Union between objects as a feature for linking. This option only applies if your input is a segmentation. The provided weights will be multiplied by the IoU to get the cost for linking two objects. This weight should usually be negative, so that higher IoUs are more likely to be linked.
-
-
-Viewing run results
-*******************
-Clicking the ``Run Tracking`` button will automatically take you to the ``Run Viewer``.
-It contains the following:
-
-- The ``Solver status`` label, which will display if the solver is in progress or
-  done solving.
-- The ``Graph of solver gap``, which is mostly for debugging purposes.
-  The solver gap is an optimization value that should decrease at each iteration.
-- The run settings, including ``Hyperparameters``, ``Costs``, and ``Attribute weights``.
-- The ``Save run`` button. This button will take you to a file dialog to save the
-  whole run, so that if you close napari and re-open it, you can load the run
-  and see the results.
-- The ``Export tracks to CSV`` button, which will take you to a file dialog for saving
-  a csv file containing the tracks. If your input was a Labels layer, the
-  ``node_id`` will be determined by the time and the original segmentation label id.
-  If your input was a Points layer, the ``node_id`` is simply the index of the
-  node in the list of points.
-    - Note: This does not save the output segmentation. If you want to save
-    the relabeled segmentation, you can do so through napari by selecting the
-    layer and then selecting ``File``-> ``Save selected layers``
-- The ``Back to editing`` button, which will return you to the ``Run Editor`` in its
-  previous state.
-- The ``Edit this run`` button. This button will take you back to the ``Run Editor``,
-  but will overwrite the previous settings with the settings of the run you are
-  viewing.
-
-Once the solver completes, you will also see a tracks layer
-in the napari viewer. If your input was a segmentation, there will also be
-a new segmentation layer where the IDs have been relabeled to match across time.
-
-A list of ``Tracking Runs`` will also appear at the bottom of the widget.
+Viewing and editing run results
+*******************************
+Clicking the ``Run Tracking`` button will automatically take you to the motile ``Run Viewer``
+menu, display a points and a tracks layer in the napari viewer, and populate the Lineage Tree view. If your input was a segmentation, there will also be
+a new segmentation layer where the IDs have been relabeled to match across time, and the input segmentation layer will be hidden to avoid confusion.
+
+You can :doc:`view the results <tree_view>` using the synchronized napari layers and tree view, and :doc:`edit the detections and links <editing>` to correct any mistakes that you find. You can also re-run the tracking step with different parameters. Re-running the motile tracking will only take into account the detection corrections
+if you select the new labels/points layer as input: our next major feature to add
+is incorporating the detection and linking corrections into the optimization task in a more principled manner.
+
+Each ``Tracking Run`` will be stored in the ``Results List`` widget.
 These are the runs that are stored in memory - if you run tracking multiple
 times with different inputs or parameters, you can click back and forth
-between the results here. Deleting runs you do not want to keep viewing
-is a good idea, since these are stored in memory. Runs that were saved in
-previous sessions do not appear here until you load them from disk with the
-``Load Run`` button.
-
+between the results here. Here you can also save any runs that you want to store for later.
+Deleting runs you do not want to keep viewing is a good idea, since these are stored in memory.
+Runs that were saved in previous sessions do not appear here until you load them from disk with the ``Load Tracks`` button.
 The tracking results can also be visualized as a lineage tree.
 You can open the lineage tree widget via ``Plugins`` > ``Motile`` > ``Lineage View``.
 For more details, go to the :doc:`Tree View <tree_view>` documentation.
 
 .. _Issue #48: https://github.com/funkelab/motile_napari_plugin/issues/48
 .. _Cell Tracking Challenge: https://celltrackingchallenge.net/
+.. _Fabrèges et al (2024): https://www.science.org/doi/10.1126/science.adh1145
+.. _zenodo: https://zenodo.org/records/13903500

docs/source/index.rst

@@ -21,8 +21,10 @@ See the `motile documentation`_ for more details on the concepts and method.
    :maxdepth: 3
 
    getting_started
-   key_bindings
+   motile
    tree_view
+   editing
+   key_bindings
 
 .. _motile: https://github.com/funkelab/motile
 .. _github link: https://github.com/funkelab/motile_napari_plugin

docs/source/key_bindings.rst

@@ -13,10 +13,10 @@ Napari viewer and layer key bindings and mouse functions
    * - Click on a point or label
      - Select this node (center view if necessary)
    * - SHIFT + click on point or label
-     - Sdd this node to selection
+     - Add this node to selection
    * - Mouse drag with point layer selection tool active
      - Select multiple nodes at once
-   * - T
+   * - Q
      - | Toggle between viewing all nodes in the
        | points/labels or only those for the currently
        | selected lineages
@@ -35,20 +35,20 @@ Tree view key and mouse functions
      - Add this node to selection
    * - Scroll
      - Zoom in or out
-   * - Scroll + X
+   * - Scroll + X / Right mouse click + drag horizontally
      - Restrict zoom to the x-axis of the tree view
-   * - Scroll + Y
+   * - Scroll + Y / Right mouse click + drag vertically
      - Restrict zoom to the y-axis of the tree view
    * - Mouse drag
      - Pan
    * - SHIFT + Mouse drag
-     - Rectangular zoom
+     - Rectangular selection of nodes
    * - Right mouse click
      - Reset view
-   * - L
+   * - Q
      - | Switch between viewing all lineages (vertically)\
        | or the currently selected lineages (horizontally)
-   * - A
+   * - W
      - | Switch between plotting the lineage tree and the
        | object size
    * - Left arrow
@@ -64,12 +64,32 @@ Tree view key and mouse functions
        | lineages) or the previous adjacent lineage
        | (horizontal view of selected lineage)
 
+Key bindings for editing the tracks
+***********************************
+.. list-table::
+   :widths: 25 25
+   :header-rows: 1
+
+   * - Mouse / Key binding
+     - Action
+   * - D
+     - Delete selected nodes
+   * - B
+     - Break edge between two selected nodes, if existing
+   * - A
+     - Create edge between two selected nodes, if valid
+   * - Z
+     - Undo last editing action
+   * - R
+     - Redo last editing action
+
+
 Key bindings tutorial video
 ***************************
-This `video`_ shows how to use the different key bindings in the napari layers and the tree view.
+This `video`_ shows how to use the different mouse and key functions, as well as their corresponding buttons, in the napari layers and the Tree View.
 
 .. raw:: html
 
-  <iframe src="https://drive.google.com/file/d/1oYgH6PjimofYr4o7HbSTRPMr6mtVE1S5/preview" width="640" height="480" allow="autoplay"></iframe>
+  <iframe src="https://drive.google.com/file/d/1cv5FbYqc5RbkNbh0YyWAL64A3tm-ugTs/preview" width="640" height="480" allow="autoplay"></iframe>
 
-.. _video: https://drive.google.com/file/d/1oYgH6PjimofYr4o7HbSTRPMr6mtVE1S5/preview
+.. _video: https://drive.google.com/file/d/1cv5FbYqc5RbkNbh0YyWAL64A3tm-ugTs/preview