Add ability to track progress/status of license detection review

[johnmhoran]

The License Detections Explorer is very useful and a pleasure to work with. I've noticed that the scrollable list of detections on the left can be quite long depending on the data involved, and I've seen cases where the list contained over 400 detections that needed to be vetted. This task could be spread over several days or longer, and also over 1+ reboots or similar workflow interruptions.

It would be handy if we could add some way for a user to keep track of which detections she or he had already reviewed, e.g., a checkbox -- this would enable the user to check a vetted detection and move on to those detections not yet reviewed.

In addition, perhaps we could add the ability to filter the list of detections to display, e.g., all vetted or all un-vetted detections.

[OmkarPh]

I can think of 3 ways to store this set of reviewed licenses

• we can store this in the sqlite file, with other data, but the next time user opens the workbench, he should open the same sqlite file (opening the JSON file won't retain that)
• Store scan-specific review status in local storage of the desktop app itself (key could be the imported file name?)
• Edit the scan JSON file itself, but ig that's a bad idea

@pombredanne @AyanSinhaMahapatra your thoughts on this?

[AyanSinhaMahapatra]

So I'm assuming that you're storing an extra field for each license detection object: is_reviewed or something similar, then:

Edit the scan JSON file itself, but ig that's a bad idea

Yup, this is not doable.

Store scan-specific review status in local storage of the desktop app itself (key could be the imported file name?)

This file will become really big then, as this is per license detection. And if we keep doing this for multiple scans, it will keep growing.

we can store this in the sqlite file, with other data, but the next time user opens the workbench, he should open the same sqlite file (opening the JSON file won't retain that)

I think this makes sense, if we're working on the same scan, we anyway use the .sqlite to import.

[mjherzog]

I realize that this will not help with the near-term issues, but I am skeptical about adding this feature to SCTK. It should be in SCIO where the data storage and sharing will be much cleaner. We have been planning to add a SCWB-like UI to SCIO so perhaps we could start small in SCIO with a License Detections Explorer mini-app. This would also require that we finalize the database design for curations.

[OmkarPh]

So I'm assuming that you're storing an extra field for each license detection object: is_reviewed or something similar, then:

yep, that's right

Store scan-specific review status in local storage of the desktop app itself (key could be the imported file name?)

This file will become really big then, as this is per license detection. And if we keep doing this for multiple scans, it will keep growing.

yeah, it will keep on piling all the data, and since we can't decide when user won't need a particular scan anymore there's no way to clear unused data

we can store this in the sqlite file, with other data, but the next time user opens the workbench, he should open the same sqlite file (opening the JSON file won't retain that)

I think this makes sense, if we're working on the same scan, we anyway use the .sqlite to import.

yep, this looks good

[OmkarPh]

Also, in the history section, if we import a previously imported JSON file, it parses the JSON again
I think we should change that to opening the generated SQLite instead
(For development mode, we can keep it as it is so we can test the application conveniently)

[OmkarPh]

I've created a PR for this

What should be ideal filter labels for this?

• All / Vetted / Unvetted
• All / Reviewed / Unreviewed

On weekly status call, @AyanSinhaMahapatra @JonoYang suggested having All / Reviewed / Unreviewed

[mjherzog]

I agree with the review terminology - in American English "vetting" is usually associated with investigating a person's background

[AyanSinhaMahapatra]

@johnmhoran @mjherzog do check out how the Progress Tracking feature works now at #610 (review) (And also highlighting potential issues to simplify the review process)