Support filtering of pipelines

[noklam]

Description

https://demo.kedro.org/?types=nodes,datasets&pid=__default__&expandAllPipelines=false
support filtering via the pannel, this is missing in the extension and several user has mentioned this make most project unusable.

I recalled @astrojuanlu mentioned somewhere, base on our telemetry this select filter has very low usage.

Context

This make the viz integration unusable for realistic pipeline (100+ nodes). See notes #140

Alternative

• Reuse the component in viz web app.
• Add a new way of filtering in extension via command
• Launch viz with options to select sub-pipelines (most inflexible but maybe easiest as adhoc solution)

[astrojuanlu]

base on our telemetry this select filter has very low usage.

Based on anecdotal evidence 😅 we haven't analysed the telemetry yet

[stephkaiser]

(The summary below tackles both #123 & #136)

Summary

• We discussed multiple ideas for both tickets (Support filtering of pipelines #123 & Collapse namespace pipeline #136) - we discussed pros and cons for users, ideal UX experience, as well as technical feasibility.
• We agreed its important to consider a keyboard approach (rather than clicking) as primary users of VS Code mostly prefer a keyboard-friendly method.
• The least-preferred ideas can be found on our Miro board - https://miro.com/app/board/uXjVLPLhXgs=/?share_link_id=17393556673

Our chosen solution:

• We decided on a combined solution of both UI + Keyboard approaches - we think adding the actions bar with a filter button and the expand/collapse button would help users discover these functionalities as its a familiar component in Kedro-Viz. As a first version, clicking on the filter icon will open the command dropdown showing the new filter command and when selected, is pre-populated with filter options (showing available sub-pipelines to filter). We can reuse the actions bar component we have in Kedro-Viz but remove the export and minimise panel buttons.
• The command to filter Viz by sub-pipelines can also be triggered via keyboard input (with autocompletion) without clicking on the filter button. The button is there to aid discovery of the command functionality. Since the behaviour of this filter button is different to the other buttons in the utility bar, there is a divider line to separate this action.
• I think its not necessary to add a new command for expanding/collapsing all pipelines, this can easily be done using the button to toggle, and its the same consistent behaviour as the other buttons in the utility bar.

Designs can be found here - https://www.figma.com/design/3kSpvIO1veLKfy9qHxuXwF/Kedro-WIP?node-id=2936-125807&t=Km8oiXjUlRyVBUMk-4

[astrojuanlu]

Thanks a lot for the summary @stephkaiser . One thing I don't understand - is the idea that, after clicking the filter button, the VS Code Command Palette would open with the value >kedro pre-populated?

For the UI, have we explored the possibility of using the left menus?

[noklam]

Base on the last discussion, I learn that the interaction between that sidebar and the visualisation is one of the most complicated part in Viz, so it is a bigger engineering challenge and will take more time. I assume you want to use the sidebar with native VSCode component? Or do you mean re-using the kedro-viz react sidebar?

On the other hand, having a command -> trigger a drop down does not conflict with this, since it enable a keyboard workflow with shortcut, and the implementation is clear (can be achieved in a relatively short time).

On design side @stephkaiser will have more comment on this.

[stephkaiser]

Thanks a lot for the summary @stephkaiser . One thing I don't understand - is the idea that, after clicking the filter button, the VS Code Command Palette would open with the value >kedro pre-populated?

Yes thats the idea, it will open with the filter command pre-populated so for example: >kedro: Filter Viz by pipeline

For the UI, have we explored the possibility of using the left menus?

Yes we have, you can see all the other ideas in the Miro board link above. The TL;DR is this solution wasn't the ideal user experience because of its positioning on the left side - this list panel is related to viz flowchart but code is separating both of these kedro-viz components so interacting and selecting something in the left side and viewing it on the right side felt disjointed. Interacting with the list panel also won't affect the code section next to it, only the viz flowchart. This list component is also complex to implement from an engineering perspective.

[astrojuanlu]

Oh I hadn't seen the Miro, only Figma 👍🏼 thank you for the extra context!

[stephkaiser]

no problem! 🙂 keep in mind this is the v1 proposal that we thought was relatively low effort and easy to implement. When we get more feedback and ideas to explore for the kedro viz extension, we can revisit the list menu idea on the left

[astrojuanlu]

We have to create specific tickets cc @jitu5