Sequence Templates #1642

JosephVolosin · 2025-03-10T17:51:09Z

___REQUIRES_AERIE_PR___="1638"

This PR implements the UI for sequence templates (see aerie PR for more information)

New Environment Variable

This feature introduces a new environment variable called PUBLIC_SEQUENCING_MODE which is set to either templating or expansion depending on the feature the user wants. Command Expansion and Sequence Templating are two ways to get the same outputs, so the user must decide when they build their instance which they'd like to use.

Sequence Template Editor

To create/edit/delete sequence templates, there is a new route /sequence-templates that brings the user to an editor based on the sequence editor in /sequencing.

The editor window utilizes CodeMirror and implements an 'overlay' language to support Handlebars formatting. Additionally, the same parcel/dictionary capabilities from the sequence editor are available for linting, formatting, auto-complete, etc.

Plan View - Sequences & Expansion

Within the Plan view, there is a new tab called Sequences & Expansion which is used to create sequences and expand them using either command expansion or sequence templating. The functionality from the previous command expansion panel has been rolled into this panel. New sequences and sequence filters can be created through the New button next to the table filter.

The table shows both expansion sequences and sequence filters. Expansion sequences can be deleted, expanded (using command expansion or sequence templating), and have the expanded output viewed if it exists.

Sequence filters can be deleted, modified after creation, and applied. 'Applying' a sequence filter generates a new expansion sequence with spans that match the filter criteria automatically attached to the sequence ID.

Step-by-step guide

Upload/Create a: Model, Plan, Dictionary, and Parcel
Navigate to Sequence Templates (/sequence-templates) through the Aerie logo in the top-left
Click New Template and enter a 1) Template Name, 2) Template Language (STOL, SEQN, or TEXT), 3) Parcel Id, 4) Model Id, 5) Activity Type. Click Create Template
Click the template in the table and then write expansion instructions in the center panel (ex: /C FSW_CMD_0 "ON" false 1). Click Save or use Ctrl+S to save the template.
Navigate to a plan through the Aerie logo in the top-left & selecting a plan from the table
Add at least one activity directive to the timeline and run a simulation
Change one of the UI panels to the Sequences & Expansion panel
Click New and create a new sequence filter. In the filter builder modal, enter a name and at least one filter constraint that will capture the previously added activity directive
Hover over the sequence filter in the table and click the Create Sequence button that appears, select a time range in the modal that appears that includes the previously added activity directive (or, leave it as the default - the plan time range)
Hover over the expansion sequence that is generated and click Expand Sequence
Hover over the same expansion sequence and click Show Expanded Sequence

.gitignore

package.json

src/stores/sequencing.ts

mattdailis · 2025-03-10T22:46:58Z

src/components/expansion/SequencesAndExpansionPanel.svelte

+    </ActivityFilterBuilder>
+    <div class="sne-controls">
+      <div class="sne-filter">
+        <input


This input gets out of sync with the input in the filter builder modal. E.g.:

Fill in the text box next to the new filter button

Make a filter, but don't give it a name

Save the filter

The created filter doesn't have the name from step 1, and the text box is still populated.

I think what I expect is that the text box should clear after I click "new". If I'm creating a sequence filter, that value should be used to seed the name of the filter in the text box in the modal.

I may have misunderstood the intent of that text box - it says "Filter text", but I guess it means "filter the items in this list so I can find what I'm looking for" rather than "this is the name of the SequenceFilter"

Ha, ah yeah that is confusing now that I think about it.. that box (which is unfortunately next to that 'New' button) is meant to filter the results in the table underneath. The name of the sequence filter you're creating or viewing is just bound to that entry in the filter builder. Maybe we should update the text in the filter box to say 'Filter items...'?

duranb · 2025-03-12T00:18:42Z

src/types/sequencing.ts

@@ -22,6 +22,18 @@ import type { SequenceTypes } from '../enums/sequencing';
 import type { ArgDelegator } from '../utilities/sequence-editor/extension-points';
 import type { UserId } from './app';
 import type { GlobalType } from './global-type';
+import type { ActivityLayerFilter } from './timeline';
+
+export type SequenceActivityFilter = ActivityLayerFilter;


Would it be correct to say that this is the same as an ActivityLayerFilter? The purpose of the ActivityLayerFilter is more geared towards activities.

Yep - I made SequenceActivityFilter assuming that it would be somewhat different from ActivityLayerFilter, but that didn't really end up being the case so I've re-factored everything back to using ActivityLayerFilter.

Are you using the SequenceActivityFilters in the timeline somewhere? It looks like you're utilizing the "filter" part of the ActivityLayerFilter. If it's not really a timeline feature, then I think it would be less confusing to maybe rename and rework ActivityLayerFilter and its surrounding types to something more reusable for other features by using generic types.

duranb · 2025-03-12T00:21:38Z

src/components/expansion/SequencesAndExpansionPanel.svelte

+      <div class="sne-buttons">
+        <button
+          class="st-button secondary new-button"
+          style="position: relative; z-index: 1"


Do you mind avoiding static inline styles? It makes it difficult to maintain when the styles are spread out throughout the component.

Removed this since I don't believe it's needed!

src/components/timeline/form/TimelineEditor/ActivityFilterBuilder.svelte

Another approach to do the same as the last commit - some issues w. displaying the ActivityFilterBuilder from within the modal

…ing actions

…e generation

Debugging a naive overlay approach to integrating handlebars into the editor.

Fix Seq-language autocompletion to work outside, and only outside, of template insert fields.

Doesn't include any back-end support, but this does stub out the back end as a store that should be easy-ish to replace with a real version.

pranav-super

A few minor comments and questions. Everything looks fine otherwise!

pranav-super · 2025-03-18T16:40:53Z

.env

@@ -7,5 +7,6 @@ PUBLIC_HASURA_SERVER_URL=http://localhost:8080/v1/graphql
 PUBLIC_HASURA_WEB_SOCKET_URL=ws://localhost:8080/v1/graphql
 PUBLIC_AUTH_SSO_ENABLED=false
 PUBLIC_TIME_PLUGIN_ENABLED=false
+PUBLIC_SEQUENCING_MODE=templating


Whether this is the default mode is something being deliberated on AERIE main (see here).

Looks like we settled this over there and Matt updated it here!

pranav-super · 2025-03-18T18:26:59Z

src/components/menus/AppMenu.svelte

-    <MenuItem on:click={() => showAboutModal()}>
+    <MenuItem
+      on:click={e => {
+        if (e.detail.shiftKey) {


Should we keep this switch on? I think having this functionality available somewhere in the UI is helpful, but it seemed like from an earlier conversation, the decision was to delegate this to a configuration setting and to leave it at that. Is this something we mean to leave in?

We had a long conversation about this and decided the best way to do it, for now, is through a hidden switch in the UI like this. Hopefully though in the future we'll be able to have a better way to allow the E2E tests to get to both Typescript & Templating sides of expansion

pranav-super · 2025-03-18T18:34:01Z

src/components/modals/ExpansionSequenceModal.svelte

+    outputStr = expandedTemplate?.expanded_template ?? `No output found for sequence "${expansionSequence.seq_id}"'`;
+  } else {
+    effects
+      .getExpansionSequenceSeqJson(expansionSequence.seq_id, expansionSequence.simulation_dataset_id, user)


Question to @mattdailis and @JosephVolosin:

This makes use of the expanded_sequences table, which makes sense - have usages of the getSequenceSeqJson endpoint been fully phased out in the UI at this point? It seems like we can do away with that endpoint entirely, especially if the UI has demonstrated that it can be replaced with a call to the expanded_sequences.

What is the reason we're moving away from getSequenceSeqJson? FWIW, pulling directly from expanded_sequences (as defined currently) has the unexpected result that if you've re-run expansion within the same simulation dataset, you end up with an array of results. We could filter that down to only provide the latest one via. the creation_time field, but just want to make sure we really do want to swap this functionality

Talked w. Pranav and Matt on this - querying expanded_sequences is the correct way to go forward for this PR, and we are OK grabbing the latest expansion based on creation time. Updated on the UI!

pranav-super · 2025-03-18T19:21:54Z

src/components/expansion/SequencesAndExpansionPanel.svelte

+                  <FilterIcon />
+                </button>
+              </div>
+              <div use:tooltip={{ content: 'Apply Filter', placement: 'top' }}>


This button doesn't do anything if the plan is empty. So creating a filter and then a sequence from that filter with an empty plan, before adding activities, is not possible.

That shouldn't be an issue but might be worth noting, especially if filter reapplication and such become desired behavior later?

Fair question - in general, I'm not 100% certain how we want to handle situations where the action will 'fail' because there is no initial simulation

pranav-super · 2025-03-18T19:23:32Z

src/components/modals/TimeRangeModal.svelte

+</script>
+
+<Modal {height} {width}>
+  <ModalHeader on:close>Apply Filter To Range</ModalHeader>


Maybe this should have a more intuitive title like "Create Sequence from Filter", and then "Apply Filter to Range" could be the body text? Just to make it clear that in applying a filter we are creating a sequence?

Agreed, I swapped the header content around and added some text under:

cartermak

A couple comments from a first pass today. I spent a while trying to debug a crash when I try to view an expanded sequence in legacy expansion mode but cut myself off before spending my entire evening on it.

src/enums/sequencing.ts

src/stores/sequencing.ts

cartermak · 2025-03-20T00:07:41Z

src/components/expansion/SequencesAndExpansionPanel.svelte

+              <div use:tooltip={{ content: 'Expand Sequence', placement: 'top' }}>
+                <button
+                  aria-label={`Expand '${sequenceOrFilter.seq_id}'`}
+                  class="st-button icon"
+                  on:click|stopPropagation={() => {
+                    if (isExpansionSequence(sequenceOrFilter)) {
+                      onExpandSequence(sequenceOrFilter);
+                    }
+                  }}
+                >
+                  <PlayIcon />
+                </button>
+              </div>


It's misleading that each sequence gets an "Expand Sequence" button: it seems like it'll only expand that single sequence. Is there a reason why we moved away from a single button to expand the entire plan?

I'll be honest I think I may have biased this UI decision more towards GSFC use because we generally don't have the idea of 'multiple sequences' - instead one sequence representing our whole load makes more sense. However, I think a 'Run All' button definitely makes sense especially if you're using Typescript/Legacy/Command Expansion mode - so I'm okay with adding it back in.

OK nice, yeah I totally see how we ended up there. I think, though, that even in the case of template expansion, we expand all sequences at once? I think with either legacy or template expansion, one "run" will affect all sequences?

Added an Expand All button for both modes - the row-specific 'expand' buttons will still only expand that sequence, but Expand All will do all that are configured for the current/latest simulation dataset

JosephVolosin · 2025-03-20T14:12:21Z

@AaronPlave / @duranb, with our new panel for Sequences & Expansion (and retiring the Expansion panel), what (if anything) needs to change with the view JSON schema?

…e the word typescript instead of legacy

cartermak · 2025-03-21T00:26:50Z

Some things I've observed but don't yet understand:

I can sometimes get the UI into a state where the app menu shows me neither expansion nor templates as an option:
For some reason, the input value passed to seqJsonToSequence when viewing an expansion run has changed from a stringified JSON to a parsed object. I can't figure out why, because it doesn't look like the underlying stores or processing changed at all. Unless this is broken on develop, too?

JosephVolosin · 2025-03-21T15:34:21Z

Some things I've observed but don't yet understand:

1. I can sometimes get the UI into a state where the app menu shows me neither expansion nor templates as an option:
   ![image](https://private-user-images.githubusercontent.com/49076171/425226428-afe046e2-cc85-4467-8b85-e19b8c800e09.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NDI1Njc5MzksIm5iZiI6MTc0MjU2NzYzOSwicGF0aCI6Ii80OTA3NjE3MS80MjUyMjY0MjgtYWZlMDQ2ZTItY2M4NS00NDY3LThiODUtZTE5YjhjODAwZTA5LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTAzMjElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwMzIxVDE0MzM1OVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTljOWZlZmNlMmNiOTNmMTU0M2JjMDU1YTY2YmEyZTVjOWZmZmY1ZTlmMTIyZmRhZWUzN2YwNWMzMzFmMmRjMDAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.jij_0Bv-JPlHB9C5PReE8gSCoDn0p3uQ8uF8T4Jz_Js)

2. For some reason, the `input` value passed to `seqJsonToSequence` when viewing an expansion run has changed from a stringified JSON to a parsed object. I can't figure out why, because it doesn't look like the underlying stores or processing changed at all. Unless this is broken on develop, too?

Can you try to replicate? I think this was being caused by missing a few updates in the UI after we switched around some naming related to the env variable.
I think this is because we moved from calling a getSequenceSeqJson to directly referencing expanded_sequences. See convo here: Sequence Templates #1642 (comment)

…nsion set is selected

sonarqubecloud · 2025-03-21T16:56:47Z

Quality Gate passed

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

duranb · 2025-03-12T00:24:18Z

src/components/expansion/SequencesAndExpansionPanel.svelte

+
+  $: sequencesAndFilters = [...$expansionSequences, ...$sequenceFilters];
+
+  // TODO: Better cleanup of activeSequenceFilterName and activeSequenceFilterId


Do you mind elaborating more as to what needs to be cleaned up for activeSequenceFilterName and activeSequenceFilterId?

duranb · 2025-03-12T00:25:55Z

src/components/modals/ExpansionSequenceModal.svelte

+      expandedTemplate => expandedTemplate.seq_id === expansionSequence.seq_id,
+    );
+    outputStr = expandedTemplate?.expanded_template ?? `No output found for sequence "${expansionSequence.seq_id}"'`;
+    // TODO: Language - see question in Notes


Do you mind pasting the question directly here if you intend to keep this TODO?

duranb · 2025-03-12T17:47:13Z

src/types/sequencing.ts

@@ -22,6 +22,18 @@ import type { SequenceTypes } from '../enums/sequencing';
 import type { ArgDelegator } from '../utilities/sequence-editor/extension-points';
 import type { UserId } from './app';
 import type { GlobalType } from './global-type';
+import type { ActivityLayerFilter } from './timeline';
+
+export type SequenceActivityFilter = ActivityLayerFilter;


Are you using the SequenceActivityFilters in the timeline somewhere? It looks like you're utilizing the "filter" part of the ActivityLayerFilter. If it's not really a timeline feature, then I think it would be less confusing to maybe rename and rework ActivityLayerFilter and its surrounding types to something more reusable for other features by using generic types.

duranb · 2025-03-24T19:42:03Z

e2e-tests/fixtures/ExpansionRules.ts

@@ -111,6 +114,11 @@ export class ExpansionRules {
    await this.page.goto('/expansion/rules', { waitUntil: 'networkidle' });
    await this.page.waitForTimeout(250);
    await expect(this.rulesNavButton).toHaveClass(/selected/);
+    const typescriptIsToggledOff = await this.page.getByText('COMMAND_EXPANSION_MODE').isVisible();


What specifically is this trying to select? Is this some button or label? It's not immediately clear where this value is being placed in the HTML and I worry that it'll make it hard to debug later on if the text ever changes.

duranb · 2025-03-24T19:42:57Z

src/utilities/sequence-editor/languages/handlebars/handlebars.ts

@@ -0,0 +1,17 @@
+import { foldInside, foldNodeProp, indentNodeProp, LRLanguage } from '@codemirror/language';


Is handlebars itself a standalone language and is seq-n-handlebars a sub-language?

duranb reviewed Mar 10, 2025

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

duranb reviewed Mar 10, 2025

View reviewed changes

package.json Outdated Show resolved Hide resolved

mattdailis reviewed Mar 10, 2025

View reviewed changes

src/stores/sequencing.ts Outdated Show resolved Hide resolved

mattdailis reviewed Mar 10, 2025

View reviewed changes

duranb reviewed Mar 12, 2025

View reviewed changes

AaronPlave reviewed Mar 12, 2025

View reviewed changes

src/components/timeline/form/TimelineEditor/ActivityFilterBuilder.svelte Outdated Show resolved Hide resolved

JosephVolosin and others added 23 commits March 12, 2025 13:24

Add activity filtering to sequence creation

2aae88c

Move sequence creation to a modal

ec70716

Another approach to do the same as the last commit - some issues w. displaying the ActivityFilterBuilder from within the modal

Create Sequencing panel in plan view

af02524

Draft of rigging templating button

1ae99d2

Add bulk functionality to SequencingPanel

6c11841

Refactor SequenceDefinition => SequenceFilter; draft work for templat…

9c55c38

…ing actions

WIP UI work for expanded sequences

049caf4

WIP UI working for sequencing in plan, expanded templates, etc.

b8cc22f

mark expansions as dirty on timeline change; remove ephemeral sequenc…

0f2dd90

…e generation

WIP re-organization of sequencing panel, back-end linkage work

be3cbc6

WIP debugging template expansion

1f99b9e

fix with rebase

3edfbfa

fix imports

3c20b37

Debugging naive Handlebars overlay

e1a359e

Debugging a naive overlay approach to integrating handlebars into the editor.

Fix autocompletion

8a8645d

Fix Seq-language autocompletion to work outside, and only outside, of template insert fields.

Ignore symlink to aerie

987b662

WIP - building sequence template editor

80d6102

WIP - building sequence template editor

c32a964

WIP - adding "new template" functionality

6d83e3b

Doesn't include any back-end support, but this does stub out the back end as a store that should be easy-ish to replace with a real version.

WIP - add "new template" modal

bbd402e

WIP - use editor change event to save template content

fed6750

WIP - debug parcel selection

8bb8099

WIP - stop cursor jumping in editor

418bc38

Add Handlebars CodeMirror language support

a044e6d

JosephVolosin had a problem deploying to test-workflow March 18, 2025 16:23 — with GitHub Actions Failure

pranav-super reviewed Mar 18, 2025

View reviewed changes

change default sequencing mode to legacy

8b7629f

pranav-super had a problem deploying to test-workflow March 19, 2025 19:45 — with GitHub Actions Failure

addTemplateExpansion -> addTemplate

070fb62

pranav-super had a problem deploying to test-workflow March 19, 2025 19:47 — with GitHub Actions Failure

cartermak reviewed Mar 20, 2025

View reviewed changes

Rename PUBLIC_SEQUENCING_MODE to PUBLIC_COMMAND_EXPANSION_MODE and us…

653ec11

…e the word typescript instead of legacy

mattdailis had a problem deploying to test-workflow March 20, 2025 16:56 — with GitHub Actions Failure

Move 'LEGACY' to 'TYPESCRIPT' for sequence mode flag; E2E test fixes

a2ab166

JosephVolosin had a problem deploying to test-workflow March 21, 2025 14:23 — with GitHub Actions Failure

Improve text on TimeRangeModal

4457697

JosephVolosin had a problem deploying to test-workflow March 21, 2025 15:31 — with GitHub Actions Failure

Add 'Expand All' button, fix GQL query for retrieving expanded sequences

7e5dd8c

JosephVolosin had a problem deploying to test-workflow March 21, 2025 16:12 — with GitHub Actions Failure

Disable expansion buttons if using 'Typescript' expansion and no expa…

2e639f5

…nsion set is selected

JosephVolosin had a problem deploying to test-workflow March 21, 2025 16:17 — with GitHub Actions Failure

Lint/format fixes

7c98393

JosephVolosin had a problem deploying to test-workflow March 21, 2025 16:19 — with GitHub Actions Failure

Fix 'any' => 'ExpandedSequence[]' in getExpansionSequenceSeqJson

b5b29a4

JosephVolosin had a problem deploying to test-workflow March 21, 2025 16:33 — with GitHub Actions Failure

E2E test fixes

69c324a

JosephVolosin temporarily deployed to test-workflow March 21, 2025 16:55 — with GitHub Actions Inactive

duranb requested changes Mar 24, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sequence Templates #1642

Sequence Templates #1642

JosephVolosin commented Mar 10, 2025 •

edited

Loading

mattdailis Mar 10, 2025

mattdailis Mar 11, 2025

JosephVolosin Mar 11, 2025

duranb Mar 12, 2025

JosephVolosin Mar 12, 2025

duranb Mar 12, 2025

duranb Mar 12, 2025

JosephVolosin Mar 12, 2025

pranav-super left a comment

pranav-super Mar 18, 2025

JosephVolosin Mar 20, 2025

pranav-super Mar 18, 2025

JosephVolosin Mar 20, 2025

pranav-super Mar 18, 2025

JosephVolosin Mar 21, 2025

JosephVolosin Mar 21, 2025

pranav-super Mar 18, 2025

JosephVolosin Mar 20, 2025

pranav-super Mar 18, 2025

JosephVolosin Mar 21, 2025

cartermak left a comment

cartermak Mar 20, 2025

JosephVolosin Mar 20, 2025

cartermak Mar 20, 2025

JosephVolosin Mar 21, 2025

JosephVolosin commented Mar 20, 2025

cartermak commented Mar 21, 2025 •

edited

Loading

JosephVolosin commented Mar 21, 2025

sonarqubecloud bot commented Mar 21, 2025

duranb Mar 12, 2025

duranb Mar 12, 2025

duranb Mar 12, 2025

duranb Mar 24, 2025

duranb Mar 24, 2025


		$: sequencesAndFilters = [...$expansionSequences, ...$sequenceFilters];

		// TODO: Better cleanup of activeSequenceFilterName and activeSequenceFilterId

		@@ -0,0 +1,17 @@
		import { foldInside, foldNodeProp, indentNodeProp, LRLanguage } from '@codemirror/language';

Sequence Templates #1642

Are you sure you want to change the base?

Sequence Templates #1642

Conversation

JosephVolosin commented Mar 10, 2025 • edited Loading

New Environment Variable

Sequence Template Editor

Plan View - Sequences & Expansion

Step-by-step guide

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pranav-super left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cartermak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JosephVolosin commented Mar 20, 2025

cartermak commented Mar 21, 2025 • edited Loading

JosephVolosin commented Mar 21, 2025

sonarqubecloud bot commented Mar 21, 2025

Quality Gate passed

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JosephVolosin commented Mar 10, 2025 •

edited

Loading

cartermak commented Mar 21, 2025 •

edited

Loading