chore(dashboard): user journey smoke tests #7124

LetItRock · 2024-11-25T13:04:27Z

What changed? Why was the change needed?

Dashboard - the core user journeys smoke tests that cover:

dashboard-created workflow
code-created workflow

TODO (in the future):

we have one drawback: we need to establish independent clerk and api sessions for the tests, I left a comment about it in the code

Screenshots

linear · 2024-11-25T13:04:30Z

NV-4795 Write two E2E user journey as a smoke test for new Dashboard

netlify · 2024-11-25T13:04:46Z

✅ Deploy Preview for novu-stg-vite-dashboard-poc ready!

Name	Link
🔨 Latest commit	`fa860f3`
🔍 Latest deploy log	https://app.netlify.com/sites/novu-stg-vite-dashboard-poc/deploys/674d7a4f4008e900085d165d
😎 Deploy Preview	https://deploy-preview-7124.dashboard-v2.novu-staging.co
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

LetItRock · 2024-11-25T13:06:12Z

apps/api/.gitignore

@@ -121,3 +121,4 @@ backups/

 # Nest.js auto-generated metadata (https://docs.nestjs.com/recipes/swc#monorepo-and-cli-plugins)
 src/metadata.ts
+src/.env.test


removed the .env.test file from git, because the local file that I used for the tests had sensitive clerk credentials

LetItRock · 2024-11-25T13:06:45Z

apps/dashboard/.gitignore

@@ -28,4 +28,6 @@ tsconfig.node.tsbuildinfo
 /test-results/
 /playwright-report/
 /blob-report/
-/playwright/.cache/
+/playwright/
+.env.test


.env.test is the file for the Dashboard env variables

LetItRock · 2024-11-25T13:07:21Z

apps/dashboard/.gitignore

-/playwright/.cache/
+/playwright/
+.env.test
+.env.playwright


.env.playwright is the file for the playwright that has all env variables to setup the clerk and api session

LetItRock · 2024-11-25T13:07:48Z

apps/dashboard/package.json

@@ -5,6 +5,7 @@
  "type": "module",
  "scripts": {
    "start": "vite",
+    "start:test": "vite --mode test",


run dashboard and use .env.test file

LetItRock · 2024-11-25T13:08:21Z

apps/dashboard/package.json

+    "@novu/dal": "workspace:*",
+    "@novu/testing": "workspace:*",


used to initialize the api session

LetItRock · 2024-11-25T13:22:45Z

apps/dashboard/tests/code-created-flow.spec.ts

+  bridgeServer = new BridgeServer({ secretKey: session.environment.apiKeys[0].key, apiUrl: process.env.API_URL });
+
+  const newWorkflow = workflow(workflowId, async ({ step }) => {
+    await step.inApp(
+      inAppStepId,
+      async (controls) => {
+        return {
+          subject: `Hi ${controls.name}! You've been invited to join the Novu project`,
+          body,
+        };
+      },
+      {
+        controlSchema: {
+          type: 'object',
+          properties: {
+            name: { type: 'string', default: 'John' },
+          },
+        } as const,
+      }
+    );
+  });
+  await bridgeServer.start({ workflows: [newWorkflow] });
+  await session.testAgent.post(`/v1/bridge/sync`).send({
+    bridgeUrl: bridgeServer.serverPath,
+  });


Run the bridge server and sync the workflow

LetItRock · 2024-11-25T13:23:04Z

apps/dashboard/tests/code-created-flow.spec.ts

+  await session.testAgent.post(`/v1/bridge/sync`).send({
+    bridgeUrl: bridgeServer.serverPath,
+  });
+  await page.waitForTimeout(2000);


should be removed

LetItRock · 2024-11-25T13:23:44Z

apps/dashboard/tests/global.setup.ts

+  /**
+   * TODO: Currently we are running tests with a single Clerk user and organization.
+   * This approach has a drawback that the tests are not independent from each other and write the data to the same organization.
+   * We should consider creating a new Clerk user and organization for each test using the @clerk/backend package.
+   * Then initialize the BE session with the new Clerk user and organization and update Clerk user metadata from the newly created session.
+   */


hopefully, it explains the problem well

Correct! We should leverage Clerk Backend API, create a user before every run, sign in with that user, and then delete the user at the end.

Regarding the need for Clerk webhooks during user and organization creation, I suggest connecting to Mongo directly from the test code using the response of the Clerk API to create the user and organization record the application needs to function.

On user deletion, I don't think we need to do anything in Mongo as we run in a dockerized environment.

I'll think about this, as it will require reimplementing all the current setup.

LetItRock · 2024-11-25T13:26:31Z

apps/dashboard/tests/package.json

@@ -0,0 +1,3 @@
+{
+  "type": "commonjs"


The dal and testing packages are CJS modules which can't be imported by the ESM modules.
So this is the trick (together with tsconfig.json) to say "please allow me to import" in my test files :D

LetItRock · 2024-11-25T13:30:28Z

libs/testing/src/ee/ee.organization.service.ts

-  private memberRepository = getEERepository<MemberRepository>('MemberRepository');
+  private memberRepository: MemberRepository;
+
+  constructor({ mockClerkClient = true }: { mockClerkClient?: boolean } = {}) {


For the Dashboard E2E tests we don't want to mock the Clerk client, because we initialize the Clerk session with a real user that has the org and metadata that is used by the Dashboard code.

pkg-pr-new · 2024-11-25T13:41:28Z

Open in Stackblitz

@novu/client

npm i https://pkg.pr.new/novuhq/novu/@novu/client@7124

@novu/framework

npm i https://pkg.pr.new/novuhq/novu/@novu/framework@7124

@novu/js

npm i https://pkg.pr.new/novuhq/novu/@novu/js@7124

@novu/nest

npm i https://pkg.pr.new/novuhq/novu/@novu/nest@7124

@novu/headless

npm i https://pkg.pr.new/novuhq/novu/@novu/headless@7124

@novu/nextjs

npm i https://pkg.pr.new/novuhq/novu/@novu/nextjs@7124

@novu/node

npm i https://pkg.pr.new/novuhq/novu/@novu/node@7124

@novu/notification-center

npm i https://pkg.pr.new/novuhq/novu/@novu/notification-center@7124

novu

npm i https://pkg.pr.new/novuhq/novu@7124

@novu/providers

npm i https://pkg.pr.new/novuhq/novu/@novu/providers@7124

@novu/react

npm i https://pkg.pr.new/novuhq/novu/@novu/react@7124

@novu/react-native

npm i https://pkg.pr.new/novuhq/novu/@novu/react-native@7124

@novu/shared

npm i https://pkg.pr.new/novuhq/novu/@novu/shared@7124

@novu/stateless

npm i https://pkg.pr.new/novuhq/novu/@novu/stateless@7124

commit: 5d0de32

apps/dashboard/tests/dashboard-created-flow.spec.ts

apps/dashboard/tests/code-created-flow.spec.ts

apps/dashboard/tests/dashboard-created-flow.spec.ts

SokratisVidros · 2024-12-10T09:44:29Z

apps/dashboard/tests/global.setup.ts

+  /**
+   * TODO: Currently we are running tests with a single Clerk user and organization.
+   * This approach has a drawback that the tests are not independent from each other and write the data to the same organization.
+   * We should consider creating a new Clerk user and organization for each test using the @clerk/backend package.
+   * Then initialize the BE session with the new Clerk user and organization and update Clerk user metadata from the newly created session.
+   */


Correct! We should leverage Clerk Backend API, create a user before every run, sign in with that user, and then delete the user at the end.

Regarding the need for Clerk webhooks during user and organization creation, I suggest connecting to Mongo directly from the test code using the response of the Clerk API to create the user and organization record the application needs to function.

On user deletion, I don't think we need to do anything in Mongo as we run in a dockerized environment.

apps/dashboard/tests/page-models/create-workflow-sidebar.ts

SokratisVidros · 2024-12-10T09:51:39Z

apps/dashboard/tests/tsconfig.json

@@ -0,0 +1,10 @@
+{


Is the extra tsconfig.json required?

yes it is for the tests to treat them as commonjs module, otherwise the @novu/testing package won't work

SokratisVidros · 2024-12-10T09:52:23Z

apps/dashboard/tests/utils/session.ts

+  await page.addInitScript((currentSession) => {
+    window.addEventListener('DOMContentLoaded', () => {
+      localStorage.setItem('nv_auth_token', currentSession.token);
+      localStorage.setItem('nv_last_environment_id', currentSession.environment._id);


This is no longer required in the new dashboard. We can safely remove it.

partially, the nv_auth_token is used in tests setup

SokratisVidros · 2024-12-10T09:54:38Z

libs/testing/src/ee/ee.repository.factory.ts


-  return new EEMemberRepository(new CommunityOrganizationRepository(), new ClerkClientMock());
+  if (mockClerkClient) {


Please refer to my previous comment about creating a Clerk user before each test run and doing an actual sign-in so that we can test the actual interaction between Clerk and the application, not a mocked version of it.

LetItRock added 2 commits November 25, 2024 13:48

chore(dashboard): user journey smoke tests

e12abe9

chore(dashboard): fix imports in page models

9f22ad5

LetItRock self-assigned this Nov 25, 2024

LetItRock commented Nov 25, 2024

View reviewed changes

LetItRock requested review from SokratisVidros, BiswaViraj, ChmaraX and desiprisg November 25, 2024 13:31

Merge branch 'next' into nv-4795-dashboard-user-journey-smoke-tests

d0a3333

LetItRock temporarily deployed to Linting November 25, 2024 13:33 — with GitHub Actions Inactive

LetItRock had a problem deploying to Linting November 25, 2024 13:33 — with GitHub Actions Failure

chore(dashboard): remove timeout

ceabff2

LetItRock temporarily deployed to Linting November 25, 2024 13:37 — with GitHub Actions Inactive

LetItRock had a problem deploying to Linting November 25, 2024 13:37 — with GitHub Actions Failure

chore(api): revert .env.test

8ad2e31

LetItRock temporarily deployed to Linting November 25, 2024 14:23 — with GitHub Actions Inactive

LetItRock had a problem deploying to Linting November 25, 2024 14:23 — with GitHub Actions Failure

chore(dashboard): fix spell check and adjust pipelines

461275c

LetItRock requested a review from a team as a code owner November 25, 2024 15:52

LetItRock requested review from scopsy and removed request for a team November 25, 2024 15:52

LetItRock temporarily deployed to Linting November 25, 2024 15:52 — with GitHub Actions Inactive

LetItRock temporarily deployed to Linting December 5, 2024 23:47 — with GitHub Actions Inactive

github-actions bot removed the @novu/api label Dec 5, 2024

Merge branch 'next' into nv-4795-dashboard-user-journey-smoke-tests

5727282

LetItRock temporarily deployed to Linting December 6, 2024 09:14 — with GitHub Actions Inactive

chore(dashboard): polishing the e2e pipeline

5d0de32

LetItRock temporarily deployed to Linting December 6, 2024 09:42 — with GitHub Actions Inactive

SokratisVidros requested changes Dec 10, 2024

View reviewed changes

Merge branch 'next' into nv-4795-dashboard-user-journey-smoke-tests

ad68574

LetItRock temporarily deployed to Linting December 31, 2024 14:20 — with GitHub Actions Inactive

github-actions bot added the @novu/dashboard label Dec 31, 2024

chore(dashboard): add suggestions from the pr #7124

b670d88

LetItRock temporarily deployed to Linting December 31, 2024 16:48 — with GitHub Actions Inactive

LetItRock requested a review from SokratisVidros December 31, 2024 16:49

Merge branch 'next' into nv-4795-dashboard-user-journey-smoke-tests

5d69645

LetItRock temporarily deployed to Linting January 7, 2025 09:34 — with GitHub Actions Inactive

Merge branch 'next' into nv-4795-dashboard-user-journey-smoke-tests

d2ff14b

LetItRock temporarily deployed to Linting January 24, 2025 10:27 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(dashboard): user journey smoke tests #7124

chore(dashboard): user journey smoke tests #7124

LetItRock commented Nov 25, 2024

linear bot commented Nov 25, 2024

netlify bot commented Nov 25, 2024 •

edited

Loading

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

SokratisVidros Dec 10, 2024

LetItRock Dec 31, 2024

LetItRock Nov 25, 2024

LetItRock Nov 25, 2024

pkg-pr-new bot commented Nov 25, 2024 •

edited

Loading

SokratisVidros Dec 10, 2024

SokratisVidros Dec 10, 2024

LetItRock Dec 31, 2024

SokratisVidros Dec 10, 2024

LetItRock Dec 31, 2024

SokratisVidros Dec 10, 2024


		return new EEMemberRepository(new CommunityOrganizationRepository(), new ClerkClientMock());
		if (mockClerkClient) {

chore(dashboard): user journey smoke tests #7124

Are you sure you want to change the base?

chore(dashboard): user journey smoke tests #7124

Conversation

LetItRock commented Nov 25, 2024

What changed? Why was the change needed?

Screenshots

linear bot commented Nov 25, 2024

netlify bot commented Nov 25, 2024 • edited Loading

✅ Deploy Preview for novu-stg-vite-dashboard-poc ready!

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkg-pr-new bot commented Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

netlify bot commented Nov 25, 2024 •

edited

Loading

pkg-pr-new bot commented Nov 25, 2024 •

edited

Loading