Migrate data from umami old version to new version #2215

ajinkyaraj-23 · 2024-12-06T21:45:50Z

Proposed changes

(Specific to umami desktop version)
Due to addition of custom protocol in apps/desktop/public/electron.js, the localstorage key structure changed from file:// to app://assets thus the data of accounts etc. was not being visible in the upgraded version (2.3.4)

Added a logic to migrate the existing data from old key to new keys in leveldb.

Types of changes

Steps to reproduce

Upgrade to 2.3.4 and data from previous version disappers.

| Before | Now |

To test build a production grade packaged app and replace it over existing 2.3.3 version app. The data should be visible.

Checklist

Tests that prove my fix is effective or that my feature works have been added
Documentation has been added (if appropriate)
Screenshots are added (if any UI changes have been made)
All TODOs have a corresponding task created (and the link is attached to it)

vercel · 2024-12-06T21:45:54Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
umami-embed-iframe	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 9, 2025 3:10pm
umami-embed-iframe-ghostnet	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 9, 2025 3:10pm
umami-v2-web	🛑 Canceled (Inspect)			Jan 9, 2025 3:10pm
umami-v2-web-storybook	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 9, 2025 3:10pm

asiia-trilitech · 2024-12-09T10:17:34Z

apps/desktop/public/electron.js

@@ -223,7 +247,10 @@ function start() {
  // This method will be called when Electron has finished its initialization and
  // is ready to create the browser windows.
  // Some APIs can only be used after this event occurs.
-  app.whenReady().then(createWindow);
+  app.whenReady().then(async () => {
+    await readAndCopyValues();


Should this be called every time the app is open?
Would it be possible to check if the user data is absent & only then copy it (so readAndCopyValues() only runs once)?

Will log out work properly?
We should make sure to clear both dbs on logout to avoid repopulating db with the old data

Right now its called everytime the app is open. I believe that will rewrite old data again and again. I can change it like follows:
once the data is copied i can delete old data. Then there is no problem?

Also consider creating a backup for old data. This allows for recovery in case the migration encounters unforeseen issues.

once the data is copied i can delete old data. Then there is no problem?

Good idea, @ajinkyaraj-23
However, migrating back to the older version would result in no user data - we should check with Aswin if that's something we're OK with

Right that's a good question.

OKendigelyan · 2024-12-09T12:58:59Z

apps/desktop/public/electron.js

+async function readAndCopyValues() {
+  try {
+    const accountsValue = await db.get("_file://\x00\x01persist:accounts");
+    const rootValue = await db.get("_file://\x00\x01persist:root");


We should migrate all the data present in the existing (old) DB, not just root and accounts

Why?
following are the keys which are generated:

Key: b'_devtools://devtools\x00\x01experiments', Value : b'\x01{}' Key: b'_devtools://devtools\x00\x01localInspectorVersion', Value : b'\x0137' Key: b'_file://\x00\x01beacon:matrix-selected-node', Value : b'\x01beacon-node-1.hope-5.papers.tech' Key: b'_file://\x00\x01beacon:sdk-matrix-preserved-state', Value : b'\x01{"syncToken":"s44195095_1_0_1_1_1_1_2556620_1","rooms":{}}' Key: b'_file://\x00\x01beacon:sdk-secret-seed', Value : b'\x019eca6480-5621-54a9-7c26-9d5cc062bc9f' Key: b'_file://\x00\x01beacon:sdk_version', Value : b'\x014.2.2' Key: b'_file://\x00\x01chakra-ui-color-mode', Value : b'\x01dark' Key: b'_file://\x00\x01loglevel:broadcast-channel', Value : b'\x01ERROR' Key: b'_file://\x00\x01loglevel:customauth', Value : b'\x01SILENT' Key: b'_file://\x00\x01loglevel:fnd', Value : b'\x01SILENT' Key: b'_file://\x00\x01loglevel:http-helpers', Value : b'\x01INFO' Key: b'_file://\x00\x01loglevel:torus.js', Value : b'\x01SILENT' Key: b'_file://\x00\x01persist:accounts'

To keep the same functionality as the previous version and lose nothing

We only backup these two keys in backup so i guess, this is good enough, other keys need not be copied and we can still keep the same functionality. Rather trying to copy everything causes more errors.

What about Key: b'_file://\x00\x01loglevel:customauth', Value : b'\x01SILENT' - I believe that's dApp connections, they should be preserved on update I think

I think that will be enough for this fix, but any other flags or keys on the client side will be lost

apps/desktop/src/Router.tsx

asiia-trilitech

Thank you, @OKendigelyan & @ajinkyaraj-23 - amazing work 💪 💪 💪

LGTM - but please check the comments first

asiia-trilitech · 2025-01-09T12:55:52Z

packages/tezos/package.json

-    "@taquito/signer": "^20.1.2",
-    "@taquito/taquito": "^20.1.2",
-    "@taquito/utils": "^20.1.2",
+    "@taquito/ledger-signer": "^21.0.0",


We should check ledger on Win, Linux & Mac before publishing this release just in case

asiia-trilitech · 2025-01-09T13:02:30Z

apps/desktop-e2e/tsconfig.json

@@ -13,7 +13,8 @@
    "isolatedModules": true,
    "noEmit": true,
    "jsx": "react-jsx",
-    "customConditions": ["@umami/source"]
+    "customConditions": ["@umami/source"],
+    "preserveSymlinks": true


Is it needed?

asiia-trilitech · 2025-01-09T13:07:01Z

apps/desktop/public/electron.js

+async function createBackupFromPrevDB() {
+  const dbPath = path.normalize(path.join(app.getPath("userData"), "Local Storage", "leveldb"));
+  const backupPath = path.normalize(
+    path.join(app.getPath("userData"), "Local Storage", "backup_leveldb.json")


nit: maybe umami_backup_leveldb.json ?

asiia-trilitech · 2025-01-09T13:14:08Z

packages/state/src/slices/accounts/accounts.ts

@@ -20,6 +20,10 @@ export const accountsInitialState: AccountsState = {
  seedPhrases: {},
  secretKeys: {},
  password: "",
+  alerts: {


Not sure if alerts should be accounts slice, but happy for it to stay here for now.

We can do migration to a different / separate slice later if needed

asiia-trilitech · 2025-01-09T13:14:57Z

packages/state/src/slices/accounts/accounts.ts

@@ -144,6 +148,15 @@ export const accountsSlice = createSlice({
    setPassword: (state, { payload }: { payload: string }) => {
      state.password = payload;
    },
+    setAlerts: (


nit: could you please add tests for this?

asiia-trilitech · 2025-01-09T13:17:40Z

packages/state/src/beacon/WalletClient.ts

@@ -16,6 +16,13 @@ export const logout = (persistor: Persistor) =>
    .catch(() => {})
    .finally(() => {
      persistor.pause();
+      const migrationCompleted = localStorage.getItem("migration_2_3_3_to_2_3_4_completed");


Please add comments on migration flag setup here

asiia-trilitech · 2025-01-09T13:20:31Z

packages/state/src/hooks/backup.ts

@@ -80,6 +80,7 @@ export const restoreV2BackupFile = async (
  persistor.pause();

  localStorage.clear();
+  localStorage.setItem("migration_2_3_3_to_2_3_4_completed", "true");


nit: maybe migration_to_2_3_5_completed?

asiia-trilitech · 2025-01-09T13:35:20Z

apps/desktop/public/electron.js

+  app.whenReady().then(async () => {
+    // Execute createBackupFromPrevDB at the beginning
+    try {
+      await createBackupFromPrevDB();


Should we do somewhat of a version check - like if it's a migration from 2.4 & earlier, then do the backup

To avoid creating backup on the 2.6+ versions when not needed.

Discussed offline with @OKendigelyan - there is no simple way to check the version

asiia-trilitech · 2025-01-09T13:37:12Z

apps/desktop/public/electron.js

+
+ipcMain.handle("getBackupData", () => {
+  const backupPath = path.normalize(
+    path.join(app.getPath("userData"), "Local Storage", "backup_leveldb.json")


nit: maybe this should be a separate function used in both getBackup & createBackup?

asiia-trilitech · 2025-01-09T13:39:24Z

apps/desktop/public/electron.js

+    return;
+  }
+
+  if (fs.existsSync(backupPath)) {


I have some security-related concerns regarding not deleting the backup file after we already used it.

Discussed offline with @OKendigelyan - seems like there is no simple solution for this.

If we're planning to remove this data migration bit in the nearest future (& clean up the backup with it 🙂 ) - should be OK

ajinkyaraj-23 requested review from asiia-trilitech and OKendigelyan December 6, 2024 21:45

ajinkyaraj-23 self-assigned this Dec 6, 2024

ajinkyaraj-23 linked an issue Dec 6, 2024 that may be closed by this pull request

Updating Umami should not lead to re-iimporting all my accounts #2195

Closed

asiia-trilitech reviewed Dec 9, 2024

View reviewed changes

OKendigelyan reviewed Dec 9, 2024

View reviewed changes

OKendigelyan force-pushed the 2195-updating-umami-should-not-lead-to-re-iimporting-all-my-accounts branch from b8779cc to 570c4d8 Compare December 9, 2024 13:08

vercel bot deployed to Preview – umami-v2-web December 9, 2024 13:09 View deployment

vercel bot deployed to Preview – umami-embed-iframe-ghostnet December 9, 2024 13:10 View deployment

vercel bot deployed to Preview – umami-v2-web-storybook December 9, 2024 13:11 View deployment

vercel bot deployed to Preview – umami-embed-iframe December 9, 2024 13:12 View deployment

ajinkyaraj-23 force-pushed the 2195-updating-umami-should-not-lead-to-re-iimporting-all-my-accounts branch from 570c4d8 to 5e7b427 Compare December 11, 2024 23:18

vercel bot deployed to Preview – umami-v2-web-storybook December 11, 2024 23:22 View deployment

vercel bot deployed to Preview – umami-embed-iframe December 11, 2024 23:23 View deployment

vercel bot deployed to Preview – umami-embed-iframe-ghostnet December 11, 2024 23:24 View deployment

vercel bot deployed to Preview – umami-v2-web December 11, 2024 23:25 View deployment

vercel bot deployed to Preview – umami-v2-web December 13, 2024 12:48 View deployment

vercel bot deployed to Preview – umami-embed-iframe December 13, 2024 12:49 View deployment

vercel bot deployed to Preview – umami-v2-web-storybook December 13, 2024 12:50 View deployment

vercel bot deployed to Preview – umami-embed-iframe-ghostnet December 13, 2024 12:51 View deployment

vercel bot deployed to Preview – umami-embed-iframe December 13, 2024 15:37 View deployment

vercel bot deployed to Preview – umami-embed-iframe-ghostnet December 13, 2024 15:38 View deployment

vercel bot deployed to Preview – umami-v2-web-storybook December 13, 2024 15:41 View deployment

vercel bot deployed to Preview – umami-v2-web December 13, 2024 15:42 View deployment

vercel bot deployed to Preview – umami-v2-web December 13, 2024 18:54 View deployment

github-advanced-security bot found potential problems Dec 13, 2024

View reviewed changes

apps/desktop/src/Router.tsx Fixed Show fixed Hide fixed

vercel bot deployed to Preview – umami-embed-iframe December 13, 2024 18:55 View deployment

vercel bot deployed to Preview – umami-v2-web-storybook December 13, 2024 18:56 View deployment

vercel bot deployed to Preview – umami-embed-iframe-ghostnet December 13, 2024 18:57 View deployment

vercel bot deployed to Preview – umami-v2-web-storybook January 9, 2025 09:54 View deployment

OKendigelyan temporarily deployed to Production – umami-v2 January 9, 2025 11:01 — with GitHub Actions Inactive

vercel bot deployed to Preview – umami-v2-web-storybook January 9, 2025 11:15 View deployment

vercel bot deployed to Preview – umami-embed-iframe January 9, 2025 11:16 View deployment

vercel bot deployed to Preview – umami-v2-web January 9, 2025 11:17 View deployment

vercel bot deployed to Preview – umami-embed-iframe-ghostnet January 9, 2025 11:18 View deployment

Bump taquito to 21

b26b135

OKendigelyan force-pushed the 2195-updating-umami-should-not-lead-to-re-iimporting-all-my-accounts branch from ab46827 to b26b135 Compare January 9, 2025 11:26

vercel bot deployed to Preview – umami-embed-iframe-ghostnet January 9, 2025 11:27 View deployment

vercel bot deployed to Preview – umami-v2-web-storybook January 9, 2025 11:30 View deployment

vercel bot deployed to Preview – umami-v2-web January 9, 2025 11:31 View deployment

vercel bot deployed to Preview – umami-embed-iframe January 9, 2025 11:32 View deployment

OKendigelyan temporarily deployed to Production – umami-v2 January 9, 2025 11:44 — with GitHub Actions Inactive

OKendigelyan requested a review from asiia-trilitech January 9, 2025 11:57

OKendigelyan self-assigned this Jan 9, 2025

OKendigelyan self-requested a review January 9, 2025 13:48

OKendigelyan approved these changes Jan 9, 2025

View reviewed changes

asiia-trilitech approved these changes Jan 9, 2025

View reviewed changes

Fix review

cbea319

OKendigelyan force-pushed the 2195-updating-umami-should-not-lead-to-re-iimporting-all-my-accounts branch from 4a7b617 to cbea319 Compare January 9, 2025 14:59

vercel bot deployed to Preview – umami-v2-web January 9, 2025 15:06 View deployment

vercel bot deployed to Preview – umami-embed-iframe January 9, 2025 15:07 View deployment

vercel bot deployed to Preview – umami-v2-web-storybook January 9, 2025 15:09 View deployment

vercel bot deployed to Preview – umami-embed-iframe-ghostnet January 9, 2025 15:10 View deployment

OKendigelyan merged commit 558eb09 into main Jan 9, 2025
10 checks passed

OKendigelyan deleted the 2195-updating-umami-should-not-lead-to-re-iimporting-all-my-accounts branch January 9, 2025 17:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate data from umami old version to new version #2215

Migrate data from umami old version to new version #2215

ajinkyaraj-23 commented Dec 6, 2024

vercel bot commented Dec 6, 2024 •

edited

Loading

asiia-trilitech Dec 9, 2024

ajinkyaraj-23 Dec 9, 2024

OKendigelyan Dec 9, 2024

asiia-trilitech Dec 10, 2024

ajinkyaraj-23 Dec 10, 2024

OKendigelyan Dec 9, 2024

ajinkyaraj-23 Dec 9, 2024

OKendigelyan Dec 9, 2024

ajinkyaraj-23 Dec 10, 2024

asiia-trilitech Dec 10, 2024

OKendigelyan Dec 10, 2024

asiia-trilitech left a comment

asiia-trilitech Jan 9, 2025

asiia-trilitech Jan 9, 2025

asiia-trilitech Jan 9, 2025

asiia-trilitech Jan 9, 2025

asiia-trilitech Jan 9, 2025

asiia-trilitech Jan 9, 2025

asiia-trilitech Jan 9, 2025

asiia-trilitech Jan 9, 2025

asiia-trilitech Jan 9, 2025

asiia-trilitech Jan 9, 2025

asiia-trilitech Jan 9, 2025

Migrate data from umami old version to new version #2215

Migrate data from umami old version to new version #2215

Conversation

ajinkyaraj-23 commented Dec 6, 2024

Proposed changes

Types of changes

Steps to reproduce

Checklist

vercel bot commented Dec 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

asiia-trilitech left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vercel bot commented Dec 6, 2024 •

edited

Loading