Skip to content

Commit

Permalink
FIX stuff
Browse files Browse the repository at this point in the history
  • Loading branch information
@pubkey
pubkey committed Nov 6, 2021
1 parent 9061018 commit a9c3090
Show file tree
Hide file tree
Showing 8 changed files with 55 additions and 50 deletions.
31 changes: 15 additions & 16 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,9 @@ You can reproduce these values by running `sh measure-metrics.sh` in the root fo

## Investigations

### Why is WatermelonDB so much faster?
### Why is LokiJS so much faster?

WatermelonDB uses the [LokiJS](https://github.com/techfort/LokiJS) adapter which is an **in memory** database that regularly persists the data to IndexedDB either on interval, or when the browser tab is closed. Keeping and processing the data in memory has the benefit of being much faster, but it also has its downsides:
WatermelonDB and the RxDB-LokiJS project use the [LokiJS](https://github.com/techfort/LokiJS) database as storage, which is an **in memory** database that regularly persists the data to IndexedDB either on interval, or when the browser tab is closed. Keeping and processing the data in memory has the benefit of being much faster, but it also has its downsides:

- Data can be lost when the JavaScript process is killed ungracefully like when the browser crashes or the power of the PC is terminated.
- There is no multi-tab-support. The data is not shared between multiple browser tabs of the same origin.
Expand All @@ -70,9 +70,9 @@ WatermelonDB uses the [LokiJS](https://github.com/techfort/LokiJS) adapter which

On the first page load, Firebase ensures that the local data is equal to the server side state. This means that the client has to be online at application startup which is the reason why Firebase is not completely offline first. To ensure the equalness of client side data, Firebase has to perform several requests to the backend, before the database will respond to queries. This makes the inital page load slow, and it becomes even more slower, the more data exists and has to be validated.

### Why is PouchDB & RxDB so slow?
### Why is PouchDB so slow?

For the PouchDB and RxDB (based on PouchDB storage) I used the [old](https://www.npmjs.com/package/pouchdb-adapter-idb) Indexeddb adapter.
For the PouchDB and RxDB (with PouchDB storage) I used the [old](https://www.npmjs.com/package/pouchdb-adapter-idb) Indexeddb adapter.
It is much less optimized than the [new](https://www.npmjs.com/package/pouchdb-adapter-indexeddb) adapter, but the new one made problems with returning the correct query results.
Theses problems have been fixed on the PouchDB master branch, but I have to wait for the next PouchDB release. I will update the repo when this change can be done.

Expand All @@ -84,18 +84,17 @@ AWS Datastore does not save any metadata together with the documents. Instead on

<!-- !!! DO NOT EDIT THIS TABLE DIRECTLY, IT IS GENERATED BY ./scripts/generate-feature-table.ts !!! -->

| Feature \ Project | aws | firebase | pouchdb | rxdb | rxdb-lokijs | watermelondb |
| ------------------------ | ------------------ | -------------------------------------------- | ------- | ---- | ----------- | ---------------------------------------------------------------------------------------- |
| Offline First | No, login required | Partially, must be online on first page load | Yes | Yes | Yes | Yes |
| Realtime Replication | Yes | Yes | Yes | Yes | Yes | Partially, must be implemented by hand |
| Multi Tab Support | Yes | Yes | No | Yes | Yes | Partially, relies on online sync |
| Observable Queries | No | Yes | No | Yes | Yes | Yes |
| Complex Queries | No | Yes | Yes | Yes | Yes | Partially, limit/skip/sort not working with LokiJS adapter (fixed in latest pre-release) |
| Client Side Encryption | No | No | Yes | Yes | Yes | No |
| Schema Support | Yes | No | No | Yes | Yes | Yes |
| Custom Backend | No | No | No | Yes | Yes | Yes |
| Custom Conflict Handling | Yes | No | Yes | Yes | No | No |

| Feature \ Project | aws | firebase | pouchdb | rxdb-pouchdb | rxdb-lokijs | watermelondb |
| ------------------------ | ------------------ | -------------------------------------------- | ------- | ------------ | ----------- | ---------------------------------------------------------------------------------------- |
| Offline First | No, login required | Partially, must be online on first page load | **Yes** | **Yes** | **Yes** | **Yes** |
| Realtime Replication | **Yes** | **Yes** | **Yes** | **Yes** | **Yes** | Partially, must be implemented by hand |
| Multi Tab Support | **Yes** | **Yes** | No | **Yes** | **Yes** | Partially, relies on online sync |
| Observable Queries | No | **Yes** | No | **Yes** | **Yes** | **Yes** |
| Complex Queries | No | **Yes** | **Yes** | **Yes** | **Yes** | Partially, limit/skip/sort not working with LokiJS adapter (fixed in latest pre-release) |
| Client Side Encryption | No | No | **Yes** | **Yes** | **Yes** | No |
| Schema Support | **Yes** | No | No | **Yes** | **Yes** | **Yes** |
| Custom Backend | No | No | No | **Yes** | **Yes** | **Yes** |
| Custom Conflict Handling | **Yes** | No | **Yes** | **Yes** | No | No |


## Starting the projects
Expand Down
48 changes: 24 additions & 24 deletions angular.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -122,25 +122,25 @@
"strict": true
}
},
"root": "projects/rxdb",
"sourceRoot": "projects/rxdb/src",
"root": "projects/rxdb-pouchdb",
"sourceRoot": "projects/rxdb-pouchdb/src",
"prefix": "app",
"architect": {
"build": {
"builder": "@angular-devkit/build-angular:browser",
"options": {
"outputPath": "dist/rxdb",
"index": "projects/rxdb/src/index.html",
"main": "projects/rxdb/src/main.ts",
"polyfills": "projects/rxdb/src/polyfills.ts",
"tsConfig": "projects/rxdb/tsconfig.app.json",
"outputPath": "dist/rxdb-pouchdb",
"index": "projects/rxdb-pouchdb/src/index.html",
"main": "projects/rxdb-pouchdb/src/main.ts",
"polyfills": "projects/rxdb-pouchdb/src/polyfills.ts",
"tsConfig": "projects/rxdb-pouchdb/tsconfig.app.json",
"inlineStyleLanguage": "less",
"assets": [
"projects/rxdb/src/favicon.ico",
"projects/rxdb/src/assets"
"projects/rxdb-pouchdb/src/favicon.ico",
"projects/rxdb-pouchdb/src/assets"
],
"styles": [
"projects/rxdb/src/styles.less"
"projects/rxdb-pouchdb/src/styles.less"
],
"scripts": []
},
Expand All @@ -160,8 +160,8 @@
],
"fileReplacements": [
{
"replace": "projects/rxdb/src/environments/environment.ts",
"with": "projects/rxdb/src/environments/environment.prod.ts"
"replace": "projects/rxdb-pouchdb/src/environments/environment.ts",
"with": "projects/rxdb-pouchdb/src/environments/environment.prod.ts"
}
],
"optimization": true,
Expand All @@ -186,34 +186,34 @@
"builder": "@angular-devkit/build-angular:dev-server",
"configurations": {
"production": {
"browserTarget": "rxdb:build:production"
"browserTarget": "rxdb-pouchdb:build:production"
},
"development": {
"browserTarget": "rxdb:build:development"
"browserTarget": "rxdb-pouchdb:build:development"
}
},
"defaultConfiguration": "development"
},
"extract-i18n": {
"builder": "@angular-devkit/build-angular:extract-i18n",
"options": {
"browserTarget": "rxdb:build"
"browserTarget": "rxdb-pouchdb:build"
}
},
"test": {
"builder": "@angular-devkit/build-angular:karma",
"options": {
"main": "projects/rxdb/src/test.ts",
"polyfills": "projects/rxdb/src/polyfills.ts",
"tsConfig": "projects/rxdb/tsconfig.spec.json",
"karmaConfig": "projects/rxdb/karma.conf.js",
"main": "projects/rxdb-pouchdb/src/test.ts",
"polyfills": "projects/rxdb-pouchdb/src/polyfills.ts",
"tsConfig": "projects/rxdb-pouchdb/tsconfig.spec.json",
"karmaConfig": "projects/rxdb-pouchdb/karma.conf.js",
"inlineStyleLanguage": "less",
"assets": [
"projects/rxdb/src/favicon.ico",
"projects/rxdb/src/assets"
"projects/rxdb-pouchdb/src/favicon.ico",
"projects/rxdb-pouchdb/src/assets"
],
"styles": [
"projects/rxdb/src/styles.less"
"projects/rxdb-pouchdb/src/styles.less"
],
"scripts": []
}
Expand All @@ -222,8 +222,8 @@
"builder": "@angular-eslint/builder:lint",
"options": {
"lintFilePatterns": [
"projects/rxdb/**/*.ts",
"projects/rxdb/**/*.html"
"projects/rxdb-pouchdb/**/*.ts",
"projects/rxdb-pouchdb/**/*.html"
]
}
}
Expand Down
4 changes: 3 additions & 1 deletion measure-metrics.sh
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
#!/bin/sh
set -e

npm run build
npm run measure:size

npm run test:aws
npm run server:firebase:setup && npm run test:firebase
npm run test:pouchdb
npm run test:rxdb
npm run test:rxdb-pouchdb
npm run test:rxdb-lokijs
npm run test:watermelondb

Expand Down
6 changes: 5 additions & 1 deletion package-lock.json
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion projects/aws/src/app/app.logic.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ import {
import {
unixTimestampToGraphql,
graphQLTimestampToUnix
} from 'projects/rxdb/src/app/shared';
} from 'projects/rxdb-pouchdb/src/app/shared';
import {
sortByNewestFirst,
sortMessagesByDateNewestFirst,
Expand Down
2 changes: 1 addition & 1 deletion projects/rxdb-lokijs/src/app/app.logic.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import {
RxMessageDocument,
RxMessageDocumentType,
RxUserDocument,
} from '../../../rxdb/src/app/types/rxdb.d';
} from '../../../rxdb-pouchdb/src/app/types/rxdb.d';
import {
Message,
UserWithLastMessage,
Expand Down
8 changes: 4 additions & 4 deletions projects/rxdb-lokijs/src/app/services/database.service.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,19 @@ import {
import type {
RxChatDatabase,
RxChatCollections
} from '../../../../rxdb/src/app/types/rxdb.d';
} from '../../../../rxdb-pouchdb/src/app/types/rxdb.d';

import {
GRAPHQL_PORT,
GRAPHQL_PATH,
GRAPHQL_SUBSCRIPTION_PORT,
GRAPHQL_SUBSCRIPTION_PATH
} from '../../../../rxdb/src/app/shared';
} from '../../../../rxdb-pouchdb/src/app/shared';
export const GRAPHQL_HTTP_PATH = 'http://' + window.location.hostname + ':' + GRAPHQL_PORT + GRAPHQL_PATH;
export const GRAPHQL_WS_PATH = 'ws://' + window.location.hostname + ':' + GRAPHQL_SUBSCRIPTION_PORT + GRAPHQL_SUBSCRIPTION_PATH;

import { RxMessagesSchema } from '../../../../rxdb/src/app/schemas/message.schema';
import { RxUsersSchema } from '../../../../rxdb/src/app/schemas/user.schema';
import { RxMessagesSchema } from '../../../../rxdb-pouchdb/src/app/schemas/message.schema';
import { RxUsersSchema } from '../../../../rxdb-pouchdb/src/app/schemas/user.schema';

/**
* Instead of using the default rxdb-import,
Expand Down
4 changes: 2 additions & 2 deletions scripts/generate-feature-table.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ export const FEATURE_TABLE: { [projectKey: string]: FeatureTableRow } = {
customConflictHandling: true,
customBackend: false
},
rxdb: {
'rxdb-pouchdb': {
offlineFirst: true,
observableQueries: true,
realtimeReplication: true,
Expand Down Expand Up @@ -131,7 +131,7 @@ function valueToString(value: FeatureTableValue): string {
return value;
} else {
if (value) {
return 'Yes';
return '**Yes**';
} else {
return 'No';
}
Expand Down

0 comments on commit a9c3090

Please sign in to comment.