Feature Request: Support Async filter Function in Dexie Export-Import Add-on for Conflict Resolution

[dsfx3d]

Hello, Dexie team! I am using the Dexie Export-Import add-on to backup/restore data from an object store like S3. While implementing conflict resolution, I encountered limitations with the filter option, which currently does not support asynchronous operations.

Problem:

In P2P sync scenarios, conflict resolution often requires querying the database to determine if a record should be imported based on custom criteria (e.g., timestamp comparison or "soft delete" status). However, the current filter function is synchronous, which limits the ability to query the database during the filtering process.

As a workaround, I preload the entire database into memory and then resolve conflicts within filter. While this approach works in small databases, it becomes impractical for larger datasets due to memory limitations and performance concerns.

Proposed Solution:

Allow the filter function to return a Promise, enabling asynchronous operations. This would allow developers to query the database directly within filter and make more informed decisions.

Example Usage:

importInto(db, blob, {
  filter: async (table, record) => {
    const existingRecord = await db[table].get(record.id);
    // Resolve conflict based on custom logic
    return !existingRecord || record.updated_at > existingRecord.updated_at;
  }
});

Benefits:

• Flexible Conflict Resolution: Enables conflict handling that requires database queries, which is essential for complex sync requirements in P2P environments.
• Optimized Memory Usage: Eliminates the need to preload the entire database for conflict checking, making it more suitable for large databases.
• Improved Sync Accuracy: By querying the database in real-time, sync operations can be more accurate and tailored to specific application needs.

Adding async support to filter would greatly expand the use cases for Dexie Export-Import, especially in P2P sync scenarios. This enhancement would improve both performance and flexibility for developers working with large datasets. Thank you for considering this request, and thank you for your continued work on Dexie!