Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

Commit

Permalink
Merge pull request #8 from cmu-db/yenjuw
Browse files Browse the repository at this point in the history
update design_doc
  • Loading branch information
yenjuw authored Jan 31, 2024
2 parents a94c9fc + d9a5129 commit 35d2c21
Showing 1 changed file with 4 additions and 5 deletions.
9 changes: 4 additions & 5 deletions doc/design_doc.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,15 @@
* The catalog service ensures data consistency and correctness by adhering to the Iceberg Catalog REST interface.
* Data durability mechanisms will be implemented to prevent data loss during restarts.
* Performance:
* Optimized data retrieval and storage strategies to minimize latency in metadata access.
* Efficient indexing mechanisms to speed up metadata search operations.
* Caching strategies to reduce redundant metadata queries and enhance overall performance.
* Optimization on data retrieval and storage strategies to minimize latency in metadata access.
* Efficient indexing mechanisms, such as Bloom filters, enhance query performance.
* Partitioning strategies facilitate data pruning and improve query execution performance.
* Engineering Complexity / Maintainability:
* The microservices architecture simplifies maintenance by encapsulating each building block.
* Centralized metadata management achieved by separating data and metadata, reducing complexity and facilitating consistent metadata handling.
* Code modularity and clear interfaces facilitate easier updates and improvements.
* Testing:
* Comprehensive testing plans cover correctness through unit tests and performance through long-running regression tests. Unit tests focus on individual components of the catalog service, while regression tests evaluate system-wide performance and stability.
* Other Implementations:
* Considered alternative implementations, including a monolithic catalog structure and alternative REST specifications.
* Chose the Iceberg Catalog REST interface due to its industry adoption, standardization, and compatibility with various systems.

## Testing Plan
Expand Down

0 comments on commit 35d2c21

Please sign in to comment.