Skip to content
/ pearl Public

Pearl Append only key-value blob storage on disk

License

Notifications You must be signed in to change notification settings

qoollo/pearl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

dfa1ac9 · Dec 18, 2023
Apr 13, 2021
Dec 18, 2023
Dec 18, 2023
Dec 12, 2023
Apr 13, 2021
Sep 16, 2021
Dec 18, 2023
Dec 18, 2023
Feb 25, 2019
Aug 16, 2021
Oct 2, 2021
Nov 20, 2019

Repository files navigation

Pearl

build tests Crate Status Docs Status

Append only key-value blob storage on disk

Table of Contents

Overview

Storage scheme

pearl storage scheme

Blob

Header

Structure:

struct Header {
    magic_byte: u64,
    version: u32,
    flags: u64,
}

Description

Field Size, B Description
magic_byte 8 marks pearl blob
version any used to check compatibility
flags 1 additional file props

Record

Header

Structure:

pub struct Header {
    magic_byte: u64,
    key: Vec<u8>,
    meta_size: u64,
    data_size: u64,
    flags: u8,
    blob_offset: u64,
    created: u64,
    data_checksum: u32,
    header_checksum: u32,
}

Description

Field Size, B Description
magic_byte 8 separates records in blob
key (any) key for record location and searching
meta_size 8 meta length
data_size 8 data length (without header)
flags 1 additional record metadata
blob_offset 8 record offset from blob start
created 8 created timestamp
data_checksum 4 data crc32 checksum (without header)
header_checksum 4 header crc32 checksum (only record header)

Rust Version

pearl works on stable rust and above

$ rustup update
$ cargo build