Skip to content

proposal: NeoFS block storage format #194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

AliceInHunterland
Copy link
Contributor

No description provided.

Copy link
Contributor

@roman-khimov roman-khimov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Index files? Synchronization algorithm?

@Jim8y
Copy link
Contributor

Jim8y commented Apr 11, 2025

@AliceInHunterland please update accordingly, this is not likely can be reviewed with any useful suggestions from other coredevs since we know little about neofs, please just follow roman's suggestion

Comment on lines 38 to 82
An [https://rest.fs.neo.org/v1/objects/3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV/by_id/Fu7yQzspvLJwSGJNK64xeeyMdWXtU5B5b1es6KSxUag1 example]
of a block object is as follows:
<pre>
ID: Fu7yQzspvLJwSGJNK64xeeyMdWXtU5B5b1es6KSxUag1
CID: 3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV
Owner: NVvY1FF67XJ2GTVhy9FqiZGC4jEQtvjmHt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it correct to attach object link as an example? This object/container may be removed in future whereas we expect NEP to be consistent. @roman-khimov?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example can be a proper example with this data if needed. It doesn't imply any of the IDs are relevant for mainnet or whatever else.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted.

@AnnaShaleva
Copy link
Member

@roman-khimov, let's review one more time.

NEP: TBD
Title: NeoFS block storage format
Author: Ekaterina Pavlova <[email protected]>
Type: Standard
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

==Abstract==

This proposal outlines the specification for storing Neo blockchain blocks within
the NeoFS container.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/the//

These attributes are stored as NeoFS object metadata; the list of attributes MAY be extended.
The object MAY include additional data after the block.

An example of a [https://rest.fs.neo.org/v1/objects/3RCdP3ZubyKyo8qFeo7EJPryidTZaGCMdUjqFJaaEKBV/by_id/Fu7yQzspvLJwSGJNK64xeeyMdWXtU5B5b1es6KSxUag1 block object]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use Test cases section for it?

Blocks stored in NeoFS and those synchronized via P2P both use the standard Neo block
serialization format, ensuring compatibility across different synchronization methods.
The NeoFS container serves as verified storage for downloading blocks. Blocks should be
searched one by one directly by block attribute.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be dog slow with a per-object search, GE searches should be used. It also better be described in terms of NeoFS API (https://github.com/nspcc-dev/neofs-api/blob/master/object/service.proto).

@Jim8y
Copy link
Contributor

Jim8y commented Jul 11, 2025

LGTM. I prefer to approve the general logic as long as its correct and complete, then polish the format later.

|-
| BlockTime || Millisecond-precision timestamp when the block was created.
|-
| Timestamp || Second-precision timestamp when the block was uploaded.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the difference between Timestamp and BlockTime?

Maybe it would be better to rename Timestamp to UploadTime.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Timestamp just follows well-known NeoFS object attribute:
https://github.com/nspcc-dev/neofs-api/blob/b7548cfb16426256f0a92c6d70a3ec794a6facb0/object/types.proto#L245
(maybe it's worth mentioning this in the document)

Comment on lines +40 to +42
| BlockTime || Millisecond-precision timestamp when the block was created.
|-
| Timestamp || Second-precision timestamp when the block was uploaded.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| BlockTime || Millisecond-precision timestamp when the block was created.
|-
| Timestamp || Second-precision timestamp when the block was uploaded.
| BlockTime || Millisecond-precision unix timestamp when the block was created.
|-
| Timestamp || Second-precision unix timestamp when the block was uploaded.

timestamp is unix timestamp ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants