Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: typos in documentation files #42

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions in-progress/0003-native-merkle-trees.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,21 @@ This design attempts to solve the problem of slow sync and merkle tree insertion

## Introduction

We require high performance merkle tree implementations both to ensure nodes can stay synched to the network and sequencers/provers can advance the state as required to build blocks. Our cuirrent TS implementations are limited in their single-threaded nature and the unavoidable constraint of have to repeatedly call into WASM to perform a hash operation.
We require high performance merkle tree implementations both to ensure nodes can stay synched to the network and sequencers/provers can advance the state as required to build blocks. Our current TS implementations are limited in their single-threaded nature and the unavoidable constraint of have to repeatedly call into WASM to perform a hash operation.

Some analysis of the quantity of hashing and the time required can be found [here](https://hackmd.io/@aztec-network/HyfTK9U5a?type=view).

This design proposes the creation of a set of multi-threaded merkle tree implementations in C++ using LMDB. It builds upon some previous prototyping to develop concurrent indexed tree insertions.

## Implementation

There are many parts to this design, we will walk through them individiually and discuss the choices made at each stage.
There are many parts to this design, we will walk through them individually and discuss the choices made at each stage.

### Overall Architecture

A new C++ binary, World State, will be created that will be started by the node software. It will be configured with the location in which Merkle Tree data should be stored. It will then accept and respond with msgpack-ed messages over one or more streams. The initial implementation will simply used stdio, but this will be absrtacted such that this could be replaced by other stream-based mechanisms.
A new C++ binary, World State, will be created that will be started by the node software. It will be configured with the location in which Merkle Tree data should be stored. It will then accept and respond with msgpack-ed messages over one or more streams. The initial implementation will simply used stdio, but this will be abstracted such that this could be replaced by other stream-based mechanisms.

To interface with the World State, an abstraction will be created at the `MerkleTreeDb` level. This accurately models the scope of functionality provided by the binary as owner of all the trees. It was considered that the abstraction could sit at the level of individual trees, but this creates difficulty whan we want to send an entire block to the World State to be inserted. This is an important use case as synching entire blocks is where signifcant performance optimisations can be made.
To interface with the World State, an abstraction will be created at the `MerkleTreeDb` level. This accurately models the scope of functionality provided by the binary as owner of all the trees. It was considered that the abstraction could sit at the level of individual trees, but this creates difficulty when we want to send an entire block to the World State to be inserted. This is an important use case as synching entire blocks is where significant performance optimisations can be made.


``` TS
Expand All @@ -47,7 +47,7 @@ An abstract factory will then be created to construct the appropriate concrete t

### Interface

The interface will be an asynchronous message based communication protocol. Each message is provided with meta data uniquely identiying it and is responded to inidividually. It is not necessary to wait for a response to a message before sending a subsequent message. A simple message specification will be created, some examples of which are shown here:
The interface will be an asynchronous message based communication protocol. Each message is provided with meta data uniquely identifying it and is responded to individually. It is not necessary to wait for a response to a message before sending a subsequent message. A simple message specification will be created, some examples of which are shown here:

``` C++
enum WorldStateMsgTypes {
Expand All @@ -61,7 +61,7 @@ enum WorldStateMsgTypes {

struct MsgHeader {
uint32_t messageId; // Unique Id for the message
uint32_t requestId; // Id of the message this is responding too (may not be used)
uint32_t requestId; // Id of the message this is responding to (may not be used)

MSGPACK_FIELDS(messageId, requestId);

Expand Down Expand Up @@ -140,11 +140,11 @@ Examples of reads are requesting sibling paths, state roots etc.

#### Updates

As a sequencer/prover inserts transaction side-effects, the resulting new state is computed and cached in memory. This allows for the seperation of `committed` and `uncommitted` reads and the easy rolling back of unsuccessful blocks.
As a sequencer/prover inserts transaction side-effects, the resulting new state is computed and cached in memory. This allows for the separation of `committed` and `uncommitted` reads and the easy rolling back of unsuccessful blocks.

#### Commits

When a block settles, the node performs a commit. It verifies any uncommitted state it may have against that published on chain to determine if that state is canonical. If it is not, the `uncommitted` state is dicarded and the node perform an `Update` operation using the newly published side effects.
When a block settles, the node performs a commit. It verifies any uncommitted state it may have against that published on chain to determine if that state is canonical. If it is not, the `uncommitted` state is discarded and the node perform an `Update` operation using the newly published side effects.

Once the node has the correct `uncommitted` state, it commits that state to disk. This is the only time that a write transaction is required against the database.

Expand All @@ -154,7 +154,7 @@ The `Update` operation involves inserting side-effects into one or more trees. D

#### Append Only

Append only trees don't support the updating of any leaves. New leaves are inserted at the right-most location and nodes above these are updated to reflect their newly hashed values. Optimisation here is simply a case of dividing the set of leaves into smaller batches and hashing each of these batches into a sub-tree in seperate threads. Finally, the roots are used to build the sub-tree on top before hashing to the root of the main tree.
Append only trees don't support the updating of any leaves. New leaves are inserted at the right-most location and nodes above these are updated to reflect their newly hashed values. Optimisation here is simply a case of dividing the set of leaves into smaller batches and hashing each of these batches into a sub-tree in separate threads. Finally, the roots are used to build the sub-tree on top before hashing to the root of the main tree.

#### Indexed Tree

Expand All @@ -163,7 +163,7 @@ Indexed Trees require significantly more hashing than append only trees. In fact
For each leaf being inserted:

1. Identify the location of the leaf whose value immediately precedes that being inserted.
2. Retrieve the sibling path of the preceeding leaf before any modification.
2. Retrieve the sibling path of the preceding leaf before any modification.
3. Set the 'next' value and index to point to the leaf being inserted.
4. Set the 'next' value and index of the leaf being inserted to the leaf previously pointed to by the leaf just updated.
5. Re-hash the updated leaf and update the leaf with this hash, requiring the tree to be re-hashed up to the root.
Expand All @@ -179,7 +179,7 @@ For example, we have a depth 3 Indexed Tree and 2 leaves to insert. The first re

In the above example, Thread 2 will follow Thread 1 up the tree, providing a degree of concurrency to the update operation. Obviously, this example if limited, in a 40 depth tree it is possible to have many threads working concurrently to build the new state without collision.

In this concurrent model, each thread would use it's own single read transaction to retrieve `committed` state and all new `uncommitted` state is written to the cache in a lock free manner as every thread is writing to a different level of the tree.
In this concurrent model, each thread would use its own single read transaction to retrieve `committed` state and all new `uncommitted` state is written to the cache in a lock free manner as every thread is writing to a different level of the tree.

## Change Set

Expand Down Expand Up @@ -209,4 +209,4 @@ As the World State is used heavily in all operations, we will gain confidence th

## Prototypes

Areas of this work have been prototyped already. The latest being [here](https://github.com/AztecProtocol/aztec-packages/pull/7037).
Areas of this work have been prototyped already. The latest being [here](https://github.com/AztecProtocol/aztec-packages/pull/7037).
4 changes: 2 additions & 2 deletions in-progress/7520-testnet-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ A deployment of the Aztec Network includes several contracts running on L1.

The Test Token (TST) will be an ERC20 token that will be used to pay for transaction fees on the Aztec Network.

It will also used on L1 as part of the validator selection process.
It will be also used on L1 as part of the validator selection process.

Protocol incentives are paid out in TST.

Expand Down Expand Up @@ -239,7 +239,7 @@ Each slot in an epoch will be randomly assigned to a validator in the committee.

## Fees

Every transaction in the Aztec Network has a fee associated with it. The fee is payed in TST which has been bridged to L2.
Every transaction in the Aztec Network has a fee associated with it. The fee is paid in TST which has been bridged to L2.

Transactions consume gas. There are two types of gas:

Expand Down
8 changes: 4 additions & 4 deletions in-progress/8131-forced-inclusion.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,25 +25,25 @@ We use a similar definition of a censored transaction as the one outlined in [Th

> _"a transaction is censored if a third party can prevent it from achieving its goal."_

For a system as the ours, even with the addition of the [based fallback mechanism](8404-based-fallback.md), there is a verity of methods a censor can use to keep a transaction out.
For a system as the ours, even with the addition of the [based fallback mechanism](8404-based-fallback.md), there is a verify of methods a censor can use to keep a transaction out.

The simplest is that the committee simply ignore the transaction.
In this case, the user would need to wait until a more friendly committee comes along, and he is fully dependent on the consensus mechanism of our network being honest.

Note, that there is a case where a honest committee would ignore your transaction, you might be paying an insufficient fee.
This case should be easily solved, pay up you cheapskate!

But lets assume that this is not the case you were in, you paid a sufficient fee and they keep excluding it.
But let's assume that this is not the case you were in, you paid a sufficient fee and they keep excluding it.

In rollups such as Arbitrum and Optimism both have a mechanism that allow the user to take his transactions directly to the base layer, and insert it into a "delayed" queue.
After some delay have passed, the elements of the delayed queue can be forced into the ordering, and the sequencer is required to include it, or he will enter a game of fraud or not where he already lost.

The delay is introduced into the system to ensure that the forced inclusions cannot be used as a way to censor the rollup itself.

Curtesy of [The Hand-off Problem](https://blog.init4.technology/p/the-hand-off-problem), we borrow this great figure:
Courtesy of [The Hand-off Problem](https://blog.init4.technology/p/the-hand-off-problem), we borrow this great figure:
![There must be a hand-off from unforced to forced inclusion.](https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F35f3bee4-0a8d-4c40-9c8d-0a145be64d87_3216x1243.png)

The hand-off is here the point where we go from the ordering of transactions that the sequencer/proposer is freely choosing and the transactions that they are forced to include specifically ordered.
The hand-off is the point where we go from the ordering of transactions that the sequencer/proposer is freely choosing and the transactions that they are forced to include specifically ordered.
For Arbitrum and Optimisms that would be after this delay passes and it is forced into the ordering.

By having this forced insertion into the ordering the systems can get the same **inclusion** censorship resistance as their underlying baselayer.
Expand Down
6 changes: 3 additions & 3 deletions in-progress/8404-based-fallback.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ Based fallback provide us with these guarantees, assuming that the base-layer is

The Aztec network is a Rollup on Ethereum L1, that have its own consensus layer to support build-ahead.
Since the consensus layer is only there for the ability to have fast ledger growth, e.g., block times that are smaller than proving times, it should not be strictly required for the state of the chain to be progressed.
Therefore, we provide a fallback mechanism to handle the case where the committee fail to perform their duties - allowing anyone to grow the ledger.
Therefore, we provide a fallback mechanism to handle the case where the committee fails to perform their duties - allowing anyone to grow the ledger.

The nature in which they could fail to perform their duties varies widely.
It could be an attack to try and censor the network, or it might be that a majority of the network is running a node that corrupts its state the 13 of August every year as an homage to the Aztec empire.
It could be an attack to try and censor the network, or it might be that a majority of the network is running a node that corrupts its state the 13th of August every year as an homage to the Aztec empire.

Nevertheless, we need a mechanism to ensure the liveness of the chain, even if slower, in these events.

Expand Down Expand Up @@ -49,7 +49,7 @@ The based fallback is expected to have a significantly worse user experience sin

Because of this, we really do not want to enter the fallback too often, but we need to make it happen often enough that it is usable.
For applications that are dependent on users acting based on external data or oracle data, a low bar to enter based mode can be desirable since it will mean that they would be able to keep running more easily.
Lending and trading falls into these catagories as they usually depend on external data sources, e.g., prices of CEX'es influence how people use a trading platform and lending platforms mostly use oracles to get prices.
Lending and trading falls into these categories as they usually depend on external data sources, e.g., prices of CEX'es influence how people use a trading platform and lending platforms mostly use oracles to get prices.

We suggest defining the time where Based fallback can be entered to be $T_{\textsf{fallback}, \textsf{enter}}$ after the last proven block.
The minimum acceptable value for $T_{\textsf{fallback}, \textsf{enter}}$ should therefore be if a committee fails to performs its proving duties as specified as a full epoch $E$ in https://github.com/AztecProtocol/engineering-designs/pull/22 * 2.
Expand Down
Loading