jito patch

only reroute if relayer connected (#123)
feat: add client tls config (#121)
remove extra val (#129)
fix clippy (#130)
copy all binaries to docker-output (#131)
Ledger tool halts at slot passed to create-snapshot (#118)
update program submodule (#133)
quick fix for tips and clearing old bundles (#135)
update submodule to new program (#136)
Improve stake-meta-generator usability (#134)
pinning submodule head (#140)
Use BundleAccountLocker when handling tip txs (#147)
Add metrics for relayer + block engine proxy (#149)
Build claim-mev in docker (#141)
Rework bundle receiving and add metrics (#152) (#154)
update submodule + dev files (#158)
Deterministically find tip amounts, add meta to stake info, and cleanup pubkey/strings in MEV tips (#159)
update jito-programs submodule (#160)
Separate MEV tip related workflow (#161)
Add block builder fee protos (#162)
fix jito programs (#163)
update submodule so autosnapshot exits out of ledger tool early (#164)
Pipe through block builder fee (#167)
pull in new snapshot code (#171)
block builder bug (#172)

Pull in new slack autosnapshot submodule (#174)

sort stake meta json and use int math (#176)

add accountsdb conn submod (#169)

Update tip distribution parameters (#177)

new submodules (#180)

Add buildkite link for jito CI (#183)

Fixed broken links to repositories (#184)

Changed from ssh to https transfer for clone

Seg/update submods (#187)

fix tests (#190)

rm geyser submod (#192)

rm dangling geyser references (#193)

fix syntax err (#195)

use deterministic req ids in batch calls (#199)

update jito-programs

revert cargo

update Cargo lock

update with path fix

fix cargo

update autosnapshot with block lookback (#201)

[JIT-460] When claiming mev tips, skip accounts that won't have min rent exempt amount after claiming (#203)

Add logging for sol balance desired (#205)

* add logging

* add logging

* update msg

* tweak vars

update submodule (#204)

use efficient data structures when calling batch_simulate_bundles (#206)

[JIT-504] Add low balance check in uploading merkle roots (#209)

add config to simulate on top of working bank (#211)

rm frozen bank check

simulate_bundle rpc bugfixes (#214)

rm frozen bank check in simulate_bundle rpc method

[JIT-519] Store ClaimStatus address in merkle-root-json (#210)

* add files

* switch to include bump

update submodule (#217)

add amount filter (#218)

update autosnapshot (#222)

Print TX error in Bundles (#223)

add new args to support single relayer and block-engine endpoints (#224)

point to new jito-programs submod and invoke updated init tda instruction (#228)

fix clippy errors (#230)

fix validator start scripts (#232)

Point README to gitbook (#237)

use packaged cargo bin to build (#239)

Add validator identity pubkey to StakeMeta (#226)

The vote account associated with a validator is not a permanent link, so log the validator identity as well.

bugfix: conditionally compile with debug flags (#240)

Seg/tip distributor master (#242)

* validate tree nodes

* fix unit tests

* pr feedback

* bump jito-programs submod

Simplify bootstrapping (#241)

* startup without precompile

* update spacing

* use release mode

* spacing

fix validation

rm validation skip

Account for block builder fee when generating excess tip balance (#247)

Improve docker caching

delay constructing claim mev txs (#253)

fix stake meta tests from bb fee (#254)

fix tests

Buffer bundles that exceed cost model (#225)

* buffer bundles that exceed cost model

clear qos failed bundles buffer if not leader soon (#260)

update Cargo.lock to correct solana versions in jito-programs submodule (#265)

fix simulate_bundle client and better error handling (#267)

update submod (#272)

Preallocate Bundle Cost (#238)

fix Dockerfile (#278)

Fix Tests (#279)

Fix Tests (#281)

* fix tests

update jito-programs submod (#282)

add reclaim rent workflow (#283)

update jito-programs submod

fix clippy errs

rm wrong assertion and swap out file write fn call (#292)

Remove security.md (#293)

demote frequent relayer_stage-stream_error to warn (#275)

account for case where TDA exists but not allocated (#295)

implement better retries for tip-distributor workflows (#297)

limit number of concurrent rpc calls (#298)

Discard Empty Packet Batches (#299)

Identity Hotswap (#290)

small fixes (#305)

Set backend config from admin rpc (#304)

Admin Shred Receiver Change (#306)

Seg/rm bundle UUID (#309)

Fix github workflow to recursively clone (#327)

Add recursive checkout for downstream-project-spl.yaml (#341)

Use cluster info functions for tpu (#345)

Use git rev-parse for git sha

Remove blacklisted tx from message_hash_to_transaction (#374)

Updates bootstrap and start scripts needed for local dev. (#384)

Remove Deprecated Cli Args (#387)

Master Rebase

improve simulate_bundle errors and response (#404)

derive Clone on accountoverrides (#416)

Add upsert to AccountOverrides (#419)

update jito-programs (#430)

[JIT-1661] Faster Autosnapshot (#436)

Reverts simulate_transaction result calls to upstream (#446)

Don't unlock accounts in TransactionBatches used during simulation (#449)

first pass at wiring up jito-plugin (#428)

[JIT-1713] Fix bundle's blockspace preallocation (#489)

[JIT-1708] Fix TOC TOU condition for relayer and block engine config (#491)

[JIT-1710] - Optimize Bundle Consumer Checks (#490)

Add Blockhash Metrics to Bundle Committer (#500)

add priority fee ix to mev-claim (#520)

Update Autosnapshot (#548)

Run MEV claims + reclaiming rent-exempt amounts in parallel. (#582)

Update CI (#584)
- Add recursive submodule checkouts.
- Re-add solana-secondary step

Add more release fixes (#585)

Fix more release urls (#588)

[JIT-1812] Fix blocking mutexs (#495)

 [JIT-1711] Compare the unprocessed transaction storage BundleStorage against a constant instead of VecDeque::capacity() (#587)

Automatically rebase Jito-Solana on a periodic basis. Send message on slack during any failures or success.

Fix periodic rebase #594

Fixes the following bugs in the periodic rebase:
Sends multiple messages on failure instead of one
Cancels entire job if one branch fails

Ignore buildkite curl errors for rebasing and try to keep curling until job times out (#597)

Sleep longer waiting for buildkite to start (#598)

correctly initialize account overrides (#595)

Fix: Ensure set contact info to UDP port instead of QUIC (#603)

Add fast replay branch to daily rebase (#607)

take a snapshot of all bundle accounts before sim (#13) (#615)

update jito-programs submodule

.dockerignore

@@ -0,0 +1,9 @@
+.dockerignore
+.git/
+.github/
+.gitignore
+.idea/
+README.md
+Dockerfile
+f
+target/

.github/dependabot.yml

@@ -2,15 +2,15 @@
 # package ecosystems to update and where the package manifests are located.
 # Please see the documentation for all configuration options:
 # https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
-
-version: 2
-updates:
-- package-ecosystem: cargo
-  directory: "/"
-  schedule:
-    interval: daily
-    time: "01:00"
-    timezone: America/Los_Angeles
-  #labels:
-  #  - "automerge"
-  open-pull-requests-limit: 0
+#
+#version: 2
+#updates:
+#- package-ecosystem: cargo
+#  directory: "/"
+#  schedule:
+#    interval: daily
+#    time: "01:00"
+#    timezone: America/Los_Angeles
+#  #labels:
+#  #  - "automerge"
+#  open-pull-requests-limit: 0

.github/workflows/cargo.yml

@@ -34,6 +34,8 @@ jobs:
     runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v4
+        with:
+          submodules: 'recursive'
 
       - uses: mozilla-actions/[email protected]
         with:
@@ -56,6 +58,8 @@ jobs:
     runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v4
+        with:
+          submodules: 'recursive'
 
       - uses: mozilla-actions/[email protected]
         with:

.github/workflows/changelog-label.yml

@@ -13,6 +13,7 @@ jobs:
     - uses: actions/checkout@v3
       with:
         fetch-depth: 0
+        submodules: 'recursive'
     - name: Check if changes to CHANGELOG.md
       shell: bash
       env:

.github/workflows/client-targets.yml

@@ -32,6 +32,8 @@ jobs:
     runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v3
+        with:
+          submodules: 'recursive'
 
       - run: cargo install [email protected]
 
@@ -56,6 +58,8 @@ jobs:
     runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v3
+        with:
+          submodules: 'recursive'
 
       - name: Setup Rust
         run: |

.github/workflows/crate-check.yml

@@ -18,6 +18,7 @@ jobs:
       - uses: actions/checkout@v3
         with:
           fetch-depth: 0
+          submodules: 'recursive'
 
       - name: Get commit range (push)
         if: ${{ github.event_name == 'push' }}

.github/workflows/docs.yml

@@ -22,6 +22,7 @@ jobs:
         uses: actions/checkout@v3
         with:
           fetch-depth: 0
+          submodules: 'recursive'
 
       - name: Get commit range (push)
         if: ${{ github.event_name == 'push' }}
@@ -77,6 +78,8 @@ jobs:
     steps:
       - name: Checkout
         uses: actions/checkout@v3
+        with:
+          submodules: 'recursive'
 
       - name: Setup Node
         uses: actions/setup-node@v3

.github/workflows/downstream-project-anchor.yml

@@ -45,6 +45,8 @@ jobs:
         version: ["v0.29.0"]
     steps:
       - uses: actions/checkout@v3
+        with:
+          submodules: 'recursive'
 
       - shell: bash
         run: |

.github/workflows/downstream-project-spl.yml

@@ -41,6 +41,8 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v3
+        with:
+          submodules: 'recursive'
 
       - shell: bash
         run: |
@@ -90,6 +92,8 @@ jobs:
           ]
     steps:
       - uses: actions/checkout@v3
+        with:
+          submodules: 'recursive'
 
       - shell: bash
         run: |
@@ -141,6 +145,8 @@ jobs:
 
     steps:
       - uses: actions/checkout@v3
+        with:
+          submodules: 'recursive'
 
       - shell: bash
         run: |

.github/workflows/increment-cargo-version-on-release.yml

@@ -11,6 +11,8 @@ jobs:
     steps:
       - name: Checkout Repository
         uses: actions/checkout@v3
+        with:
+          submodules: 'recursive'
 
       # This script confirms two assumptions:
       # 1) Tag should be branch.<patch_version>

.github/workflows/rebase.yaml

@@ -0,0 +1,181 @@
+# This workflow runs a periodic rebase process, pulling in updates from an upstream repository
+# The workflow for rebasing a jito-solana branch to a solana labs branch locally is typically:
+# $ git checkout v1.17
+# $ git pull --rebase # --rebase needed locally
+# $ git branch -D lb/v1.17_rebase # deletes branch from last v1.17 rebase
+# $ git checkout -b lb/v1.17_rebase
+# $ git fetch upstream
+# $ git rebase upstream/v1.17 # rebase + fix merge conflicts
+# $ git rebase --continue
+# $ git push origin +lb/v1.17_rebase # force needed to overwrite remote. wait for CI, fix if any issues
+# $ git checkout v1.17
+# $ git reset --hard lb/v1.17_rebase
+# $ git push origin +v1.17
+#
+# This workflow automates this process, with periodic status updates over slack.
+# It will also run CI and wait for it to pass before performing the force push to v1.17.
+# In the event there's a failure in the process, it's reported to slack and the job stops.
+
+name: "Rebase jito-solana from upstream solana-labs/solana"
+
+on:
+  #  push:
+  schedule:
+    - cron: "30 18 * * 1-5"
+
+jobs:
+  rebase:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        include:
+          - branch: master
+            rebase: upstream/master
+          - branch: v1.18
+            rebase: upstream/v1.18
+          - branch: v1.17
+            rebase: upstream/v1.17
+            # note: this will always be a day behind because we're rebasing from the previous day's rebase
+            # and NOT upstream
+          - branch: v1.17-fast-replay
+            rebase: origin/v1.17
+      fail-fast: false
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          ref: ${{ matrix.branch }}
+          submodules: recursive
+          fetch-depth: 0
+          token: ${{ secrets.JITO_SOLANA_RELEASE_TOKEN }}
+      - name: Add upstream
+        run: git remote add upstream https://github.com/solana-labs/solana.git
+      - name: Fetch upstream
+        run: git fetch upstream
+      - name: Fetch origin
+        run: git fetch origin
+      - name: Set REBASE_BRANCH
+        run: echo "REBASE_BRANCH=ci/nightly/${{ matrix.branch }}/$(date +'%Y-%m-%d-%H-%M')" >> $GITHUB_ENV
+      - name: echo $REBASE_BRANCH
+        run: echo $REBASE_BRANCH
+      - name: Create rebase branch
+        run: git checkout -b $REBASE_BRANCH
+      - name: Setup email
+        run: |
+          git config --global user.email "[email protected]"
+          git config --global user.name "Jito Infrastructure"
+      - name: Rebase
+        id: rebase
+        run: git rebase ${{ matrix.rebase }}
+      - name: Send warning for rebase error
+        if: failure() && steps.rebase.outcome == 'failure'
+        uses: slackapi/[email protected]
+        with:
+          payload: |
+            {
+              "text": "Nightly rebase on branch ${{ matrix.branch }}\nStatus: Rebase failed to apply cleanly"
+            }
+        env:
+          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
+      - name: Check if rebase applied
+        id: check_rebase_applied
+        run: |
+          PRE_REBASE_SHA=$(git rev-parse ${{ matrix.branch }})
+          POST_REBASE_SHA=$(git rev-parse HEAD)
+          if [ "$PRE_REBASE_SHA" = "$POST_REBASE_SHA" ]; then
+            echo "No rebase was applied, exiting..."
+            exit 1
+          else
+            echo "Rebase applied successfully."
+          fi
+      - name: Send warning for rebase error
+        if: failure() && steps.check_rebase_applied.outcome == 'failure'
+        uses: slackapi/[email protected]
+        with:
+          payload: |
+            {
+              "text": "Nightly rebase on branch ${{ matrix.branch }}\nStatus: Rebase not needed"
+            }
+        env:
+          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
+      - name: Set REBASE_SHA
+        run: echo "REBASE_SHA=$(git rev-parse HEAD)" >> $GITHUB_ENV
+      - name: Push changes
+        uses: ad-m/github-push-action@master
+        with:
+          github_token: ${{ secrets.GITHUB_TOKEN }}
+          branch: ${{ env.REBASE_BRANCH }}
+      - name: Wait for buildkite to start build
+        run: sleep 300
+      - name: Wait for buildkite to finish
+        id: wait_for_buildkite
+        timeout-minutes: 300
+        run: |
+          while true; do
+            response=$(curl -s -f -H "Authorization: Bearer ${{ secrets.BUILDKITE_TOKEN }}" "https://api.buildkite.com/v2/organizations/jito/pipelines/jito-solana/builds?commit=${{ env.REBASE_SHA }}")
+            if [ $? -ne 0 ]; then
+              echo "Curl request failed."
+              exit 1
+            fi
+
+            state=$(echo $response | jq --exit-status -r '.[0].state')
+            echo "Current build state: $state"
+
+            # Check if the state is one of the finished states
+            case $state in
+              "passed"|"finished")
+                echo "Build finished successfully."
+                exit 0
+              ;;
+              "canceled"|"canceling"|"not_run")
+                # ignoring "failing"|"failed" because flaky CI, can restart and hope it finishes or times out
+                echo "Build failed or was cancelled."
+                exit 2
+              ;;
+            esac
+
+          sleep 30
+          done
+      - name: Send failure update
+        uses: slackapi/[email protected]
+        if: failure() && steps.wait_for_buildkite.outcome == 'failure'
+        with:
+          payload: |
+            {
+              "text": "Nightly rebase on branch ${{ matrix.branch }}\nStatus: CI failed\nBranch: ${{ env.REBASE_BRANCH}}\nBuild: https://buildkite.com/jito/jito-solana/builds?commit=${{ env.REBASE_SHA }}"
+            }
+        env:
+          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
+      # check to see if different branch since CI build can take awhile and these steps are not atomic
+      - name: Fetch the latest remote changes
+        run: git fetch origin ${{ matrix.branch }}
+      - name: Check if origin HEAD has changed from the beginning of the workflow
+        run: |
+          LOCAL_SHA=$(git rev-parse ${{ matrix.branch }})
+          ORIGIN_SHA=$(git rev-parse origin/${{ matrix.branch }})
+          if [ "$ORIGIN_SHA" != "$LOCAL_SHA" ]; then
+            echo "The remote HEAD of ${{ matrix.branch }} does not match the local HEAD of ${{ matrix.branch }} at the beginning of CI."
+            echo "origin sha: $ORIGIN_SHA"
+            echo "local sha: $LOCAL_SHA"
+            exit 1
+          else
+            echo "The remote HEAD matches the local REBASE_SHA at the beginning of CI. Proceeding."
+          fi
+      - name: Reset ${{ matrix.branch }} to ${{ env.REBASE_BRANCH }}
+        run: |
+          git checkout ${{ matrix.branch }}
+          git reset --hard ${{ env.REBASE_BRANCH }}
+      - name: Push rebased %{{ matrix.branch }}
+        uses: ad-m/github-push-action@master
+        with:
+          github_token: ${{ secrets.JITO_SOLANA_RELEASE_TOKEN }}
+          branch: ${{ matrix.branch }}
+          force: true
+      - name: Send success update
+        uses: slackapi/[email protected]
+        with:
+          payload: |
+            {
+              "text": "Nightly rebase on branch ${{ matrix.branch }}\nStatus: CI success, rebased, and pushed\nBranch: ${{ env.REBASE_BRANCH}}\nBuild: https://buildkite.com/jito/jito-solana/builds?commit=${{ env.REBASE_SHA }}"
+            }
+        env:
+          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

.github/workflows/release-artifacts.yml

@@ -26,6 +26,7 @@ jobs:
         with:
           ref: master
           fetch-depth: 0
+          submodules: 'recursive'
 
       - name: Setup Rust
         shell: bash