-
Notifications
You must be signed in to change notification settings - Fork 1k
feat: add weighted rpc steering #6090
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add weighted rpc steering #6090
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds weighted random steering functionality to Graph Node's Ethereum RPC provider selection system, allowing operators to configure fractional weights (0.0-1.0) for load balancing across different RPC endpoints.
Key Changes
- Added weighted random steering configuration flag and CLI option
- Implemented weight-based provider selection using fractional values (0.0-1.0)
- Modified adapter selection logic to support both weighted and traditional selection strategies
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.
Show a summary per file
File | Description |
---|---|
node/src/opt.rs | Added CLI flag for enabling weighted RPC steering |
node/src/config.rs | Added weight field to Provider struct with validation and serde support |
node/src/network_setup.rs | Integrated weighted steering flag into network configuration |
node/src/chain.rs | Updated adapter creation to pass weight parameter |
chain/ethereum/src/network.rs | Implemented weighted selection algorithm using WeightedIndex |
node/resources/tests/full_config.toml | Updated test configuration with weight examples |
50b2cca
to
c3d0c5b
Compare
8acb42a
to
c5f3ffc
Compare
…ements This commit introduces a complete weighted load balancing system for RPC endpoints with traffic distribution based on configurable provider weights (0.0-1.0). ## Core Features ### Weighted Load Balancing Algorithm - Implements probabilistic selection using WeightedIndex from rand crate - Supports decimal weights (0.0-1.0) for precise traffic distribution - Weights are relative and don't need to sum to 1.0 (normalized internally) - Graceful fallback to random selection if weights are invalid ### Enhanced Error Handling & Resilience - Improved error retesting logic that preserves weight distribution - Error retesting now occurs AFTER weight-based selection to minimize skew - Maintains existing failover capabilities while respecting configured weights - Robust handling of edge cases (all zero weights, invalid configurations) ### Configuration & Validation - Added `weighted_rpc_steering` flag to enable/disable weighted selection - Provider weight validation ensures values are between 0.0 and 1.0 - Validation prevents all-zero weight configurations - Comprehensive configuration documentation with usage examples ## Implementation Details ### Network Layer Changes (chain/ethereum/src/network.rs) - Refactored adapter selection into modular, well-documented functions: - `select_best_adapter()`: Chooses between weighted/random strategies - `select_weighted_adapter()`: Implements WeightedIndex-based selection - `select_random_adapter()`: Enhanced random selection with error consideration - Added comprehensive inline documentation explaining algorithms - Maintains thread safety with proper Arc usage and thread-safe RNG - Added test coverage for weighted selection with statistical validation ### Configuration System (node/src/config.rs) - Extended Provider struct with f64 weight field (default: 1.0) - Added weight validation in Provider::validate() method - Added Chain-level validation to prevent all-zero weight configurations - Integrated with existing configuration validation pipeline ### CLI & Setup Integration - Added --weighted-rpc-steering command line flag (node/src/opt.rs) - Integrated weighted flag through network setup pipeline (node/src/network_setup.rs) - Updated chain configuration to pass weight values to adapters (node/src/chain.rs) ### Documentation & Examples - Added comprehensive configuration documentation in full_config.toml - Includes weight range explanation, distribution examples, and usage guidelines - Clear examples showing relative weight calculations and traffic distribution ## Technical Improvements ### Dependency Management - Updated rand dependency to use appropriate version with WeightedIndex support - Proper import paths for rand 0.9 distribution modules - Fixed compilation issues with correct trait imports (Distribution) ### Code Quality & Maintenance - Comprehensive inline documentation for all weight-related methods - Clear separation of concerns with single-responsibility functions - Maintained backward compatibility with existing random selection - Added statistical test validation for weight distribution accuracy ## Validation & Testing - Comprehensive test suite validates weight distribution over 1000 iterations - Statistical validation with 10% tolerance for weight accuracy - All existing tests continue to pass, ensuring no regression - Build verification across all affected packages ## Configuration Example ```toml weighted_rpc_steering = true [chains.mainnet] provider = [ { label = "primary", url = "http://rpc1.io", weight = 0.7 }, # 70% traffic { label = "backup", url = "http://rpc2.io", weight = 0.3 }, # 30% traffic ] ``` This implementation provides production-ready weighted load balancing with robust error handling, comprehensive validation, and excellent maintainability. 🤖 Generated with Claude Code
c5f3ffc
to
ecf7000
Compare
f3ed6a2
into
graphprotocol:feature/add-weighted-random-steering-load-balancing
Summary
This PR introduces a complete weighted load-balancing system for RPC endpoints
with traffic distribution based on configurable provider weights (0.0-1.0).
Core Features
Weighted Load Balancing Algorithm
Enhanced Error Handling & Resilience
Configuration & Validation
weighted_rpc_steering
flag to enable/disable weighted selectionImplementation Details
Network Layer Changes (chain/ethereum/src/network.rs)
select_best_adapter()
: Chooses between weighted/random strategiesselect_weighted_adapter()
: Implements WeightedIndex-based selectionselect_random_adapter()
: Enhanced random selection with error considerationConfiguration System (node/src/config.rs)
CLI & Setup Integration
Documentation & Examples
Technical Improvements
Dependency Management
Code Quality & Maintenance
Validation & Testing
Configuration Example
This implementation provides production-ready weighted load balancing with
robust error handling, comprehensive validation, and excellent maintainability.