Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use prover client from env #232

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

smuu
Copy link
Collaborator

@smuu smuu commented Feb 7, 2025

Summary by CodeRabbit

  • New Features

    • Introduced dynamic, environment-based configuration for client setup.
    • Enhanced logging during key operations for improved traceability.
  • Refactor

    • Streamlined the client initialization and proof generation process to a unified approach.
  • Tests

    • Updated test configurations to align with the new environment-based setup.
  • Chores

    • Added environment variable exports for logging and prover configuration in the Justfile.

Copy link

vercel bot commented Feb 7, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
prism ⬜️ Ignored (Inspect) Visit Preview Mar 3, 2025 8:26am

Copy link
Contributor

coderabbitai bot commented Feb 7, 2025

Walkthrough

This pull request updates the instantiation and usage of the ProverClient across several modules. The changes transition from a builder or mock-based initialization to an environment-driven configuration using from_env(), with added logging of the SP1_PROVER variable. In addition, the prover module now replaces CpuProver with EnvProver and unifies the proof generation process. The data availability layer’s logging has been enhanced to include more detailed information about epoch submissions.

Changes

File(s) Change Summary
crates/cli/src/main.rs, crates/tests/src/lib.rs Switched from a builder/mocked instantiation of ProverClient to using from_env(), with added logging for the SP1_PROVER environment variable.
crates/da/.../full_node.rs Enhanced logging in the submit_finalized_epoch method to show the data size in bytes and the complete epoch object.
crates/node_types/prover/src/prover/mod.rs Replaced CpuProver with EnvProver, updated the prover_client field type, removed conditional compilation, and streamlined the proof generation to always use the groth16 method.
justfile Introduced environment variable exports for RUST_LOG and SP1_PROVER across testing sections to enhance logging and configuration.

Sequence Diagram(s)

sequenceDiagram
    participant CLI as Main Function
    participant ENV as Environment
    participant PC as ProverClient
    CLI->>ENV: Read SP1_PROVER variable
    CLI->>PC: Invoke from_env()
    PC-->>CLI: Return configured client
    CLI->>CLI: Log SP1_PROVER value and proceed
Loading
sequenceDiagram
    participant Prover as Prover Module
    participant ENV as Environment
    participant PC as EnvProver
    Prover->>ENV: Retrieve SP1_PROVER config
    Prover->>PC: Initialize via from_env()
    PC-->>Prover: Return EnvProver instance
    Prover->>PC: Invoke groth16-based prove()
    PC-->>Prover: Provide proof result
Loading

Possibly related PRs

  • refactor(prism-bin): renaming to prism-cli, refactoring args/config, and adding full-node type #145: The changes in the main PR, which involve modifications to the ProverClient and its instantiation method, are related to the changes in the retrieved PR, as both involve updates to the configuration and usage of the Prover functionality within the CLI context.
  • refactor(prism-bin): extracting tests #139: The changes in the main PR regarding the ProverClient instantiation and logging enhancements are related to the modifications in the test setup for ProverClient in the retrieved PR, as both involve transitioning from a mock configuration to an environment-based configuration.
  • feat(prover): more convenient user API #229: The changes in the main PR regarding the ProverClient instantiation and configuration are related to the modifications in the mod.rs file of the retrieved PR, as both involve alterations to the Prover struct and its client setup.

Suggested reviewers

  • distractedm1nd

Poem

In a burrow of code I hop with delight,
Where environment variables guide me through the night.
From builder to env-based, the changes alight,
Logging each secret, shining ever so bright.
I nibble on proofs with a groovy byte,
Celebrating a smoother path that’s just right! 🐇💻

✨ Finishing Touches
  • 📝 Generate Docstrings

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@smuu smuu changed the title feat: allow keystore as path feat: use prover client from env Feb 7, 2025
@smuu smuu marked this pull request as ready for review February 27, 2025 15:03
@@ -40,7 +40,9 @@ async fn test_light_client_prover_talking() -> Result<()> {
);
pretty_env_logger::init();

let prover_client = ProverClient::builder().mock().build();
std::env::set_var("SP1_PROVER", "mock");
Copy link
Contributor

@jns-ps jns-ps Feb 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably shouldn't be using std::env::set_var any more in the code. In Rust version 2024 it will be an "unsafe" operation.
Can we move this to the Justfile?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

smuu added 2 commits February 27, 2025 16:25
Signed-off-by: Smuu <[email protected]>
Copy link
Contributor

@sebasti810 sebasti810 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
crates/node_types/prover/src/prover/mod.rs (2)

140-140: Simplified prover client initialization using environment variables

The initialization has been simplified to use ProverClient::from_env() instead of conditional compilation blocks. This is a good improvement for configurability, but it lacks error handling or logging for cases where environment variables are missing or misconfigured.

Consider adding error handling and logging for the environment variable configuration:

- let prover_client = ProverClient::from_env();
+ info!("Initializing prover client from environment variables");
+ let prover_client = match ProverClient::from_env() {
+     Ok(client) => {
+         info!("Using prover: {}", std::env::var("SP1_PROVER").unwrap_or_else(|_| "default".to_string()));
+         client
+     },
+     Err(e) => {
+         error!("Failed to initialize prover client from environment: {}", e);
+         return Err(anyhow!("Failed to initialize prover client from environment: {}", e));
+     }
+ };

1-562: Document the environment variable requirements

The transition to environment-based configuration is significant, yet there is no documentation in the code about what environment variables are required and what their valid values should be.

Add a documentation comment at the module level that explains the environment variables used by this module, particularly SP1_PROVER. This will help users understand the configuration options and troubleshoot issues.

1
+ // This module uses the following environment variables:
+ // - SP1_PROVER: Specifies the prover implementation to use. 
+ //   Possible values include: "cpu", "mock", etc. 
+ //   See sp1_sdk documentation for details.
+ //
mod timer;
2
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 70c72ed and 6682866.

📒 Files selected for processing (2)
  • crates/node_types/prover/src/prover/mod.rs (3 hunks)
  • crates/tests/src/lib.rs (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • crates/tests/src/lib.rs
⏰ Context from checks skipped due to timeout of 90000ms (6)
  • GitHub Check: unused dependencies
  • GitHub Check: unit-test
  • GitHub Check: integration-test
  • GitHub Check: build-and-push-image
  • GitHub Check: clippy
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (4)
crates/node_types/prover/src/prover/mod.rs (4)

35-35: Updated import statement to use EnvProver from sp1_sdk

The import statement has been updated to include EnvProver instead of CpuProver, which aligns with the transition to using environment-based configuration for the prover client.


117-117: Field type updated to use EnvProver instead of CpuProver

The prover_client field type has been changed from Arc<RwLock<CpuProver>> to Arc<RwLock<EnvProver>>. This change correctly reflects the shift from a CPU-specific prover to an environment-determined prover.


1-562:

❓ Verification inconclusive

Verify integration with external systems

The change from a CPU-specific prover to an environment-determined prover could affect interactions with external systems or services. Ensure that all relevant components are updated to handle this change.


🏁 Script executed:

#!/bin/bash
# Check if there are any other places in the codebase that might need updates
# due to the change from CpuProver to EnvProver

# Look for places that use CpuProver
rg "CpuProver" --glob "**/*.rs"

# Look for places that might be referencing prover types
rg "Prover(Client|Type)" --glob "**/*.rs"

# Check for environment variable usage in other files
rg "SP1_PROVER" --glob "**/*.rs"

Length of output: 1144


Integration with external systems verified – Review external dependencies and confirm environment-specific configurations

  • Our codebase search did not reveal any lingering references to the old CPU-specific prover; all instantiations now correctly rely on ProverClient::from_env(), especially in crates/node_types/prover.
  • References in other modules (such as in CLI and tests) also consistently use the environment-based prover.
  • Note that crates/zk/sp1-script uses ProverClient::builder().cuda().build(), so please verify that this usage aligns with the intended environment-determined behavior.
  • Additionally, the logging of the SP1_PROVER environment variable in crates/cli/src/main.rs indicates that environment-based configuration is in use.

Please ensure through integration testing that all external services and clients (including those with CUDA-specific builds) correctly handle this change.


410-413:

❓ Verification inconclusive

Unified proof generation approach

The code now uses a consistent approach for proof generation with .groth16().run(), without conditional compilation for different proving methods. This simplifies the codebase and makes it more maintainable.

However, ensure that your integration tests verify that this approach still covers all necessary proof generation scenarios.


🏁 Script executed:

#!/bin/bash
# Check if there are any tests that might be affected by the proof generation change

# Look for tests related to proof generation
rg -A 2 -B 2 "test.*prove" --glob "**/*.rs"

# Look for any code paths that might be affected by the removal of conditional compilation
rg -A 2 -B 2 "cfg\(feature" --glob "**/prover/**.rs"

Length of output: 2996


Unified proof generation approach verified – please double-check integration coverage

The code in crates/node_types/prover/src/prover/mod.rs now consistently uses .groth16().run(), eliminating earlier conditional compilation paths. Our search showed that proof-related tests (e.g., in crates/node_types/prover/tests.rs) are present, and no conditional compilation code is found in the prover module.

Please ensure that your integration tests fully cover all proof generation scenarios, as even subtle edge cases might otherwise be missed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants