Skip to content

Fix: remove I/O Blocking code on message processing parts #820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: 1yam-sync-db-to-async
Choose a base branch
from

Conversation

1yam
Copy link
Member

@1yam 1yam commented Jul 8, 2025

This PR remove I/O blocking operations on parts used by message processing to bring back message parallelization

Related Clickup or Jira tickets : ALEPH-526

Self proofreading checklist

  • Is my code clear enough and well documented
  • Are my files well typed
  • New translations have been added or updated if new strings have been introduced in the frontend
  • Database migrations file are included
  • Are there enough tests
  • Documentation has been included (for new feature)

Changes

This pull request introduces significant changes to improve asynchronous handling across multiple modules, enhance efficiency, and ensure non-blocking operations. Key updates include replacing synchronous file and I/O operations with asynchronous alternatives, introducing semaphores to limit concurrent requests, and refactoring code to use shared aiohttp.ClientSession instances for HTTP requests.

Asynchronous Enhancements:

HTTP Request Optimization:

  • src/aleph/services/p2p/http.py: Refactored api_get_request to use a shared aiohttp.ClientSession instead of maintaining separate session instances, improving resource management. Added semaphores to limit concurrent calls in get_peer_hash_content and request_hash. [1] [2] [3]

  • src/aleph/services/p2p/jobs.py: Introduced request_version to handle peer version checks using aiohttp.ClientSession and asynchronous tasks. Refactored tidy_http_peers_job to utilize request_version for better scalability. [1] [2]

Code Cleanup:

These changes collectively enhance the system's scalability, responsiveness, and maintainability by leveraging asynchronous programming patterns effectively.

How to test

This PR doesn't add any breaking change, it's just running things in Non I/O blockign way

@1yam
Copy link
Member Author

1yam commented Jul 8, 2025

this PR replace #819

@1yam 1yam force-pushed the 1yam-non-io-blocking branch from a869f13 to feb0dce Compare July 11, 2025 12:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant