Implemented optimized Neo4j queries and setup #593
+140
−2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hi @northdpole, could you please take a look at this PR and provide feedback? |
|
@northdpole Could you please review this? I have fixed the linting errors. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title: Improve Map Analysis Performance with Optimized Neo4j Queries
Description:
Summary: This PR implements several improvements to make the Map Analysis feature faster and more resource-efficient. It includes optimized Cypher queries, indexing for faster lookups, caching mechanisms to avoid redundant computations, and usage of APOC procedures for parallel processing.
Key Changes:
Added Indexes on Standard(name) and Framework(name) to reduce NodeByLabelScan operations.
Refined Cypher Queries using WITH and LIMIT clauses for smaller, more efficient batches.
Implemented Caching for frequently accessed paths, minimizing redundant path calculations.
Utilized APOC for timeboxed queries and parallel batch processing.
Why: Addresses Issue #587, ensuring faster response times and lower resource usage.
Testing:
Verified performance gains via PROFILE in Neo4j Browser.
Measured query execution time before and after applying indexes and caching.
Next Steps:
Further refine queries if new performance bottlenecks arise.
Update documentation as needed.
How to Test:
Pull these changes and install any required dependencies (e.g., python-dotenv, neo4j).
Start Neo4j and run setup_database.py to create nodes/indexes.
Run query_database.py and observe improved execution times using PROFILE or logs.