move RemoteAssetGraph.from_external_repository to cached property on ExternalRepository #20468
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This stack of pull requests is managed by Graphite. Learn more about stacking. Join @alangenfeld and the rest of your teammates on |
alangenfeld
force-pushed
the
al/03-13-move_RemoteAssetGraph.from_external_repository_to_cached_property_on_ExternalRepository
branch
from
b94c965 to
e1b34f7
Compare
alangenfeld
force-pushed
the
al/03-13-move_RemoteAssetGraph.from_external_repository_to_cached_property_on_ExternalRepository
branch
2 times, most recently
from
7e0f9e8 to
7a446a9
Compare
schrockn
approved these changes
Mar 14, 2024
alangenfeld
force-pushed
the
al/03-13-move_RemoteAssetGraph.from_external_repository_to_cached_property_on_ExternalRepository
branch
from
7a446a9 to
28c758d
Compare
…ExternalRepository
alangenfeld
force-pushed
the
al/03-13-move_RemoteAssetGraph.from_external_repository_to_cached_property_on_ExternalRepository
branch
from
28c758d to
a9d5a2d
Compare
alangenfeld
deleted the
al/03-13-move_RemoteAssetGraph.from_external_repository_to_cached_property_on_ExternalRepository
branch
alangenfeld
added a commit
that referenced
this pull request
Mar 14, 2024
…ExternalRepository (#20468) `RemoteAssetGraph` is a relatively heavy object. Given a large asset graph, if any code ends up creating copies on the order of number of assets, it can be very expensive. We already do a good job of memoizing `ExternalRepository` via `CodeLocation` via `WorkspaceRequestContext` via `WorkspaceProcessContext`, so caching these `RemoteAssetGraph` on the `ExternalRepository` that they are a pure function of should avoid excess computation and memory until the `WorkspaceProcessContext` refreshes its snapshots on code server update. ## How I Tested These Changes existing coverage a `memray` run of a script doing per asset `RemoteAssetGraph` showed drastic reduction in memory
PedramNavid
pushed a commit
that referenced
this pull request
Mar 28, 2024
…ExternalRepository (#20468) `RemoteAssetGraph` is a relatively heavy object. Given a large asset graph, if any code ends up creating copies on the order of number of assets, it can be very expensive. We already do a good job of memoizing `ExternalRepository` via `CodeLocation` via `WorkspaceRequestContext` via `WorkspaceProcessContext`, so caching these `RemoteAssetGraph` on the `ExternalRepository` that they are a pure function of should avoid excess computation and memory until the `WorkspaceProcessContext` refreshes its snapshots on code server update. ## How I Tested These Changes existing coverage a `memray` run of a script doing per asset `RemoteAssetGraph` showed drastic reduction in memory
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
RemoteAssetGraphis a relatively heavy object. Given a large asset graph, if any code ends up creating copies on the order of number of assets, it can be very expensive.We already do a good job of memoizing
ExternalRepositoryviaCodeLocationviaWorkspaceRequestContextviaWorkspaceProcessContext, so caching theseRemoteAssetGraphon theExternalRepositorythat they are a pure function of should avoid excess computation and memory until theWorkspaceProcessContextrefreshes its snapshots on code server update.How I Tested These Changes
existing coverage
a
memrayrun of a script doing per assetRemoteAssetGraphshowed drastic reduction in memory