Estimate memory usage of TensorFlow model

[ndeepesh]

Question

Is there a way to estimate TF Memory usage before serving the model. We would like to take actions based on this information like if we want to undeploy some models to meet memory usage limit. Also, can we get this information for before and after model warmup? Since Model Warmup will lead to increased memory usage (based on the fact that tf serving only initializes internal data structures lazily)

[singhniraj08]

@ndeepesh,

Could you try the memory profiling tool to see if it helps?
You could also take a look at get_memory_info, which provides the current and peak memory that TensorFlow is actually using. If you are more interested in monitoring GPU usage, we already have a ongoing feature request #1407 in progress. Requesting you to close this issue and follow and +1 similar thread for updates.
Thank you!

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No