You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Create a single process version of sharding_single_rank_test (#3085)
Summary:
Pull Request resolved: #3085
Extracts the main logic of `sharding_single_rank_test` from the multiprocess context into a new function called `sharding_single_rank_test_single_process`. It now requires `pg` and `device` as extra input to replace the multiprocess context.
### Context
In certain tests, we would like to compare the unsharded (global) vs sharded (local) model in a single process to make the test more efficient and easier to debug. For example, we would like to add a new test in [`test_model_parallel_nccl_ssd_single_gpu.py`](https://www.internalfb.com/code/fbsource/[5f477259031a]/fbcode/torchrec/distributed/tests/test_model_parallel_nccl_ssd_single_gpu.py) to test SSD TBE with VBE enabled where only 1 GPU is actually needed to verify the functionality.
Reviewed By: TroyGarden
Differential Revision: D76455105
fbshipit-source-id: 29f4d682c5d95408b2ec999f032197555f5b41cf
0 commit comments