Skip to content

Commit ddbe9e0

Browse files
Shuangping Liufacebook-github-bot
authored andcommitted
Create a single process version of sharding_single_rank_test (#3085)
Summary: Pull Request resolved: #3085 Extracts the main logic of `sharding_single_rank_test` from the multiprocess context into a new function called `sharding_single_rank_test_single_process`. It now requires `pg` and `device` as extra input to replace the multiprocess context. ### Context In certain tests, we would like to compare the unsharded (global) vs sharded (local) model in a single process to make the test more efficient and easier to debug. For example, we would like to add a new test in [`test_model_parallel_nccl_ssd_single_gpu.py`](https://www.internalfb.com/code/fbsource/[5f477259031a]/fbcode/torchrec/distributed/tests/test_model_parallel_nccl_ssd_single_gpu.py) to test SSD TBE with VBE enabled where only 1 GPU is actually needed to verify the functionality. Reviewed By: TroyGarden Differential Revision: D76455105 fbshipit-source-id: 29f4d682c5d95408b2ec999f032197555f5b41cf
1 parent 3b6b537 commit ddbe9e0

File tree

1 file changed

+275
-211
lines changed

1 file changed

+275
-211
lines changed

0 commit comments

Comments
 (0)