[moe training] add test case for shared expert in distributed tests

[danielvegamyhre]

Stacked PRs:

• ->[moe training] add test case for shared expert in distributed tests #2856

---

[moe training] add test case for shared expert in distributed tests

Summary

• After [MoE/EP] apply dim-1 FSDP sharding for routed experts and rewrite shared experts with FFN torchtitan#1561 landed in torchtitan, the hang when using quantized training techniques on MoEs is now resolved. We can add test cases for this now.

Test plan

• cd test/prototype/moe_training && ./test_everything.sh

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2856

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 45fb201 with merge base 253d65a ():

NEW FAILURES - The following jobs have failed:

• Code Analysis with Ruff / build (3.9) (gh)
  test/prototype/mx_formats/test_nvfp4_tensor.py:530:9: F821 Undefined name TORCH_VERSION_AT_LEAST_2_8``
• PR Label Check / Check PR Labels (gh)
  Process completed with exit code 1.
• Run Regression Tests / test (CPU 2.8, linux.4xlarge, torch==2.8.0 --index-url https://download.pytorch.org/whl/cpu, cpu) / linux-job (gh)
  RuntimeError: Command docker exec -t 58d36ee6a725d8a9f0073bf9a3e21f3d3d844aaa80515ecd71e9a009a05a4c3e /exec failed with exit code 2
• Run Regression Tests / test (CUDA 2.8, linux.g5.12xlarge.nvidia.gpu, torch==2.8.0, cuda, 12.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 11a891c56af59c034122b0d7437a7d82f1379c7b2aceb5ee6212f51674a08d5d /exec failed with exit code 2
• Run Regression Tests / test-nightly (CPU Nightly, linux.4xlarge, --pre torch --index-url https://download.pytorch.org/wh... / linux-job (gh)
  RuntimeError: Command docker exec -t da7dcaba79cca3cd31706663b016d22709488053d0f1351a0343286cf5c969f5 /exec failed with exit code 2
• Run Regression Tests / test-nightly (CUDA Nightly, linux.g5.12xlarge.nvidia.gpu, --pre torch --index-url https://downloa... / linux-job (gh)
  RuntimeError: Command docker exec -t 8af6f26069c5382c7376ef30fbffe88fac54bbb28ae10d1e5a71b58c91001653 /exec failed with exit code 2

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Run 1xH100 Tests / test (H100, linux.aws.h100, --pre torch torchvision torchaudio fbgemm-gpu-genai --index-url https... / linux-job (gh) (trunk failure)
  test/integration/test_integration.py::TestAutoQuant::test_autoquant_hp_float
• Run 1xL4 Tests / test (SM-89, linux.g6.4xlarge.experimental.nvidia.gpu, --pre torch --index-url https://download.p... / linux-job (gh) (trunk failure)
  test/integration/test_integration.py::TestAutoQuant::test_autoquant_hp_float

This comment was automatically generated by Dr. CI and updates every 15 minutes.