You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Resnet50 accuracy on Wormhole, when running ImageNet dataset is 0.369.
This is also visible when running a standalone demo (without dataset), from the repo:
Bisecting revealed that this is a regression, coming from this commit: 282a7b2
It is not caught with regular tests, since the PCC check logic is also broken for wormhole, and is accidentally picking up PCC targets from Grayskull, which are lower.
To Reproduce
Steps to reproduce the behavior:
Single demo run: WH_ARCH_YAML=wormhole_b0_80_arch_eth_dispatch.yaml pytest --disable-warnings models/demos/wormhole/resnet50/demo/demo.py::test_demo_sample
The first conv in Resnet50 has a config which doesn't work (self.conv1_config.act_block_h_override = 49 * 32)
Setting it to 2 * 32 restores the accuracy back to 0.76.
Other values: 7 * 32, 14 * 32 also aren't working.
Using the working ConvConfig and fixing the PCC checker here: #16896
Leaving the issue open until the conv is fixed
Describe the bug
Resnet50 accuracy on Wormhole, when running ImageNet dataset is 0.369.
This is also visible when running a standalone demo (without dataset), from the repo:
Bisecting revealed that this is a regression, coming from this commit: 282a7b2
It is not caught with regular tests, since the PCC check logic is also broken for wormhole, and is accidentally picking up PCC targets from Grayskull, which are lower.
To Reproduce
Steps to reproduce the behavior:
Single demo run:
WH_ARCH_YAML=wormhole_b0_80_arch_eth_dispatch.yaml pytest --disable-warnings models/demos/wormhole/resnet50/demo/demo.py::test_demo_sample
ImageNet dataset run:
WH_ARCH_YAML=wormhole_b0_80_arch_eth_dispatch.yaml pytest --disable-warnings models/demos/wormhole/resnet50/demo/demo.py::test_demo_imagenet
Expected behavior
Accuracy should be 0.76 on ImageNet dataset.
The text was updated successfully, but these errors were encountered: