Skip to content

V0.3.0-MMNIST-Weights
Compare
Choose a tag to compare
@Lupin1998 Lupin1998 released this 18 Jun 22:24
mmnist-weights
f09ce01

We provide benchmark results on the popular Moving MNIST dataset using $10\rightarrow 10$ frames prediction setting following PredRNN. Metrics (MSE, MAE, SSIM, pSNR) of the best models are reported in three trials. Parameters (M), FLOPs (G), and V100 inference FPS (s) are also reported for all methods. All methods are trained by Adam optimizer with Onecycle scheduler and single GPU.

  • For a fair comparison of different methods, we provide config files in configs/mmnist.
  • We also benchmark popular Metaformer architectures on SimVP with training times of 200-epoch and 2000-epoch. We provide config files in configs/mmnist/simvp.

STL Benchmarks on MMNIST

Method Setting Params FLOPs FPS MSE MAE SSIM PSNR Download
ConvLSTM-S 200 epoch 15.0M 56.8G 113 29.80 90.64 0.9288 22.10 model | log
ConvLSTM-L 200 epoch 33.8M 127.0G 50 27.78 86.14 0.9343 22.44 model | log
PredNet 200 epoch 12.5M 8.6G 659 161.38 201.16 0.7783 14.33 model | log
PhyDNet 200 epoch 3.1M 15.3G 182 28.19 78.64 0.9374 22.62 model | log
PredRNN 200 epoch 23.8M 116.0G 54 23.97 72.82 0.9462 23.28 model | log
PredRNN++ 200 epoch 38.6M 171.7G 38 22.06 69.58 0.9509 23.65 model | log
MIM 200 epoch 38.0M 179.2G 37 22.55 69.97 0.9498 23.56 model | log
MAU 200 epoch 4.5M 17.8G 201 26.86 78.22 0.9398 22.76 model | log
E3D-LSTM 200 epoch 51.0M 298.9G 18 35.97 78.28 0.9320 21.11 model | log
CrevNet 200 epoch 5.0M 270.7G 10 30.15 86.28 0.9350 model | log
PredRNN.V2 200 epoch 23.9M 116.6G 52 24.13 73.73 0.9453 23.21 model | log
DMVFN 200 epoch 3.5M 0.2G 1145 123.67 179.96 0.8140 16.15 model | log
SimVP+IncepU 200 epoch 58.0M 19.4G 209 32.15 89.05 0.9268 37.97 model | log
SimVP+gSTA-S 200 epoch 46.8M 16.5G 282 26.69 77.19 0.9402 38.35 model | log
TAU 200 epoch 44.7M 16.0G 283 24.60 71.93 0.9454 23.19 model | log
ConvLSTM-S 2000 epoch 15.0M 56.8G 113 22.41 73.07 0.9480 23.54 model | log
PredNet 2000 epoch 12.5M 8.6G 659 31.85 90.01 0.9273 21.85 model | log
PhyDNet 2000 epoch 3.1M 15.3G 182 20.35 61.47 0.9559 24.21 model | log
PredRNN 2000 epoch 23.8M 116.0G 54 26.43 77.52 0.9411 22.90 model | log
PredRNN++ 2000 epoch 38.6M 171.7G 38 14.07 48.91 0.9698 26.37 model | log
MIM 2000 epoch 38.0M 179.2G 37 14.73 52.31 0.9678 25.99 model | log
MAU 2000 epoch 4.5M 17.8G 201 22.25 67.96 0.9511 23.68 model | log
E3D-LSTM 2000 epoch 51.0M 298.9G 18 24.07 77.49 0.9436 23.19 model | log
PredRNN.V2 2000 epoch 23.9M 116.6G 52 17.26 57.22 0.9624 25.01 model | log
SimVP+IncepU 2000 epoch 58.0M 19.4G 209 21.15 64.15 0.9536 23.99 model | log
SimVP+gSTA-S 2000 epoch 46.8M 16.5G 282 15.05 49.80 0.9675 25.97 model | log
TAU 2000 epoch 44.7M 16.0G 283 15.69 51.46 0.9661 25.71 model | log

Benchmark of MetaFormers Based on SimVP (MetaVP)

MetaVP Setting Params FLOPs FPS MSE MAE SSIM PSNR Download
IncepU (SimVPv1) 200 epoch 58.0M 19.4G 209 32.15 89.05 0.9268 21.84 model | log
gSTA (SimVPv2) 200 epoch 46.8M 16.5G 282 26.69 77.19 0.9402 22.78 model | log
ViT 200 epoch 46.1M 16.9G 290 35.15 95.87 0.9139 21.67 model | log
Swin Transformer 200 epoch 46.1M 16.4G 294 29.70 84.05 0.9331 22.22 model | log
Uniformer 200 epoch 44.8M 16.5G 296 30.38 85.87 0.9308 22.13 model | log
MLP-Mixer 200 epoch 38.2M 14.7G 334 29.52 83.36 0.9338 22.22 model | log
ConvMixer 200 epoch 3.9M 5.5G 658 32.09 88.93 0.9259 21.93 model | log
Poolformer 200 epoch 37.1M 14.1G 341 31.79 88.48 0.9271 22.03 model | log
ConvNeXt 200 epoch 37.3M 14.1G 344 26.94 77.23 0.9397 22.74 model | log
VAN 200 epoch 44.5M 16.0G 288 26.10 76.11 0.9417 22.89 model | log
HorNet 200 epoch 45.7M 16.3G 287 29.64 83.26 0.9331 22.26 model | log
MogaNet 200 epoch 46.8M 16.5G 255 25.57 75.19 0.9429 22.99 model | log
TAU 200 epoch 44.7M 16.0G 283 24.60 71.93 0.9454 23.19 model | log
IncepU (SimVPv1) 2000 epoch 58.0M 19.4G 209 21.15 64.15 0.9536 23.99 model | log
gSTA (SimVPv2) 2000 epoch 46.8M 16.5G 282 15.05 49.80 0.9675 25.97 model | log
ViT 2000 epoch 46.1M 16.9.G 290 19.74 61.65 0.9539 24.59 model | log
Swin Transformer 2000 epoch 46.1M 16.4G 294 19.11 59.84 0.9584 24.53 model | log
Uniformer 2000 epoch 44.8M 16.5G 296 18.01 57.52 0.9609 24.92 model | log
MLP-Mixer 2000 epoch 38.2M 14.7G 334 18.85 59.86 0.9589 24.58 model | log
ConvMixer 2000 epoch 3.9M 5.5G 658 22.30 67.37 0.9507 23.73 model | log
Poolformer 2000 epoch 37.1M 14.1G 341 20.96 64.31 0.9539 24.15 model | log
ConvNeXt 2000 epoch 37.3M 14.1G 344 17.58 55.76 0.9617 25.06 model | log
VAN 2000 epoch 44.5M 16.0G 288 16.21 53.57 0.9646 25.49 model | log
HorNet 2000 epoch 45.7M 16.3G 287 17.40 55.70 0.9624 25.14 model | log
MogaNet 2000 epoch 46.8M 16.5G 255 15.67 51.84 0.9661 25.70 model | log
TAU 2000 epoch 44.7M 16.0G 283 15.69 51.46 0.9661 25.71 model | log
Assets 96
Loading