Questions about E3DLSTM and SwinLSTM

[BlackRab]

Hi, everyone. Recently, when I used the E3DLSTM and SwinLSTM, some errors occurred. I hope someone can help me solve this problem. For E3DLSTM, the error is:

Traceback (most recent call last):
  File "./pycharm-community-2022.2.2/plugins/python-ce/helpers/pydev/pydevd.py", line 1496, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "./pycharm-community-2022.2.2/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "./E3D-LSTM_test.py", line 334, in <module>
    exp = BaseExperiment(args, dataloaders=(dataloader_train, dataloader_val, dataloader_test), strategy='auto')
  File "./Downloads/OpenSTL/openstl/api/exp.py", line 36, in __init__
    self.method = method_maps[self.args.method](steps_per_epoch=len(self.data.train_loader), \
  File "./OpenSTL/openstl/methods/e3dlstm.py", line 14, in __init__
    PredRNN.__init__(self, **args)
  File "./OpenSTL/openstl/methods/predrnn.py", line 17, in __init__
    super().__init__(**args)
  File "./OpenSTL/openstl/methods/base_method.py", line 22, in __init__
    self.model = self._build_model(**args)
  File "./Downloads/OpenSTL/openstl/methods/e3dlstm.py", line 17, in _build_model
    num_hidden = [int(x) for x in self.args.num_hidden.split(',')]
  File "./miniconda3/envs/OpenSTL/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'E3DLSTM' object has no attribute 'args'

The parameters used in E3DLSTM are as follows:

custom_training_config = {
    'pre_seq_length': 12,
    'aft_seq_length': 24,
    'total_length': 36,
    'batch_size': 16,
    'val_batch_size': 16,
    'epoch': 80,
    'lr': 1e-3,
    'metrics': ['mse', 'mae'],

    'ex_name': 'custom_exp',
    'dataname': 'custom',
    'in_shape': [12, 2, 64, 64],
    'warmup_epoch': 0,
    'drop_path': 0.1,
    'sched': 'cosine'
}

custom_model_config = {
    'method': 'e3dlstm',
    'num_hidden': '128,128,128,128',
    'filter_size': 5,
    'stride': 1,
    'patch_size': 4,
    'layer_norm': 0,
}

custom_schedule_config = {
    'reverse_scheduled_sampling': 0,
    'r_sampling_step_1': 25000,
    'r_sampling_step_2': 50000,
    'r_exp_alpha': 5000,
    # scheduled sampling
    'scheduled_sampling': 1,
    'sampling_stop_iter': 50000,
    'sampling_start_value': 1.0,
    'sampling_changing_rate': 0.00002
}

For SwinLSTM, the error is as follows:

Traceback (most recent call last):
  File "./SwinLSTM_test.py", line 321, in <module>
    exp = BaseExperiment(args, dataloaders=(dataloader_train, dataloader_val, dataloader_test), strategy='auto')
  File "./Downloads/OpenSTL/openstl/api/exp.py", line 36, in __init__
    self.method = method_maps[self.args.method](steps_per_epoch=len(self.data.train_loader), \
  File "./Downloads/OpenSTL/openstl/methods/swinlstm.py", line 15, in __init__
    super().__init__(**args)
  File "./Downloads/OpenSTL/openstl/methods/base_method.py", line 22, in __init__
    self.model = self._build_model(**args)
  File "./Downloads/OpenSTL/openstl/methods/swinlstm.py", line 21, in _build_model
    return SwinLSTM_D_Model(depths_downsample, depths_upsample, num_heads, self.hparams)       
  File "./Downloads/OpenSTL/openstl/models/swinlstm_model.py", line 20, in __init__
    self.Downsample = DownSample(img_size=H, patch_size=configs.patch_size, in_chans=C,
  File "./Downloads/OpenSTL/openstl/modules/swinlstm_modules.py", line 254, in __init__
    downsample = PatchMerging(input_resolution=(patches_resolution[0] // (2 ** i_layer),
TypeError: __init__() got an unexpected keyword argument 'input_resolution'

and the parameters for SwinLSTM are as follows:

custom_training_config = {
    'pre_seq_length': 12,
    'aft_seq_length': 24,
    'total_length': 36,
    'batch_size': 16,
    'val_batch_size': 16,
    'epoch': 80,
    'lr': 1e-3,
    'metrics': ['mse', 'mae', 'rmse'],
    'sched': 'onecycle',

    'ex_name': 'custom_exp',
    'dataname': 'custom',
    'in_shape': [12, 2, 64, 64],
    'warmup_epoch': 0,
    'drop_path': 0.1
}

custom_model_config = {
    'method': 'SwinLSTM_D',
    'depths_downsample': '2,6',
    'depths_upsample': '6,2',
    'num_heads': '4,8',
    'patch_size': 2,
    'window_size': 4,
    'embed_dim': 128
}

The training code for the model is as follows:

args = create_parser().parse_args([])
config = args.__dict__
config.update(custom_training_config)
config.update(custom_model_config)
default_values = default_parser()
for attribute in default_values.keys():
    if config[attribute] is None:
        config[attribute] = default_values[attribute]
exp = BaseExperiment(args, dataloaders=(dataloader_train, dataloader_val, dataloader_test), strategy='auto')
print('>'*35 + ' training ' + '<'*35)
exp.train()

The version of OpenSTL is 1.0.0. May I ask how to solve the above error? Thank you for your help!

[osherr1996]

HI,
I encountered the same issue with SwinLSTM, which requires an older timm version. To fix it, I downgraded timm to 0.6.11 and modified the imports in all files inside the OpenSTL folder:

Changed imports to ensure compatibility:

from timm.layers → from timm.models.layers
timm.models._efficientnet_blocks → timm.models.efficientnet_blocks
timm.models._resnet → timm.models.resnet
timm.models._helpers → timm.models.helpers

works for me (not official).
Not sure if this is the most elegant solution, but it resolves the issue for now.

[BlackRab]

Hi, @osherr1996

Thank you for getting back to me! I checked the version of timm, it is 0.9.2. I remember initially having problems with incompatible timm versions when I tested if openstl was installed successfully.

If I need to use SwinLST later, I'll try the method you provided. Thanks again!