torch.Size does not match expected state_dim

[xjuzyw0319]

Describe the issue
I am going to improve the replay_buffer in your source code and add the PrioritizedReplayBuffer model. I am currently facing a dimension mismatch problem. The torch.Size dimension printed each time I run the code is inconsistent. I originally analyzed that it was caused by the random start and target points. The following figure 1 is the result of my printing, and the other pictures are some of my modifications to the source code. Please give me some suggestions, thank you very much.

Screenshots
图一

Additional context
Add any other context about the problem here.

[reiniscimurs]

Hi,
At some point your code has a mistake in how it returns a state from your replay buffer. Again, it will not be the start and goal points, but rather some return from the buffer or else you would never be able to add any experiences to it.

However, you have not shared all the changes you have made and without that I cannot help you here as it is not possible for me to guess how you have implemented sampling.

Good luck

[xjuzyw0319]

I rewrote your experience pool (as shown in Figure 1), but during the training process, I found that after a few rounds of training, the agent kept spinning in place（as shown in Figure 2） and the success rate was always 0 (as shown in Figure 3). Could you give me some suggestions?

图1

图2
图3

[reiniscimurs]

So is the size mismatch not an issue anymore?

In any case, I do not see where this memory class is called and the full implementation pipeline of the buffer. I would double check that the state that is returned from buffer is the same as the input into it.

Rotation on the spot either happens early in the training process or when the network is not converging. For that you can see any issues tagged with the 'convergence' label. If your state used for training is wrong or the sampling is ineffective, the network would not learn much so double check the implementation for bugs.