Sometimes Actions of the agent does not take effect to envs!

[Yonggie]

🐛 Bug

On the tag home-robot-ovmm-challenge-2024 branch, the actions of any agent does not take effect to the env no matter what kind of agent I choose.

The visualization stays the same as follows:

My agent keeps saying turn right but the observation from env stays the same:

I just checked my earlier challenge project at nips'23, it turns to be just the same, which was not...

I'm not sure this is a package trouble or my machine trouble...

I wonder what kind of hardware problem would cause such weird issue>>? GPU/CPU out of memory?

[Yonggie]

I highly suspect that this could be a GPU memory insufficient problem! It gets back to normal the other day! But how come no warnings or exceptions coming out?

[GuanxingLu]

I have also encountered a similar problem...

[yvsriram]

The agent wouldn't move if the action takes it outside the navmesh. We can print warnings here if it tries to move outside the navmesh.

But @Yonggie it should be able to rotate in the screenshot you shared. I doubt if it is a hardware issue. Can you check if you are using the right habitat-lab commit?

@GuanxingLu were you running the default agent too?

[Yonggie]

I think it's about the hardware, cus it got back to normal yesterday with the same device and same codes. But I wonder how come there's no warnings or exceptions even when encoutered insufficient hardware requirements?

Fyi, this is the habitat-lab version & commit:
version:

habitat-baselines             0.2.5              /share/home-robot/src/third_party/habitat-lab/habitat-baselines
habitat-lab                   0.2.5              /share/home-robot/src/third_party/habitat-lab/habitat-lab

git log:

commit b727ca9f7123101aaedb737ca9ccc1b153525dd9 (HEAD)
Author: Sriram Yenamandra <[email protected]>
Date:   Thu Nov 2 12:27:23 2023 -0400

    fix camera_pose update it to be relative to agent start

commit d965bd0264f857845d3d0167806ff17b994c6441
Author: Sriram Yenamandra <[email protected]>
:...skipping...
commit b727ca9f7123101aaedb737ca9ccc1b153525dd9 (HEAD)
Author: Sriram Yenamandra <[email protected]>
Date:   Thu Nov 2 12:27:23 2023 -0400

    fix camera_pose update it to be relative to agent start

commit d965bd0264f857845d3d0167806ff17b994c6441
Author: Sriram Yenamandra <[email protected]>
Date:   Sun Sep 24 15:03:24 2023 -0400

    remove debug statements

commit 10a6e87380ef6eb36e6b2f04af62b74df69a2a7e
Author: Sriram Yenamandra <[email protected]>
Date:   Sat Sep 23 00:59:21 2023 -0400

    fix: saved ckpts no longer have actor_critic prefix

commit 6af72390cdcae262d63ab8755209b8ed8d2d9272
Merge: aef3de86 17ec6b3c
Author: Sriram Yenamandra <[email protected]>
Date:   Fri Sep 22 16:09:16 2023 -0400