How can I create dynamic NeRF scenes?

[JianxiangL]

No description provided.

[maximeraafat]

Hi @JianxiangL, this question goes beyond the scope of this repository, but I’m happy to provide some guidance.

Dynamic NeRFs typically require capturing multiple views of an object at each time step, for example, 50 images per frame in your dynamic scene. To create such a dataset, one approach is to animate the scene so it remains static for 50 frames, then update the scene and repeat the process for the next 50 frames, etc. Alternatively, you could modify the code to freeze the animation at each frame while capturing the required views, and this for each time step. If your dynamic NeRF implementation supports continuous animations (essentially requiring a single view per frame while moving forward in time), you can use BlenderNeRF naïvely as is. I hope this helps clarify things!

[JianxiangL]

Thanks！！！！！！！！！！！！！！！！！！！！！！！

[HaoqinHong]

I think D-NeRF Dataset also use his Projects.

[maximeraafat]

Glad I could help

[JianxiangL]

Hi @JianxiangL, this question goes beyond the scope of this repository, but I’m happy to provide some guidance.

Dynamic NeRFs typically require capturing multiple views of an object at each time step, for example, 50 images per frame in your dynamic scene. To create such a dataset, one approach is to animate the scene so it remains static for 50 frames, then update the scene and repeat the process for the next 50 frames, etc. Alternatively, you could modify the code to freeze the animation at each frame while capturing the required views, and this for each time step. If your dynamic NeRF implementation supports continuous animations (essentially requiring a single view per frame while moving forward in time), you can use BlenderNeRF naïvely as is. I hope this helps clarify things!

Hello, may I ask if it's possible to read the timestamp information for each image? Where should I modify the code?
the output file "transforms_train.json" like:
{ "file_path": "./train/r_1", "rotation": 0.12566370614359174, "time": 0.020202020202020204, "transform_matrix": [ [ 0.40335917472839355, -0.6805809736251831, 0.6116460561752319, 2.4656240940093994 ], [ 0.9150418043136597, 0.30000659823417664, -0.2696194350719452, -1.0868706703186035 ], [ 0.0, 0.6684350371360779, 0.7437704205513, 2.998234272003174 ], [ 0.0, 0.0, 0.0, 1.0 ] ] },

[maximeraafat]

Hi @JianxiangL, you will probably want to make adjustments to the frame_data dictionary within the get_camera_extrinsics function, located in the blender_nerf_operator.py file :)

[JianxiangL]

Hi @JianxiangL, you will probably want to make adjustments to the frame_data dictionary within the get_camera_extrinsics function, located in the blender_nerf_operator.py file :)

Thank you very very very much for your help!!!!!! I've solved the timestamp issue. However, I’ve encountered a small problem. When I render the depth map using 3D Gaussian splatting, I notice a diagonal line appears in the depth map. I'm quite puzzled by this. Could it be an issue with my dataset? However, this line does not appear in the ground truth image. I'm really confused about this, and I would really appreciate it if you could take a look and help me out!

[maximeraafat]

Hi @JianxiangL, I am not entirely sure what might be causing this. I would recommend checking the .blend file to see if anything unusual is happening there or reviewing the Gaussian splatting implementation you are using. Since the line does not appear in the ground truth image, it may not be dataset-related. ChatGPT might also provide more detailed guidance on this.