ROS Development Experiments

[robbiebbaker]

Please complete all items in this issue

Prerequisites:

• Download Ubuntu 22.04 Desktop
• Download ROS2 Humble Hawksbill
• Download Realsense Wrapper
• Download Tensorflow

Background Knowledge Required:

• ROS2 Knowledge
• Tensorflow Background

What We Want to Find Out

1. Python vs C++ ROS Node Performance and Output Lagtime
2. Best Method of Building Nodes for Neural Networks
3. Best Method of Building Camera Nodes and Transferring Frames

Python vs C++ ROS Node Performance

Objective:
The main objective of this experiment is to measure and analyze the performance differences between ROS2 Python and ROS2 C++ nodes when executing different neural networks. This information will help the team make informed decisions when choosing the language for their ROS2 projects involving neural networks.

Tasks:

• Create YOLO, F-RCNN, and MobileNet Neural Networks
• Implement ROS2 nodes for Python and C++ to run the specified neural networks (YOLO, F-RCNN, and MobileNet)
• Create a test environment with representative hardware and software configurations to ensure realistic performance testing
• Collect performance metrics such as execution time, CPU and memory usage, latency, stability over time and network bandwidth
• Analyze gathered data and create performance comparison report
• Document and share the results, insights and recommendations with the team

Best Method of Building Nodes for Neural Networks

Objective:
The main objective of this issue is to foster a collaborative discussion on how to structure ROS2 nodes for neural networks in a way that achieves the following goals:

• Low-Latency: Minimize the time it takes for neural network inference to produce results, which is vital for real-time robotics applications.
• Scalability: Ensure that the ROS2 node can effectively scale with increased computational demands, such as larger or more complex neural networks.
• Adaptability: Make the node easily adaptable to different neural network architectures and configurations without extensive code modifications.

Tasks:

• Node Architecture: Figure out and test the overall structure of a ROS2 node for NN. Should it be monolithic, modular, or something else? What design patterns are most suitable for low-latency performance?
• Interface Design: How should the node interface with ROS2, taking into consideration message passing, services, and actions?
• Hardware Acceleration: Report on items we could purchase or use to improve performance and by how much expected
• Configuration Management: Explore and test strategies for managing neural network configurations and adapting to different networks with minimal code changes.
• Data Preprocessing: How should data preprocessing be handled to optimize NN input and output?
• Real-time Constraints: What real time constraints and conditions can be implemented to ensure better results? How can we ensure low-latency performance within these constraints?
• Benchmarking and Profiling: Benchmark and profile the ROS2 nodes to measure and optimize latency
• Analyze gathered data and create performance comparison report
• Document and share the results, insights and recommendations with the team

Best Method of Building Camera Nodes and Transferring Frames

Objective:
The primary objective of this issue is to encourage a collaborative discussion on how to structure ROS2 nodes for RGB-D cameras, with a focus on achieving the following goals:

• Low-Latency: Minimize the time it takes to capture, process, and publish RGB-D data, ensuring real-time responsiveness for robotics applications.
• Scalability: Design nodes that can efficiently handle varying resolutions, frame rates, and camera models, and adapt to the computational demands of different applications.
• Adaptability: Create a node structure that can easily accommodate different RGB-D camera models and configurations without extensive code modifications.

Tasks:

• Node Architecture: Discuss the overall structure of ROS2 nodes for RGB-D cameras. Should it be modular, configurable, or use a specific design pattern? What strategies can be employed to reduce latency in data processing?
• Camera Interface: Explore how the node should interface with RGB and RGB-D cameras, considering ROS2 message types, configuration options, and synchronization mechanisms for RGB and depth data.
• Real-time Constraints: Address the real-time constraints involved in robotics applications and how the node design can ensure low-latency performance within these constraints.
• Configuration Management: Figure out methods for managing camera configurations, such as resolution, frame rate, and synchronization, while allowing easy adaptation to different camera models.
• Data Preprocessing: Figure out best practices for preprocessing RGB-D data to optimize the information received from the camera and reduce latency.
• Synchronization Techniques: Figure out methods for synchronizing RGB and depth data efficiently, especially when dealing with cameras that may have slight timing discrepancies.
• Benchmarking and Profiling: Benchmark and profile the Camera ROS2 nodes to measure and optimize latency
• Analyze gathered data and create performance comparison report
• Document and share the results, insights and recommendations with the team