Skip to content

feat: various enhancements #508

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Apr 8, 2025
Merged

feat: various enhancements #508

merged 16 commits into from
Apr 8, 2025

Conversation

maciejmajek
Copy link
Member

@maciejmajek maciejmajek commented Apr 4, 2025

Purpose

This PR enhances the RAI framework with improved tool abstractions and action handling, legacy tool removal, while also modernizing the ROSBotXL demo.

Proposed Changes

Major Changes

  1. Demo Simplification

    • Removed the robot identity setup step, due to missing feature
    • Updated documentation to reflect the new workflow
  2. Code Refactoring

    • Migrated from conversational agent to ReAct agent architecture
    • Introduced new simple tools (GetROS2ImageConfiguredTool, GetROS2TransformConfiguredTool)
    • Reorganized ROS2 tools into generic and simple categories
    • Introduced system prompt integration in the ReAct agent
  3. Tool Improvements

    • Fixed CancelROS2ActionTool
    • Added new configured tools
    • Improved tool organization and modularity
  4. Documentation Updates

    • Updated demo instructions to reflect the new workflow
    • Added clearer examples of interaction with the robot

Testing

  1. Running the demo
  2. Tests CI

Summary by CodeRabbit

  • Documentation

    • Updated demo installation and running instructions with clearer steps and revised commands.
    • Added a new guide detailing the integration and configuration of interactive tools.
  • New Features

    • Launched a dynamic interface with configurable titles and initial messages for improved user interaction.
    • Introduced enhanced navigation and imaging tools for a more interactive experience.
  • Refactoring & Removals

    • Removed deprecated components and restructured modules for increased clarity and performance.
  • Tests

    • Expanded test coverage including new checks for action cancellation and adjusted timing for reliability.

@maciejmajek maciejmajek requested a review from boczekbartek April 7, 2025 06:41
]
```

### Using Tools in RAI Agent (Distributed Setup)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would elaborate more on what distributed setup means

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added todo

@maciejmajek
Copy link
Member Author

@coderabbitai full review

This comment was marked as outdated.

coderabbitai[bot]

This comment was marked as outdated.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add some hints about the models. I think to make this demo to work, the requirement is that complex_model has to be set to VLM, that supports tool calling.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

st.set_page_config(
page_title="RAI ROSBotXL Demo",
page_icon=":robot:",
)


@st.cache_resource
def initialize_agent():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about using ROS2Context?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't be done in this context as ROS2Context would also shutdown the rclpy on the return of initialize_agent

Copy link
Member

@boczekbartek boczekbartek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maciejmajek
Tested again with prompts from the docs and they worked fine.
One thing that I've noticed is that recursion_limit should be increased, because it's too low. After it's fixed I'm fine with merging this PR.

langgraph.errors.GraphRecursionError: Recursion limit of 25 reached without hitting a stop condition. You can increase the limit by setting the `recursion_limit` config key. For troubleshooting, visit:

@maciejmajek
Copy link
Member Author

@boczekbartek thank you for the review. Here is a RFC regarding imports

@maciejmajek maciejmajek merged commit 557e6ca into development Apr 8, 2025
5 checks passed
@maciejmajek maciejmajek deleted the refactor/demos branch April 8, 2025 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants