[Bug]: AgentBuilder doesn't build agent based on the specified agent_path

[rjambrecic]

Describe the bug

AgentBuilder doesn't use agent_path if it is specified in the agent_configs, it just creates a default AssistantAgent

Steps to reproduce

from autogen.agentchat.contrib.captainagent.agent_builder import AgentBuilder
from autogen.agents.experimental import WebSurferAgent


agent_builder = AgentBuilder(
    # config_file_or_env=OAI_CONFIG_LIST,
    # config_file_location=KEY_LOC,
    builder_model_tags=["gpt-4o"],
    agent_model_tags=["gpt-4o"],
)

conf = {
    "building_task": "Generate one WebSurferAgent for scraping wikipedia",
    "agent_configs": [
        {
            "name": "WebSurferAgent",
            "model": ["gpt-4o"],
            "description": "A helpful assistant with access to a web browser. Ask them to perform web searches, open pages, navigate to Wikipedia, answer questions from pages, and or generate summaries.",
            "system_message": "",
            "agent_path": "autogen/agents/experimental/websurfer/websurfer/WebSurferAgent",
            "web_tool_kwargs": {
                "agent_kwargs": {
                    "max_steps": 100
                }
            }
        }
    ],
    "coding": True,
    "default_llm_config": {
        "temperature": 0
    },
    "code_execution_config": {
        "work_dir": ".",
        "use_docker": False,
        "timeout": 60,
        "last_n_messages": 2
    },
}

agents, _ = agent_builder.build(**conf)

websurfer_in_agents = False
for agent in agents:
    if isinstance(agent, WebSurferAgent):
        websurfer_in_agents = True
        break
    
assert websurfer_in_agents, "WebSurferAgent not found in agents"

==> Generating agents...
['WebScraping_Expert', 'Wikipedia_Expert', 'Python_Expert', 'Shell_Expert', 'DataExtraction_Expert'] are generated.
==> Generating system message...
Preparing system message for WebScraping_Expert
Preparing system message for Wikipedia_Expert
Preparing system message for Python_Expert
Preparing system message for Shell_Expert
Preparing system message for DataExtraction_Expert
==> Generating description...
Preparing description for WebScraping_Expert
Preparing description for Wikipedia_Expert
Preparing description for Python_Expert
Preparing description for Shell_Expert
Preparing description for DataExtraction_Expert
==> Creating agents...
Creating agent WebScraping_Expert...
Creating agent Wikipedia_Expert...
Creating agent Python_Expert...
Creating agent Shell_Expert...
Creating agent DataExtraction_Expert...
Adding user console proxy...
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[2], [line 47](vscode-notebook-cell:?execution_count=2&line=47)
     [45](vscode-notebook-cell:?execution_count=2&line=45)         websurfer_in_agents = True
     [46](vscode-notebook-cell:?execution_count=2&line=46)         break
---> [47](vscode-notebook-cell:?execution_count=2&line=47) assert websurfer_in_agents, "WebSurferAgent not found in agents"

AssertionError: WebSurferAgent not found in agents

Model Used

No response

Expected Behavior

No response

Screenshots and logs

No response

Additional Information

No response