AWS Multi-Agent Orchestrator: A Guide With Examples

AWS Multi-Agent Coordinator: Building a Flexible Framework for Complex AI Applications

New generative AI technologies emerge every week, and the AWS Multi-Agent Coordinator is a recent framework for managing multiple AI agents. Whether you are building a customer support system or a dedicated multi-agent application, it is worth considering this framework.

This tutorial will explain the uniqueness of AWS MultiAgent Coordinator, provide step-by-step guidance to set up the environment, and develop a demonstration project to actually experience the framework.

What is AWS Multi-Agent Coordinator?

The AWS MultiAgent Coordinator is a flexible and powerful framework designed to manage AI agents and facilitate complex multi-round conversations. Its prebuilt components are quickly developed and deployed so you can focus on your own applications without rebuilding them from scratch.

AWS Multi-Agent Coordinator provides the following features:

• Intelligent routing: Dynamically identify the best agent for each query.
• Streaming and non-streaming response: Supports two agent response formats.
• Context Management: Manage coherence and history in multiple rounds of conversations.
• Scalability: Create a new agent or modify an existing agent.
• General deployment: Can run on AWS Lambda, on-premises environments, and cloud platforms.

This framework supports Python and TypeScript.

Overview of the working principle of multi-agent coordinator. (Source)

The above image shows that the classifier considers available agents, user prompts, and previous conversation history to select the agent that is best suited for user input. The agent then processes the request. The workflow is simple and effective.

Set up AWS multi-agent coordinator

To quickly set up the environment, you can follow the instructions in the documentation.

First, create a new folder and a new Python environment to install the required libraries.

<code>mkdir test_multi_agent_orchestrator
cd test_multi_agent_orchestrator
python -m venv venv
source venv/bin/activate  # 在Windows上使用venv\Scripts\activate</code>

After activate the new virtual environment, install the library

<code>pip install multi-agent-orchestrator</code>

Next, you need to configure your AWS account. If you don't have an AWS account, sign up for a free account to use the free tier. After registering, download the AWS CLI.

AWS CLI also requires configuration. For detailed instructions, follow the steps in Setting up the AWS CLI, but you can take a easier approach using the command aws configure and provide the AWS access key ID and secret access key. You can obtain these keys after creating a new user in the dashboard.

Access key provided when creating a new user.

When you are ready to access the key, run aws configure and provide the key, select the region name that is closest to you (the full list is provided here), and set the default output format to json.

If your CLI is configured correctly, running the command aws sts get-caller-identity should display your AWS account ID, user ID, and ARN.

Now that we have the AWS CLI ready, we need to configure AWS Bedrock to access the required LLM. Amazon Bedrock is a service that allows you to test and invoke underlying models (such as Llama 3.2 or Claude 3.5 Sonnet) through the API. The multi-agent coordinator uses this service to call two models by default:

1. Claude 3.5 Sonnet as classifier
2. Claude 3 Haiku as an agent

Of course, these models can be changed, but let's continue with the default selection.

To access both models, go to Amazon Bedrock > Model Access and select Modify Model Access. Select both models (and others you like) and fill in any required information. This part is as follows:

After completing the request, the model will be available within 1-2 minutes. After granting access to the requested model, you should see "Access granted" ahead of it.

Note: You may need to assign policies to created users. If you have problems in the next section of the article (test your settings), you can test it out. If so, check this page. All in all, you need to grant defined users access to Amazon BedrockFullAccess.

Test your settings

To check that all previous steps have been set correctly, use the following code:

<code>mkdir test_multi_agent_orchestrator
cd test_multi_agent_orchestrator
python -m venv venv
source venv/bin/activate  # 在Windows上使用venv\Scripts\activate</code>

If you can prompt and receive an answer, everything works fine.

Demo project using AWS multi-agent coordinator

The AWS MultiAgent Coordinator Repository provides several TypeScript and Python sample projects. We will now write a simplified Python application that contains two agents: Python developer agents and ML expert agents.

We will also use Chainlit (an open source Python package) to implement a simple UI for the application. First, install the necessary libraries:

<code>pip install multi-agent-orchestrator</code>

We use the following code as our demo application, but let's explain it first:

1. We first import the necessary libraries.
2. We use "anthropic.claude-3-haiku-20240307-v1:0" as the model of our classifier. When a new user input arrives, this classifier will select the agent to use.
3. We pass the created classifier to the MultiAgentOrchestrator and define some configuration.
4. BedrockLLMAgent class is used to create our agent. Provide a name and description for each agent. For agents, you can choose any accessible model, and you can even run them locally using Ollama. Setting streaming=True and taking ChainlitAgentCallbacks() as the callback will cause the agent to return a streaming response instead of a full response. Finally, we add each agent to the coordinator.
5. Set user_session and define the main parts that handle user and agent messages.

<code>mkdir test_multi_agent_orchestrator
cd test_multi_agent_orchestrator
python -m venv venv
source venv/bin/activate  # 在Windows上使用venv\Scripts\activate</code>

It's time to run the application. To do this, first run chainlit run app.py -w .. You can now test your application in a new tab that opens in your browser.

As shown in the screenshot, we now provide a UI to test our application and chat with the agent.

Please note that since the first prompt "What is the capital of France?" has nothing to do with any of our agents, the system will not provide an answer. This is crucial to keep chatting relevant and avoid spending unnecessary points when using these models. However, our machine learning expert agents play a role to give answers when prompted with related questions, thanks to the intelligent routing of the multiagent coordinator.

Conclusion

In this blog post, we introduce the latest AWS multiagent coordinator framework, highlight some of its unique features, outline the steps to set up an environment, explore the basic models provided by Amazon Bedrock, and implement a demonstration project.

At the time of writing, this framework lacks comprehensive and detailed documentation. To take advantage of other features, such as memory and tool usage, you must read the code base and view the sample projects provided.

It is wise to keep an eye on the generative AI framework to keep up with this fast-paced field. AWS MultiAgent Coordinator is a promising choice built on the infrastructure of AWS services, and its development is worthy of attention.

The above is the detailed content of AWS Multi-Agent Orchestrator: A Guide With Examples. For more information, please follow other related articles on the PHP Chinese website!