A modular repository demonstrating how to build an agentic platform including labs and a deployable sample "agentic platform" to demonstrate how to operationalize agents from multiple open source frameworks as well as Bedrock agents.
This sample is active
The sample platform is built on EKS and uses a variety of AWS services to deploy 10+ agentic systems demonstrating how everything works together. It's instrumented with telemetry and demonstrates operational considerations when deploying agents. Code is written in an abstracted/modular way to make it easy to switch the underlying infrastructure components to suit your needs.
Each agent runs as a FastAPI server sharing a core package which contains types and client abstractions (like LLM API provider). APIs are authenticated using Cognito through a middleware layer that's simple to swap out for your own IDP.
The agents themselves do not have IAM roles attached to them. Instead they connect to AWS resources through microservices like the LLM Gateway, Memory Gateway, and Retrieval Gateway (which do have IAM roles through IRSA). Those requests are authenticated by passing JWT tokens between each request. Each pod running in EKS is authenticated using oAuth regardless of whether the request was service <> service or user <> service by validating tokens against the IDP's public cert.
Lastly, telemetry information is collected using open telemetry collectors and pushed to X-Ray for traces, Cloudwatch for metrics, and OpenSearch for logs. An agent uses the observability facade (in the common package) which is pre-configured to send to our open telemetry endpoints. The OTEL collectors then push the telemetry data to any endpoint that supports open telemetry protocol (OTLP). This makes it easy to switch vendors or different services. Anything that supports OTLP will work.
- Clone this repository
- Follow the deployment guide in DEPLOYMENT.md to set up the infrastructure
- Explore the hands-on labs in the
labs/directory
There are 5 progressive modules that increase in complexity, going from the basics to operational considerations when running an agent platform at scale:
- Module 1: Prompt Engineering & Evaluation
- Module 2: Common Agentic Patterns
- Module 3: Building Agentic Applications
- Module 4: Multi-Agent Systems & MCP
- Module 5: Deployment and Infrastructure
Only module 5 requires the agent platform to be deployed. See labs/README.md for detailed lab instructions.
To run labs locally:
# Install dependencies
uv sync
# Start Jupyter Lab
uv run jupyter labImportant Notice: This project deploys resources in your AWS environment using Terraform. You will incur costs for the AWS resources used.
For complete deployment instructions, see DEPLOYMENT.md. We provide both:
- Automated bootstrap (recommended): Uses CloudFormation templates to set up everything automatically
- Manual deployment: Step-by-step instructions for custom deployments
Make sure to run security scans if making changes to the code:
Suppressed Warnings: There are warnings suppressed in the codebase. Review these prior to using any code in your environment.
See CONTRIBUTING for more information.
We are open to contributions and this project is actively being worked on. Items on our roadmap include:
- Making deployment easier through a bootstrap terraform module
- Cleaning up deployments with a more structured approach to GitOps
- Adding additional labs on more advanced agent topics
- Building a test harness & eval suite that runs against the code base
- Adding more agent examples from the labs into the sample platform
- Tanner McRae
- Randy DeFauw
- James Levine
This library is licensed under the MIT-0 License. See the LICENSE file.
